Deep Foundation Models

Instructor: Sriram Ganapathy, Associate Professor, Department of Electrical Engineering, Indian Institute of Science
Date: 14, 21, 28 November 2025

A seminar series on Deep Foundation Models was offered by Sriram Ganapathy. He covered the topics of transformer architectures, large language models (LLMs), and diffusion models during three lectures spread over three consequent weeks. The lectures were well attended in-person and online.

The first lecture introduced self-attention, encoder–decoder attention, in-context learning, low-rank adaptation in the context of transformer architectures, and objectives and loss functions, masked language modelling in self-supervised learning. The second lecture focused on large language models (tokenizers, pre-training and post-training, multimodal alignment, model compression, reinforcement learning for fine-tuning, proximal policy optimisation, benchmarking and evaluation of LLMs). Diffusion models (deep generative models, VAEs and GANs, forward and reverse diffusion, denoising score matching, variational lower bounds, stable diffusion) were covered in the third lecture.

These lectures would be part of the NPTEL (National Programme on Technology enhanced Learning) course that would be offered by Professors Sriram Ganapathy, Ashwini Kodipalli, and Baishali Garai from the Indian Institute of Science.

MENU