And the AIR-1 in GATE goes to…

Speaker: Chandra Sekhar Seelamantula, Professor, Department of Electrical Engineering, Indian Institute of Science

Date: 19 August 2026

YouTube link: https://youtu.be/PvI8CPakcaY

In his talk, Chandra Sekhar Seelamantula introduced GATEBench, a benchmark based on the Graduate Aptitude Test in Engineering (GATE), to evaluate the scientific reasoning, mathematical problem-solving, and visual understanding capabilities of state-of-the-art large language models. This is the first-of-its-kind effort in the context of the GATE exam.

The study assessed seven frontier models across six GATE 2025 and 2026 test papers comprising computer science, data science and artificial intelligence, electrical engineering, electronics and communication engineering, and mathematics. The evaluation compared the model performance with the official All India Rank-1 (AIR-1) scores and analysed the performance across subjects, question types (multiple choice questions, multiple select questions, and numerical answer type), and image-based versus text-based questions. Beyond simply asking whether AI can top the GATE exam, the study offered an opportunity to examine what kinds of reasoning tasks remain challenging for current AI systems, and what this tells us about the evolving capabilities of AI. The talk discussed these findings and their broader implications for AI, education, and possibly the future of engineering examinations. The findings also set the goalpost for the next-generation AI systems in the context of engineering and scientific reasoning tasks.