Pranshu Malviya
AI Research Scientist Intern at DRW PhD Candidate at MILA / Polytechnique Montreal
About
I build things at the intersection of optimization and continual learning. Currently an AI Research Scientist Intern at DRW in Montreal, and a PhD candidate at MILA / Polytechnique Montreal under Prof. Sarath Chandar.
My research asks how models can keep learning without forgetting, and how optimizers can navigate loss landscapes more effectively. Some highlights: Manifold Metric (CoLLAs 2025, Oral), Lookbehind-SAM (ICML 2024), Critical Momenta (TMLR 2024), and TAG (CoLLAs 2022).
Before Montreal, I completed my MS at IIT Madras working with Prof. Balaraman Ravindran and Prof. Chandar at RBCDSAI.
When not working: sketching, hiking, reading non-fiction, cricket, travelling (4/7 wonders so far).
Latest News
New preprint: CoPeP on continual pretraining for protein language models
New preprint: CoPeP on continual pretraining for protein language models
Benchmarking how protein language models handle continual pretraining -- with Darshan Patil, Mathieu Reymond, Quentin Fournier, and Sarath Chandar.
Joined DRW as AI Research Scientist Intern
Joined DRW as AI Research Scientist Intern
Working on AI/ML research at DRW in Montreal.
Paper accepted at CoLLAs 2025 (Oral): Manifold Metric
Paper accepted at CoLLAs 2025 (Oral): Manifold Metric
A loss landscape approach for predicting model performance.
Awarded PBEEE Doctoral Research Scholarship by FRQNT Quebec
Awarded PBEEE Doctoral Research Scholarship by FRQNT Quebec
Fonds de recherche du Québec — Nature et technologies doctoral scholarship.
Paper accepted at ICML 2024: Lookbehind-SAM
Paper accepted at ICML 2024: Lookbehind-SAM
k steps back, 1 step forward -- sharpness-aware minimization with lookbehind.
Manifold Metric: A Loss Landscape Approach for Predicting Model Performance
Manifold Metric: A Loss Landscape Approach for Predicting Model Performance
Using loss landscape geometry to predict model generalization without held-out data.
Authors: P. Malviya, J. Huang, A. Baratin, Q. Fournier, S. Chandar
Lookbehind-SAM: k steps back, 1 step forward
Lookbehind-SAM: k steps back, 1 step forward
An efficient extension to Sharpness-Aware Minimization that leverages historical gradient information.
Authors: G. Mordido, P. Malviya, A. Baratin, S. Chandar
Promoting Exploration in Memory-Augmented Adam using Critical Momenta
Promoting Exploration in Memory-Augmented Adam using Critical Momenta
A memory-augmented optimizer that stores and retrieves critical momenta to promote exploration in the loss landscape.
Authors: P. Malviya, G. Mordido, A. Baratin, R. Babanezhad, J. Huang, S. Lacoste-Julien, R. Pascanu, S. Chandar
TAG: Task-based Accumulated Gradients for Lifelong Learning
TAG: Task-based Accumulated Gradients for Lifelong Learning
A gradient accumulation method for continual learning that prevents catastrophic forgetting.
Authors: P. Malviya, B. Ravindran, S. Chandar
An Introduction to Lifelong Supervised Learning
An Introduction to Lifelong Supervised Learning
A comprehensive primer on lifelong/continual supervised learning — survey of the field covering task-incremental, class-incremental, and domain-incremental settings.
Authors: S. Sodhani, M. Faramarzi, S.V. Mehta, P. Malviya, M. Abdelsalam, J. Rajendran, S. Chandar
Others
Preprints and workshop papers
D. Patil, P. Malviya, M. Reymond, Q. Fournier, S. Chandar
P. Malviya, G. Mordido, A. Baratin, R. Babanezhad, G.K. Dziugaite, R. Pascanu, S. Chandar
P. Malviya, D. Patil, M. Hashemzadeh, Q. Fournier, S. Chandar
D. Patil, P. Malviya, M. Hashemzadeh, S. Chandar
Education
MILA / Polytechnique Montreal
Indian Institute of Technology Madras
IIIT Bhubaneswar
Experience
DRW
DRW
Polytechnique Montreal
ArangoDB GmbH
RBCDSAI, IIT Madras
RISE Lab, IIT Madras
Service & Teaching
- Deep Learning Dynamics — Polytechnique Montreal
- Machine Learning — Polytechnique Montreal
- Reinforcement Learning — IIT Madras (NPTEL)
- Practical ML with TensorFlow — IIT Madras (NPTEL)
- Big Data Lab — IIT Madras
- Co-organized the DEAADIGS workshop at ACM Web Science 2021.
Earlier Work
Fast CSP and Learning-based Minesweeper Solver
Fast CSP and Learning-based Minesweeper Solver
AI solver using Constraint Satisfaction Problems, XGBoost, and Deep Q-Learning — achieving 92% win rate on standard boards.