Continual Adaptation at Scale — ICML 2026 Workshop

Invited Speakers

Razvan Pascanu

Google DeepMind / MILA

Razvan Pascanu has been a research scientist at Google DeepMind since 2014. Before this, he completed his PhD at Universite de Montréal with prof. Yoshua Bengio, where he worked on understanding deep networks, specifically recurrent neural architectures. During his career he has made significant contributions to theory of deep networks, optimization, recurrent architectures as well as deep reinforcement learning, continual learning, meta-learning and graph neural networks. He has been Program Chair for the Neural Information Processing Systems (NeurIPS) conference and currently acts as General Chair, as well as a Program Chair for the Conference on Life-long Learning Agents (CoLLAs) and the Learning on Graphs Conference (LoG). He has organized various workshops on topics such as continual learning at top-tier conferences. He is also one of the main organizers of the Eastern European Machine Learning Summer School (EEML) and EEML workshop series, as well as an organizer of the Romanian AI Days.

Bing Liu

University of Illinois at Chicago

Bing Liu is a Distinguished Professor and Peter L. and Deborah K. Wexler Professor of Computing at the University of Illinois Chicago (UIC). He earned his Ph.D. in Artificial Intelligence from the University of Edinburgh. His current research interests include continual or lifelong learning, autonomous learning from experience, learning to reason, machine learning, and natural language processing. He is the author of several books on these topics and has also received multiple Test-of-Time awards for his papers. Liu is the 2018 recipient of the ACM SIGKDD Innovation Award and a Fellow of AAAI, ACM, and IEEE.

Stephanie Chan

Google DeepMind

Stephanie Chan is a Staff Research Scientist at Google Deepmind. She received her PhD in computational neuroscience from Princeton University, and an SB in physics and an SB in brain & cognitive sciences, both from MIT. Her research covers a few broad areas. One set of work aims for a scientific understanding of modern AI models, especially in-context learning. She also works on developing AI systems for human enrichment and human empowerment. This has included work in AI for education, and AI to improve the information ecosystem. Now, her primary work revolves around understanding AI's future impacts on society.

Tinne Tuytelaars

KU Leuven

Tinne Tuytelaars is a full professor at KU Leuven and a leading researcher in computer vision and artificial intelligence. Her research focuses on image and video understanding, multimodal AI, and continual learning. She is passionate about building AI systems that can learn continuously and adapt to new situations, as well as moving towards genuine image and video understanding. Alongside her research, she enjoys sharing her knowledge and plays an active international leadership role in the AI community through major computer vision conferences and organizations. She received two ERC grants, the Koenderink test of time award, and various other prizes.

Jaehong Yoon

Nanyang Technological University

Dr. Jaehong Yoon is an Assistant Professor at Nanyang Technological University (NTU), Singapore. Prior to joining NTU, he was a postdoctoral research associate at UNC-Chapel Hill, working with Prof. Mohit Bansal. He received his Ph.D. at KAIST, advised by Prof. Sung Ju Hwang. His research focuses on building AI systems that reliably operate and continuously learn in complex real-world environments. Dr. Yoon has received several honors, including the AAAI New Faculty Highlights (2026), the NSCC Young Investigator Seed Project Award (2026), the CoLLAs Early-Career Spotlight (2025), and the Google PaliGemma Academic Program Award (2024). He will serve as the DEI Chair for CoLLAs 2026 and has served as an Area Chair for multiple venues, including NeurIPS 2025;2026, ACL 2026, NAACL 2025, and EMNLP 2024; 2025.

Parth Asawa

UC Berkeley

Parth Asawa is a PhD student at UC Berkeley advised by Professor Matei Zaharia and Professor Joey Gonzalez. Parth's research is on continual learning, studying how to enable models to stably learn from streams of experiences over time. His work focuses on sample-efficient learning and spans the stack of data, learning algorithms, and evaluation. He is grateful to be supported by the Laude Institute via their Open Research Residency.

Naman Goyal

Google DeepMind

Naman is a machine learning researcher and engineer at Google DeepMind, where he focuses on pushing the frontiers of large language model alignment and autonomous agent capabilities. Naman served as a Founding Engineer for Gemini Deep Research pioneering the long-horizon planning and context management systems that power complex, multi-step AI reasoning. He is also a co-author of the foundational Gemini 2.5 technical report. Previously he worked on multimodal foundation models at NVIDIA and vision-language systems at Apple. He holds a Masters in Computer Science from Columbia University and a Bachelors in Computer Science from IIT Ropar.

Jenny Ni

Google

Jenny is a Senior AI Security and Trust & Safety Professional at Google, specializing in the development and deployment of secure, large-scale machine learning systems. With a deep expertise in multimodal AI safety, Jenny focuses on mitigating risks and building robust safeguards for complex AI models. Prior to her current role at Google, she drove key Trust & Safety initiatives at Apple. She brings a wealth of industry experience in scaling responsible AI systems from research into global consumer products.

Schedule

All times are local to the venue.

Time	Type	Activity
08:00 – 08:05	Opening	Opening Remarks
08:05 – 08:25	Invited Talk	Parth Asawa — Beyond Static Intelligence: Evaluating Continual Learning Continual learning, the ability of AI systems to improve through sequential experience, has attracted substantial interest, but no high-quality benchmark exists to evaluate it. We introduce Continual Learning Bench (CL-Bench), the first difficult, expert-validated benchmark designed to measure whether LLM-based systems genuinely improve with experience. CL-Bench spans six diverse domains (software engineering, signal processing, disease outbreak forecasting, database querying, strategic game-playing, and demand forecasting), each validated by domain experts and designed so that tasks share a learnable latent structure (codebase layout, disease outbreak dynamics, opponent strategies) that a stateful system can discover online but a stateless one cannot. We evaluate frontier models across several agent architectures, from naive in-context learning (ICL) to dedicated memory systems, introducing a gain metric to isolate learning from prior capabilities. We find that these systems leave headroom for improved continual learning: agents frequently overfit to immediate observations or fail to reuse knowledge across instances, and dedicated memory systems do not fix this---in fact, naive ICL outperforms systems dedicated to memory management. CL-Bench is the first benchmark to evaluate continual learning across diverse real-world domains with expert-validated tasks and isolate online learning from underlying model capability, showing a need for better continual learning systems.
08:25 – 09:05	Invited Talk	Razvan Pascanu — Fast adaptation: a CL perspective In this talk I will focus on the theme of the workshop – fast adaptation – and discuss a few challenges and perspectives, looking at the problem from a Continual Learning (CL) and Learning Dynamics point of view. I will start by framing the concept of fast adaptation in the context of CL and specifically talk about the role of catastrophic forgetting (CF). While catastrophic forgetting seems to be at odds with adaptation, I will argue that there is a sense in which solving catastrophic forgetting should be necessary to improve learning efficiency. Specifically I will focus on interference as a mechanism that slows down learning for modern deep learning. In the last part of my talk I will discuss compositionality as an alternative way of thinking of adaptation, arguing that modern models struggle with compositional learning and how compositionality could go beyond solving interference.
09:05 – 09:40	Break	Coffee / Snack Break
09:40 – 10:10	Invited Talk	Jenny Ni and Naman Goyal — Shift Happens: Robustness and Reliability of Multimodal Foundation Models Multimodal models perform well on curated benchmarks but often fail unpredictably when deployed in the real world, where input distributions shift, modalities may be missing, and data quality varies. This workshop focuses on the fundamental ML challenge of robustness under distribution shift in multimodal settings: how do models degrade when visual or textual inputs shift? Can we predict and mitigate failures before deployment? What theoretical frameworks govern cross-modal robustness?
10:10 – 10:40	Oral Session	Oral Session 1 10:10 The Geometry of Forgetting: A Fisher-Information Framework for Alignment-Preserving Continual Adaptation Soham Batra · Siddharth Karuturi · Kaustubh Bukkapatnam · Laksh Patel 10:20 Aligning Language Models from User Interactions Thomas Kleine Buening · Jonas Hübötter · Barna Pasztor · Idan Shenfeld · Giorgia Ramponi · Andreas Krause 10:30 Fast-weight Product Key Memory Tianyu Zhao · Llion Jones
10:40 – 11:20	Invited Talk	Tinne Tuytelaars — (When) do we still need continual learning Continual learning research has traditionally been evaluated on simplified datasets and highly controlled benchmarks, raising questions about its relevance in an era where powerful foundation models exhibit impressive zero-shot capabilities. In this talk, I argue for a shift toward more realistic and impactful scenarios, such as personalization, video understanding, and robotics, where the ability to learn continuously from new experiences is essential. These settings highlight the unique strengths of continual learning and demonstrate its potential to complement, rather than compete with, foundation models.
11:20 – 12:00	Invited Talk	Jaehong Yoon — Toward Continually Improving Long-Horizon Agents Long-horizon agents need to learn from evolving data, remember useful experience, and use that experience to improve future behavior. This talk presents a research trajectory toward such continually improving agents through three connected stages. I will first discuss how agents can selectively expand their capabilities as new multimodal instruction data becomes available, avoiding redundant learning while acquiring a broader range of skills. I will then move to the problem of memory, where long-form visual experiences require agents to preserve both abstract knowledge and fine-grained evidence across extended temporal contexts. Finally, I will discuss how agents can go beyond passive understanding by using their own failures to generate targeted environments for embodied skill acquisition. These directions outline a path toward agents that can select what to learn, organize what they remember, and improve their behavior over time.
12:00 – 13:00	Break	Lunch Break (on your own)
13:00 – 13:40	Invited Talk	Bing Liu — Is Continual Learning Easy with a Foundation Model? Continual learning—the ability to accumulate knowledge over a lifetime—is a hallmark of human intelligence yet remains largely missing in AI systems. While challenges like catastrophic forgetting have limited prior methods, recent results show that strong foundation models can achieve theoretical upper bounds. This raises a provocative question: Is continual learning easy with a strong foundation model? In this talk, I will present these findings and discuss their potentially controversial implications.
13:40 – 14:20	Oral Session	Oral Session 2 13:40 Is Our Benchmark Enough? An Analysis of Continual Learning for MLLMs Van-Tuan Tran · Shruthi Gowda · Merim Dzaferagic · Marco Ruffini 13:50 Merging Adapted Models via Data-Free Covariance Estimation Marawan Gamal · Derek Tam · Pascal Jr Tikeng Notsawo · Colin Raffel · Guillaume Rabusseau 14:00 On the Importance of Trivial Baselines: Re-evaluating LoRA Adapter Transfer for Generative Tasks Andreas Hochlehnert · Danni Liu · Jan Niehues · Shankar Kumar · Felix Stahlberg 14:10 Sparse structured matrices: Efficient adapter rank in fine-tuning Foundation models An Nguyen · Anh Tong
14:20 – 15:00	Invited Talk	Stephanie Chan — In-context learning can mitigate catastrophic forgetting; and what happens after we solve continual learning? This talk will have two parts. In the first part, I will discuss our new results testing the hypothesis that in-context learning (ICL) can mitigate catastrophic forgetting -- because ICL enables more accurate posterior inference about the current "task" or "context", which can in turn enable more targeting weight updating. In the second part, I will discuss the implications of a world with widely deployed continual learning agents, and how it poses major challenges for AI evaluation and alignment -- many techniques assume a single static base model, and are not suited for dynamically changing models.
15:00 – 15:30	Break	Coffee / Snack Break
15:30 – 17:00	Poster	Poster Session - Hall A (4115–4116, 4200–4216, 4300–4305) Listen, Look, and Learn: Learning Without Forgetting through SAM-Audio AVI GUPTA · Nilotpal Sinha · Vishnu Raj · Sambuddha Saha · Pratik Joshi · Koteswar Rao Jerripothula · Tammam Tillo Externalizing Plasticity: Zero-Update Continual Learning via Symbolic Memories Manojshyaam C J Semantic Grouping with Dual-Strategy Distributional Rehearsal for Continual Learning Khoa Le Nho · Hồ X Hùng · Thanh Hai Dang · Duc-Trong Le · Quynh-Trang Pham Thi SPA: A Simple but Tough-to-Beat Baseline for Knowledge Injection Kexian Tang · Jiani Wang · Shaowen Wang · Kaifeng Lyu Audit Before You Merge: Provenance, Probing, and Continual LoRA Composition Lenny Aharon · Neta Glazer · Lior Aharon Staged Continual Adaptation of Multimodal Foundation Models for Japanese Financial Documents Genshin Kakimoto · Atsushi Yanagisawa REPO: Detoxifying LLMs via Representation Erasure-based Preference Optimization Naz Sepahvand · Eleni Triantafillou · Hugo Larochelle · Doina Precup · Daniel Roy · Gintare Karolina Dziugaite Subspace Optimization for Backpropagation-Free Continual Test-Time Adaptation Damian Sójka · Sebastian Cygert · Marc Masana Hard-First: Entropy-Guided Curriculum Distillation Balances Transfer and Preservation in Biomedical Vision-Language Models Junseob Kim · Sunil Hwang · Mehak Arora · Rishikesan Kamaleswaran Functional Task Networks: Cortex-Inspired Spatial Parameter Isolation for Continual Learning Kevin L McKee · Yicong Zheng · Zacharie Bugaud · Thomas Miconi Can Large Language Models Keep Up? Benchmarking Online Adaptation to Continual Knowledge Streams Jiyeon Kim · Hyunji Lee · Dylan Zhou · Sue H Park · Seunghyun Yoon · Trung Bui · Franck Dernoncourt · Sungmin Cha · Minjoon Seo Mitigating Catastrophic Forgetting in Continual RL via Certified Alignment Maksim Anisimov · Matthew Wicker · Francesco Belardinelli DECAF: De-Clustering for Adaptive Representational Unlearning Anjie Le · Can Peng · Hongcheng Guo · J. A Noble On-Policy Adaptation Mitigates Hyperparameter-Sensitive Forgetting in Vision-Language Models Doyoon Kim · Ziyang Wu · Jingfeng Yang · Yi Ma Test-Time Training for Visual Foresight Vision-Language-Action Models Sangwu Park · Wonjoong Kim · Yeonjun In · Sein Kim · HongSeok Kang · Chanyoung Park Machine Studying: A System-Level Reframing of Continual Adaptation from Declarative Corpora Jacob Xiaochen Li · Rick Battle · Omar Khattab Mechanistic origins of catastrophic forgetting: why RL preserves circuits better than SFT? Jeanmely R Nunez · Viraj V Sawant · Nathan Allen · Nomgondalai Amgalanbaatar · Yannis Zongo · Kevin Zhu · Maheep Chaudhary Early Data Exposure Improves Robustness to Subsequent Fine-Tuning Lawrence Feng · Gaurav Ghosal · Jacob Mitchell Springer · Ziqian Zhong · Aditi Raghunathan Rotation-Preserving Supervised Fine-Tuning Hangzhan Jin · Tianwei Ni · Lu Li · Pierre-Luc Bacon · Mohammad Hamdaqa · Doina Precup Characterizing Plastic Regions in Neural Networks Gwyneth Liu · Nir Shavit · Linghao Kong Continual Learning of Physical Systems via Derivative Distillation Claudia Gentili · Alessandro Trenta · Andrea Cossu · Davide Bacciu MeMo: Memory as a Model Ryan Wei Heng Quek · Sanghyuk Lee · Alfred Leong · Arun Verma · Alok Prakash · Nancy Chen · Bryan Kian Hsiang Low · Daniela Rus · Armando Solar-Lezama Tell Me What To Learn: Generalizing Neural Memory to be Controllable in Natural Language Max S Bennett · Tom Zollo · Richard Zemel Orthogonal Mixture-of-Expert Low-Rank Adapter for Continual Learning Zhuoran Xie · Shichun Yang · Xu Mohan · Qingyun Ye · Jiong Ding · Chen rui · Fan Zhou AV-CTTA: Audio-Visual Continual Test-Time Adaptation without Forgetting Sarthak Kumar Maharana · Akshay Mehra · Bhavya Ramakrishna · Yunhui Guo · Guan-Ming Su Route, Reuse, Repurpose: Continual Adaptation of LLMs with Bounded Adapter Pools Bilge Celik · Anna Vettoruzzo · Joaquin Vanschoren A loss curvature account of fine-tuning fragility Ivaylo Dimitrov · Leo Karoubi · Sunny Howard · Dmitrii Krasheninnikov Continual Knowledge Updating in LLM Systems: Learning Through Multi-Timescale Memory Dynamics Andreas Pattichis · Constantine Dovrolis Organized Plasticity for Cost-Bounded Continual Adaptation Yourim Choi · Subeom Park · Reeahn Kim · Hyung-Sin Kim TIMEGATE: Sustainable Time-Boxed Promotion Gates for Continual ML Adaptation Under Resource Constraints Abhijit Chakraborty · Suddhasvatta Das · Yash Shah · Vivek Gupta · Kevin A Gary Calibrated Target-Aware Data Selection for Continual Mid-Training in Streams Ke Wan · Tianyi Zhao · Tianhao Huang · Guanghui Min · Chen Chen Factor Imbalance and Plasticity Loss in Low-Rank Factorized Networks Seungwon Oh · KyungJoong Kim Quantifying Subliminal Behavioral Transfer Ratios in Language Model Distillation Uwe König · Hamza Kazmi · Ruizhe Li · Maheep Chaudhary RA-LoRA: Rank-Adaptive Low-Rank Adaptation via Subspace Interference Measurement for Continual Fine-Tuning of Foundation Models Kaustubh Bukkapatnam · Siddharth Karuturi Experience-Guided Behavior Adaptation for Large Language Models Iknoor Singh · Harjot Singh · Abhishek Tripathi · Niresh Agarwal · Murat Sensoy Catastrophic Forgetting is Low-Rank: A Function-Space Theory for Continual Adaptation Ido Nitzan Hidekel · Dan Raviv Measurement Plasticity: Sensor-Level Adaptation for Vision–Language Models Boyeong Im · Wooseok Lee · Yoojin Kwon · Hyung-Sin Kim Selective Memory Retention for Long-Horizon LLM Agents Pranath Reddy Kumbam Provable Forgetting Bounds Drive Capacity Savings: Spectral Thresholding in Continual LoRA Louis Barinka · Giulia Lanzillotta · Enis Simsar · Tanguy Dieudonné · Thomas Hofmann RepSelect: Robust LLM Unlearning via Representational Selectivity Filip Sondej · Yushi Yang · Adam Mahdi Continual Causal Refinement: Learning from Sequential Perturbation Data Masayuki Nagai · Alan Murphy · Peter Koo Beyond Classification: Continual Learning for Multimodal Retrieval Alicja Dobrzeniecka · Filip Szatkowski · Sebastian Cygert · Szymon Lukasik · Bartłomiej Twardowski