MATS Winter 2027

This stream focuses on identifying tractable policy and technical interventions to gradual disempowerment, focusing on economic disempowerment and the intelligence curse. Possible project areas include:

1. Formalizing intelligence curse dynamics into a model that can be tracked and monitored.
2. More durable policy solutions to mass unemployment than UBI.
3. Technical interventions to extend the centaur period.

This stream focuses on critical challenges in AI safety and alignment, including risks from automating AI research, bottlenecks to recursive self-improvement, and the automation of safety and alignment research. Priority topics also include AGI privacy, measuring long-horizon agentic capabilities, developing new alignment methods, and advancing the science of post-training.

My stream aims to prevent catastrophic outcomes through the AI transition by focusing on three lines of effort: nuclear and AI-enabled strategic threats; cognitive security, information competition, and AI-enabled influence; and moral game theory, competitive strategy, and peace. Fellows can pursue their own research, help build the field, or support grantmaking in any of these areas.

Our stream focuses on AI verification, as in how actors can check that the use of AI compute is compliant with policy, especially for enabling international agreements on AI. This sense of verification is much broader than formal verification.

This stream will focus on preparing AI governance policies for future policy windows through scenario mapping and policy architecture.

Research papers (technical governance or ML) related to evaluating and mitigating dangerous AI capabilities, with a focus on what's actionable and relevant for AGI companies

This stream focuses on understanding model character in language model training dynamics, addressing questions around character reliability, scaled RL’s effects on character training, and how model character interacts with capabilities training. We plan on building a training-focused workstream to answer these questions, with a goal of building out the early, foundational research area on character training across ML, NLP, and safety fields.

Neel takes a pragmatic approach to interpretability: identify what stands between where we are now and where we want to be by AGI, and then focus on the subset of resulting research problems that can be tractably studied on today's models. This can look like diving deep into the internals of the model, or simpler black box methods like reading and carefully intervening on the chain of thought - whatever is the right tool for the job. This could look like studying how to detect deception, understanding why a model took a seemingly concerning action, or fixing weak points in other areas of safety, e.g. using interpretability to stop models realising they are being tested. You can learn more about Neel's approach in this podcast.

He has spent far too much time having MATS scholars, and has worked with ~60 so far - he’s excited to take on even more!