Empirical
Streams in this track include hands-on research using machine- learning experiments to understand and improve model safety including AI control, interpretability, scalable oversight, evaluations, red-teaming, and robustness. This track is defined by its methods rather than any single research agenda. If your primary tool is ML engineering, this is the track for you.
- Stage 1: Complete the general application
- Stage 2, phase 1: Complete 1–2 assessments evaluating research taste and technical implementation skills, alongside possibly reference requests
- Stage 2, phase 2: Stream selection questions
- Stage 3: Interviews and work-tests
The track is defined by its methodology more than by any single research agenda. Fellows run ML experiments to understand and improve the safety properties of frontier models, with work spanning interpretability, AI control, scalable oversight, evaluations, red-teaming, robustness, and model organisms of misalignment. The unifying thread is that progress comes from hands-on work with real models (training, probing, fine-tuning, measuring, etc.) rather than reasoning from first principles alone. This is the largest track in the program and the most common entry point into technical AI safety research.
We are looking for fellows whose primary tool is ML engineering, broadly construed. The essential requirement is the ability to design and run experiments on language models or other deep learning systems and iterate quickly on the results. In practice, that usually means having a solid understanding of Python (with and without AI coding tools), being comfortable with the infrastructure around running models at moderate scale, and knowing which experiments are worth running. Mission alignment is highly important, and fellows should be able to say why a given line of empirical work meaningfully reduces frontier risk, not just whether it yields a successful publication. Educational background and seniority are weighted lightly here relative to other tracks. Past cohorts have included strong fellows ranging from undergraduates to senior industry researchers.
Fellows are matched to mentors based on fit, and projects are scoped to produce concrete artifacts (i.e., papers, evaluation suites, open-source tooling, or technical reports) by the end of the program. The target audiences for the work produced in this track would include safety and alignment teams at frontier labs, governments and other evaluation organizations, and the broader ML research community.
If you are excited by this kind of work, we encourage you to apply.
We are excited to supervise projects:
1. Study the causes and implications of situational awareness, and the role that meta-cognition plays in (multi-agent) alignment;
2. Contribute to LawZero's Scientist AI, in the form of contextualization and uncertainty estimation.
Lee's stream will focus primarily on improving mechanistic interpretability methods for reverse-engineering neural networks.
The stream focuses on evaluating and/or mitigating catastrophic risk emerging from dangerous scientific capabilities in frontier AI systems, with an emphasis on the challenges that emerge from lab integrations and novel science. Potential research directions include evaluation design, risk mitigations and evaluation science.
This stream focuses on critical challenges in AI safety and alignment, including risks from automating AI research, bottlenecks to recursive self-improvement, and the automation of safety and alignment research. Priority topics also include AGI privacy, measuring long-horizon agentic capabilities, developing new alignment methods, and advancing the science of post-training.
This stream focuses on understanding model character in language model training dynamics, addressing questions around character reliability, scaled RL’s effects on character training, and how model character interacts with capabilities training. We plan on building a training-focused workstream to answer these questions, with a goal of building out the early, foundational research area on character training across ML, NLP, and safety fields.
Neel takes a pragmatic approach to interpretability: identify what stands between where we are now and where we want to be by AGI, and then focus on the subset of resulting research problems that can be tractably studied on today's models. This can look like diving deep into the internals of the model, or simpler black box methods like reading and carefully intervening on the chain of thought - whatever is the right tool for the job. This could look like studying how to detect deception, understanding why a model took a seemingly concerning action, or fixing weak points in other areas of safety, e.g. using interpretability to stop models realising they are being tested. You can learn more about Neel's approach in this podcast.
He has spent far too much time having MATS scholars, and has worked with ~60 so far - he’s excited to take on even more!
This stream focuses on empirical projects that improve our ability to evaluate model capabilities or enable us to understand or evaluate model monitorability. An ideal project culminates in a research output (conference/Arxiv paper or research blogpost with artifacts).
Computational/modelling problems in biosecurity.