Empirical
Streams in this track include hands-on research using machine- learning experiments to understand and improve model safety including AI control, interpretability, scalable oversight, evaluations, red-teaming, and robustness. This track is defined by its methods rather than any single research agenda. If your primary tool is ML engineering, this is the track for you.
- Stage 1: Complete the general application
- Stage 2, phase 1: Complete 1–2 assessments evaluating research taste and technical implementation skills, alongside possibly reference requests
- Stage 2, phase 2: Stream selection questions
- Stage 3: Interviews and work-tests
The track is defined by its methodology more than by any single research agenda. Fellows run ML experiments to understand and improve the safety properties of frontier models, with work spanning interpretability, AI control, scalable oversight, evaluations, red-teaming, robustness, and model organisms of misalignment. The unifying thread is that progress comes from hands-on work with real models (training, probing, fine-tuning, measuring, etc.) rather than reasoning from first principles alone. This is the largest track in the program and the most common entry point into technical AI safety research.
We are looking for fellows whose primary tool is ML engineering, broadly construed. The essential requirement is the ability to design and run experiments on language models or other deep learning systems and iterate quickly on the results. In practice, that usually means having a solid understanding of Python (with and without AI coding tools), being comfortable with the infrastructure around running models at moderate scale, and knowing which experiments are worth running. Mission alignment is highly important, and fellows should be able to say why a given line of empirical work meaningfully reduces frontier risk, not just whether it yields a successful publication. Educational background and seniority are weighted lightly here relative to other tracks. Past cohorts have included strong fellows ranging from undergraduates to senior industry researchers.
Fellows are matched to mentors based on fit, and projects are scoped to produce concrete artifacts (i.e., papers, evaluation suites, open-source tooling, or technical reports) by the end of the program. The target audiences for the work produced in this track would include safety and alignment teams at frontier labs, governments and other evaluation organizations, and the broader ML research community.
If you are excited by this kind of work, we encourage you to apply.
The stream will focus on conceptual and empirical work on scalable oversight. We will work towards making falsifiable claims about a model's deceptive intent, developing realistic model organisms, and studying generalizability of scalable oversight methods.
I work on the science of evaluating advanced AI systems for biological and CBRN risks, with a particular interest in translating technical evidence into decisions by governments and frontier AI developers. In this stream, I’m interested in developing novel capability evaluations, studying how dangerous or dual-use capabilities diffuse into increasingly accessible models, and building scalable red-teaming methods that produce rigorous, decision-relevant evidence without requiring risky real-world demonstrations.
I worked on adversarial stress-testing at METR. Fellows in this stream will build model organisms of misalignment that require real cognition to decide whether to misbehave, then test which detection methods actually catch them, with the goal of turning whatever works into an alignment evaluation we can run on real models.
In the shard theory stream, we create qualitatively new methods and fields of inquiry, from steering vectors to gradient routing to unsupervised capability elicitation to robust unlearning. If you're theory-minded, maybe you'll help us formalize shard theory itself.
We are interested in AI control and scalable oversight. I'm excited to work with scholars interested in empirical projects building and evaluating control measures and oversight techniques for LLM agents, especially those based on chain of thought monitoring. I'm also interested in the science of chain of thought monitorability, misalignment and control. An ideal project ends with a paper submitted to NeurIPS/ICML/ICLR.
We build scalable technology for AI understanding and oversight.
This stream will focus on evaluating dangerous capabilities in language models and detecting deception and dishonesty.
The team's focus is on stress-testing model alignment to detect and understand model propensities relevant to loss-of-control risks, and includes work on building realistic alignment evaluations, measuring and mitigating evaluation awareness, inferring hidden propensities and developing automated algorithms to search for misalignment behaviour. We perform pre-deployment alignment testing across frontier AI companies. See e.g.:
- https://www.aisi.gov.uk/blog/cheating-behaviour-in-frontier-model-evaluations
- https://www.aisi.gov.uk/blog/evaluating-whether-ai-models-would-sabotage-ai-safety-research
- https://www.aisi.gov.uk/blog/investigating-models-for-misalignment
- https://arxiv.org/abs/2606.12747