Marius Hobbhahn
We will continue working on black-box monitors for scheming in complex agentic settings, building on the success of the previous stream.
See here for details.
Stream overview
The entire stream will be dedicated to building high-quality black-box scheming monitors.
In a previous stream, we have had great initial success designing black box monitors for scheming and we've made substantial progress since then.
We will continue to work on black box scheming monitors. Potential directions for this version of the stream could include: a) fine-tuning monitors, b) improving monitors through adversarial training, c) more "science of monitoring", d) harder and more complex datasets for scheming to improve monitor training/selection and testing.
London
Mentors
Marius Hobbhahn is the CEO of Apollo Research, where he also leads the monitoring team. Apollo is an AI safety research organization focused on scheming, evals and control/monitoring. He is a TIME100 in AI2025 recipient. Prior to starting Apollo, Marius did a PhD in Bayesian ML and worked on AI forecasting at Epoch.
Fellows we are looking for
- You are interested in this project in particular, i.e. working on black-box monitors for scheming in agentic settings. I will only focus on this project and no other research.
- You enjoy tinkering with LLMs, e.g. prompting, building basic LM agents, and fine-tuning.
- You are happy to build datasets and synthetic data generation pipelines. We found that a substantial amount of initial work is required for the data generation process.
- You like quick empirical iteration and direct feedback loops.
- I expect that you will spend 20% on conceptual work (e.g., think about which environments could work or what techniques to try) and 80% on hands-on empirical work (e.g., implementing and running experiments).
- I prefer that scholars focus 100% of their work time on the project and not pursue any side projects. In general, I’m happy to support highly ambitious scholars who want to make a lot of progress during MATS. In the past, people have described my stream as "intense, but in a good way".
Project selection
You will work on subprojects of black box monitoring. See here for details.