MATS Summer 2026

In the face of disaster, I suspect the government will be forced to play insurer of last resort, whether for a particular lab, or society at large. (I'm not the only to suspect this – see e.g. here). Designed well, I believe a federal insurance backstop could internalize catastrophic negative externalities; designed poorly, it will simply be a subsidy for AI companies. I want to design the good version, so we have it ready.

​

I encourage people with mechanism design (a.k.a. reverse game theory) expertise to apply, but don't be deterred if you don't have this expertise.

This stream focuses on representations that underlie how language models generalize, for example representations of personas, goals, or training data components.

We study applications of singular learning theory (SLT) to AI safety, with a focus on interpretability and alignment. Ideal candidates come from a strong technical background in mathematics, physics, computer science, or biology, and aren't afraid to get their hands dirty with ML experiments. We don't expect you to have deep expertise in SLT, but a shallow familiarity will help. 

This stream will focus on monitoring, stress-testing safety methods, and evals, with a focus on risks from scheming AIs. Examples include (black-box) AI control techniques, white-box monitors (probes etc.), chain-of-thought monitoring/faithfulness, building evaluation environments, and stress-testing mitigations.

In this project, we will explore GPU side-channel attacks to extract information about model usage. A simple example is to observe (via radio, power fluctuations, acoustics, etc.) which experts were used in each forward pass of an MOE model, then use those observations to guess which tokens were produced.

I'm interested in mentoring projects related to reward hacking and monitoring (agentic) models that produces long and complex trajectories. Scholar will have freedom to propose projects within this scope. Expect 30-60min 1-1 time on zoom.