Alan Cooney
This stream focuses on empirical AI control research, including defending against AI-driven data poisoning, evaluating and attacking chain-of-thought monitorability, and related monitoring/red-teaming projects. It is well-suited to applicants already interested in AI safety with solid Python skills, and ideally prior research or familiarity with control literature/tools (e.g. Inspect/ControlArena).
Stream overview
- Control for data poisoning. Studying scenarios where an AI system covertly data poisons another AI, to e.g., instill backdoors or secret loyalties.
- Chain-of-thought monitorability. Evaluating the monitorability of reasoning verbalised by AIs & studying ways this may be compromised.
- Other Control projects (see https://alignmentproject.aisi.gov.uk/research-area/empirical-investigations-into-ai-monitoring-and-red-teaming for ideas)
London
Mentors
Alan is Head of Autonomous Systems & Control at the UK AI Security Institute, where he works on empirical AI control and monitoring. He co-authored RepliBench, an evaluation suite measuring autonomous-replication capabilities in language-model agents.
Fellows we are looking for
Essential:
- Existing interest in AI safety
- Programming experience with Python/similar
You may be a good fit if you also have some of:
- Research experience: prior AI research experience (on any topic)
- Strong Python skills (essential for CoT monitorability eval projects)
- Familiarity with the Control literature - see here
- Familiarity with Inspect/ControlArena
Not a good fit:
- Scholars primarily interested in conceptual rather than empirical research
Project selection
By default I'll propose several projects for you to choose from, but you can also pitch ideas that you're interested in.