Cozmin Ududec
I currently lead the science of evaluation team at the AI Security Institute in London. I'm interested in topics around dangerous capability measurement, and understanding agent behaviours and goals, and their implications for policy.
Stream overview
A few project ideas:
- Developing predictive statistical models that capture the relationships between LLM performance, task characteristics and LLM attributes.
- Developing methods to forecast when models might cross a capability threshold.
- Developing methods for tracking partial progress on hard tasks & estimating gaps in knowledge/skills (eg through hinting) required to cross capability thresholds.
- Developing and validating transcript analysis tools for quantitatively and systematically observing agent behaviors and failure modes.
- Develop methods measuring the degree of goal-directedness of agents, i.e., whether they possess goals and act coherently towards them.
Standard (1-2 hours of weekly 1:1s)
London
No preference for this location
No preference for this location
Mentors
I currently lead the science of evaluation team at the AI Security Institute in London. I joined AISI early in its life, and have worked in several roles, including co-leading the team responsible for our pre-deployment testing programme.
I'm generally interested in topics around dangerous capability evals, and understanding agent behaviours and their implications for policy. In particular, I'm interested in:
- Developing predictive statistical models that capture the relationships between LLM performance, task characteristics and LLM characteristics.
- Developing methods to predict when models might cross a capability threshold.
- Developing hinting-based methods for partial progress & estimating gaps in knowledge/skills required to cross thresholds (and track these over time).
Before AISI I was chief scientist at a startup in Cambridge, where I led a team of 25 researchers with a mission to optimize decision making in electricity grids, and improve economic efficiency and reduce emissions.
I have a PhD in physics from the University of Waterloo and Perimeter Institute for Theoretical Physics. My focus was on reconstructing quantum theory from simple first principles, so we can all stop worrying about the reality of the wave-function.
Fellows we are looking for
1. Epistemic rigour, strong conceptual thinking and creativity: Able to dissect flawed assumptions in evaluation designs, and spot hidden pitfalls in claims.
2. Hands‑on prompting and agent scaffolding experience; engagement with experimenting and analysing model/agent behaviours.
3. Comfort with experimental design/analysis, uncertainty quantification, statistical modelling.