Richard Ngo
My MATS fellows will do philosophical thinking about multi-agent intelligence and how agents change their values. This will likely involve trying to explore and synthesize ideas from game theory, signaling theory, reinforcement learning, and other related domains.
Stream overview
I'd like theoretically-inclined scholars to work with me towards building up a theory of coalitional agency. This work will be highly abstract, philosophical and exploratory; it's mainly suited for people with strong mathematical backgrounds who've read some of my existing writings on this topic and find them interesting.
Mentors
Richard previously worked on alignment at DeepMind and governance at OpenAI. He's currently an independent researcher focusing on multi-agent intelligence. He's particular interested in understanding how subagents aggregate to form robust larger-scale agents, and how those larger-scale agents change the values of their subagents.
Mentorship style
I'll come meet scholars in person around 2 days a week on average. On those days I'll be broadly available for discussions and brainstorming. On other days scholars can message me for guidance (though I'd prefer to spend most of my effort on this during the in-person days).
Representative papers
See "Towards a scale-free theory of intelligent agency" and the various work linked from there.
Scholars we are looking for
My main criterion for selecting scholars will be clarity of reasoning.
You will probably will work with other scholars in the stream.
Project selection
I will talk through project ideas with the scholar.