We're open to fellows suggesting projects, but here are some concrete ones that we'd support:
1. Characterizing AI pacing dynamics. What levers are available to different actors, and what are their effects?
2. Characterizing the space of possible good futures. Which tradeoffs are inevitable, and where could we look for huge Pareto improvements? E.g. Could sophisticated paternalism allow local control while avoiding disastrous outcomes? Under what conditions should we expect moral convergence?
1. Fleshing out what "political" natural selection will lead to under various UBI/governance schemes. What mechanisms might states use to tie representation to particular minds, and how might those be predictably hacked? I.e. how will the definition of "person" likely evolve, and then be spammed?
Standard (1-2 hours of weekly 1:1s)
London
Indifferent
Indifferent
我主要想研究:随着高度先进的 AI 不断发展,怎样的未来才是好的?这包括社会可能如何组织,以及未来可能出现哪些类型的 AI。我也会做一些关于语言模型心理特征的实证研究。我真正开始研究工作的契机是 MATS 4.0,当时我研究的是预测模型的能动性理论。
David Duvenaud is an Associate Professor in Computer Science and Statistics at the University of Toronto, who now works mainly on problems related to _civilizational alignment_, i.e. understanding what it will take to keep states and institutions aligned to human interests post-AGI. He holds a Sloan Research Fellowship, a Canada Research Chair in Generative Models, and a CIFAR AI chair. His postdoc was done at Harvard University and his Ph.D. at the University of Cambridge. He is a Founding Member of the Vector Institute for Artificial Intelligence. In 2023-2024 he did a sabbatical at Anthropic, leading their Alignment Evaluations team, as well as research projects on jailbreaks and sabotage. He's also a co-chair of the Schwartz Reisman Institute for Technology and Society, a director of the AI Safety Foundation, and an advisor to AVERI. He has also received a Google Faculty Award, and best paper awards at both the Neural Information Processing Systems (NeurIPS) conference and the International Conference on Machine Learning (ICML).
我们欢迎各种背景的申请者。理想人选可能类似 Robin Hanson 或 David Friedman:既能熟练运用经济学等领域的分析工具,又深入了解人类历史、制度,以及人口、文化、国家和各类组织演变所承受的压力。
1. Conceptual clarity. You should be comfortable:
- Articulating your position in ways that would be intelligible to someone who disagrees with you,
- Operationalizing claims in terms of concrete differences in outcomes,
- Identifying cruxes,
- Working with multiple related definitions.
1. Writing and communication skills. Including:
- Asking for feedback in structured ways,
- Being open to feedback at all stages of a project,
- Providing status updates even on still-conceptually fuzzy directions
1. Good at prioritizing, i.e.
- Being willing to focus even if interesting asides come up,
- Getting drafts out for feedback on a regular basis
- Frequent meta-level discussion of what pursuits are likely worthwhile