Kai Fronsdal
Published Petri @ Anthropic. Not sure if he works there.
Published Petri @ Anthropic. Not sure if he works there.
As part of MATS 6.0, Kai worked on measuring instrumental self-reasoning (a precursor to deceptive alignment) in frontier models under the mentorship of David Lindner. Prior to MATS, he explored variants of Goodhart's Law and evaluating llm's ability to self-correct. As of October 2024, he is finishing a masters at Stanford University studying math and computer science while completing the MATS extension phase.
2024 年夏季班规模显著扩大，约有 90 位学员和 40 位导师参与，导师阵容为 MATS 历届最广。本届 MATS 注册成为 501(c)(3) 非营利组织，进一步完善了组织架构。随着 AI 政策和技术治理受到更多关注，项目扩展了研究范围，除技术研究流外，至少加入了 4 位治理领域导师。学员在伯克利参加为期 10 周的研究阶段，课题涵盖机制可解释性、评估、可扩展监督和治理。部分成果包括：针对使用用户反馈训练的 LLM 所开展的定向操纵与欺骗研究入选 NeurIPS 研讨会；一篇关于通过辩论实现 AI 安全的论文则获得 ICML 2024 最佳论文奖。此外，一位学员共同创立了 Decode Research，这是一家专注于构建可解释性工具的 AI 安全组织。
Kai Fronsdal
