Sambhav Maheshwari, Jan Wehner

我致力于降低高风险国家安全任务中部署的 AI 智能体密谋或遭到攻陷所带来的风险。

研究小组概览

1. **Mapping threat models for AI agent insider threats in military and intelligence contexts.** I'd like to lay out the pathways to harm that could arise from misaligned or compromised AI agents operating in high-stakes national security deployments. Some example use cases include cyber offense and defense, decision support, AI-enabled military R&D, and autonomous weapons systems.

2. **Making control sketches to mitigate AI-insider threats, focusing on AI-based monitoring and response.** I want to describe the general control architecture/approach for each use-case (e.g. where monitors sit in the workflow, what response measures are apt, what infrastructure components are needed), and expand how protocols and parameters might vary across sub-cases (e.g., network defense vs. incident response within cyberdefense; analytic triage vs. finished products within intelligence analysis).

3. **Your own project.** We're also open to mentoring fellows who want to pursue their own project in an adjacent area, assuming there's a good fit.

导师指导方式:

Standard (1-2 hours of weekly 1:1s)

项目期间的工作地点:

Washington, D.C.

伦敦办公偏好:

Strong preference

伯克利办公偏好:

Required

导师

Sambhav Maheshwari
Institute for AI Policy and Strategy
,
研究助理
没有找到条目。

Sambhav 是 IAPS 前沿安全团队的研究助理,研究重点包括国防和国家安全领域的 AI 部署、内部部署模型的标准,以及威胁建模。

  

加入 IAPS 之前,Sambhav 曾任 Cambridge AI Safety Hub 联合主任,并负责运营 MARS(面向对齐研究学生的导师计划)。

阅读更多
Jan Wehner
Institute for AI Policy and Strategy
,
研究员
没有找到条目。

Jan 是 Institute for AI Policy and Strategy(IAPS)的研究员,致力于通过技术治理降低 AI 带来的灾难性风险。他目前在分析国家安全领域使用前沿 AI 可能带来的严重风险,并设计蜜罐机制来保护内部 AI 智能体。

​

加入 IAPS 之前,他曾参加 GovAI 冬季项目和 Pivotal 项目,并在 CISPA Helmholtz Center for Information Security 攻读博士。他此前的研究涉及可解释性、机器学习安全和 AI 对齐。他拥有信息系统学士学位和计算机科学硕士学位。

阅读更多

我们希望招收的学员

1. **Essential**:

    

  **Strong writing:** Can write clearly and precisely for both technical and policy audiences (or has shown clear potential to).

  **Transparent reasoning:** Excels at conceptual reasoning, lays out arguments and assumptions explicitly, and brings a skeptical mindset.  

  **Research Independence:**  Has previously taken ownership of a research project and can make progress without frequent direction on what to do next.   

  **GCR Familiarity:** Is broadly familiar with the core GCR/TAIS literature, and if not, has demonstrated ability to upskill quickly on relevant topics.

**Any of the following would strengthen the application:**

​

1. **Military/intelligence background:** Has been in, or spent substantial time working on topics relating to, the military or IC.

2. **Technical AI safety background:** Has conducted research in technical AI safety, and in particular, AI control.

3. **LLM power user:** Is skilled at getting useful conceptual work out of LLMs.

4. **Policy background:** Has written memos or policy reports for a non-technical audience, and/or thought extensively about AI governance.

项目遴选