Jacob Merizian

I'm interested a variety of projects, especially in multi-agent systems, AI control red-teaming, eval design, and propensity studies. Here are two multi-agent flavored research directions I would be excited about:

​

• Multi-agent control red-teaming: How effective are agents at designing multi-agent attacks in a lab R&D setting that are then delegated to sub-agents? How effective are groups of agents at executing different fidelities of plans? How are these capabilities changing over time? Can models do this with limited affordances and strict monitoring conditions? What worst case attacks can planner agents achieve? How do different monitoring regimes (e.g. partial observability) affect catch rates? Can we design more automated red-teaming methods? How well do newer / more ambitious monitoring methods perform against multi-agent attacks?
• Multi-agent propensity studies: How do propensities to undesired behavior change when agents interact with other agents? In the presence of a goal conflict, how do different collaboration/communication structures affect whether groups are more/less susceptible to taking an undesired action? Is it possible for groups of agents to perform actions that seem locally benign, but globally undesired?

​