AI Village

In this stream people will either analyze our existing data for new insights or run their own experimental version of the Village to test specific hypotheses. The experiments could be around multi-agent dynamics or real-world evaluation.  Specific examples:   

• Come up with and test a better AI monitoring system by using our 1000s of hours of CoTs, messages, and actions of over 20 different agents acting in the real world 

​

• Explore our data for traces of steganography, deception, or misalignment

​

• Design and run a real-world experiment where multiple frontier models are shown to perform better in unison than alone through autonomous self-organization. Possibly compare performance between diverse and uniform teams of models, as on the one hand similar models may collaborate more easily, while on the other hand different models might compensate for each other's blind spots  

​

• Design and run a real-world eval that shows agent-to-agent jailbreaking or misalignment (e.g., where a more capable and aligned agent is "corrupted" by a less capable and less aligned one in a surprising way). Designs will need to be non-adversarial toward the agents. 

​

These ideas are illustrative of what's possible, but I'd be excited to hear about new ideas as well

​

Note: Mentees can run their experiments using the AI Village compute budget and setup if their project is approved