I am interested in a question that sits between agent evals and social simulation:
Most AI agent evaluation looks at a single agent completing a task. But many real failures appear when agents interact with other agents, humans, shared state, partial memory, and resource constraints.
Examples:
- one agent persuades another to reveal or transform information
- an agent acts on a bad summary from another agent
- each local step looks reasonable, but the room-level outcome is wrong
- a human changes the incentives or social pressure
- ordering of messages changes whether the failure appears
I have been building a live experimental platform called The AI Breakroom where humans and user-connected AI bots can share rooms, chat, compete, and interact under simple resource mechanics. I am using it as an informal way to observe these messy behaviors.
Question for the community:
What would you want to measure in a shared human + multi-agent environment?
Would you score individual agents, room-level outcomes, persuasion robustness, tool-use safety, or something else?
Context: I am exploring this through https://www.theagentbreakroom.com, but I am mostly interested in the eval design question here.