Agent Observability: Monitoring and Understanding Agents at Internet Scale

Agent usage is exploding (if you haven't noticed) with an unprecedented transformation in the activities of both developers and other roles creating enormous volumes of new autonomous, dynamic decision making programs that can do extraordinary things but also hallucinate, misunderstand and in the worst case cause real damage.

In this talk I'll discuss Google's broad approach to figuring out what it is that agents are doing: what actions they are taking, how risky they are, what data is necessary to make good decisions, and how we can scale our approach to an enterprise the size of Google through automated monitoring and policy. I'll also discuss how we've been tackling the fundamental and critical problem of how to achieve both speed and safety.