The four golden signals (Latency, Errors, Traffic, Saturation) have been the foundation of application monitoring for years, and they are still important, but for GenAI applications, these signals alone leave significant blind spots. A request can return a 200 OK with low latency while the response generates incorrect information, leaks PII, or costs much more than expected.

This talk will walk you through the changes involved in monitoring non-deterministic, token-priced, prompt-injectable systems. We’ll cover three additional monitoring dimensions—Cost (token attribution, model-mix tracking, wasted spend on failed requests), Safety (prompt injection detection, PII scanning, jailbreak attempts), and Quality (hallucination rate, relevance scoring, user satisfaction)—and explain why each one is essential in addition to your existing monitoring tools.