Drift alerts

A screening tells you how an agent performs today. Drift alerts tell you when that has quietly changed. Badge compares each day's numbers for an agent with the days before it, and raises an alert when a metric moves further than normal noise would explain.

Drift alerts are off unless your deployment turns them on. When they are off, the Drift alerts card does not appear on the Manage page, the Trends chart shows no alert markers, and the newer series (score, measured cost, p95 latency) and version notes are empty. Those series then read "Not enough data".

What is measured

Four metrics, one point per day:

Only Verified and Traced runs feed these numbers. Simulated runs are not plotted and never raise an alert. Cost uses measured cost only: a cost the agent reports about itself does not move the cost series, because it can understate what the agent really spends.

The Trends chart draws latency as the 95th percentile, because that is where slow tails show up. Alerts compare the daily median, so a single slow run does not raise one.

Scheduled screenings are the best source of points, because they run on a steady cadence. See Scheduled screenings. Manual runs count too.

When an alert fires

For each metric Badge takes the previous days as a baseline (14 days by default, and at least 5 days with runs) and compares the latest day with it. An alert needs both:

• a move of at least the sensitivity, a z-score threshold (2.5 by default), measured in baseline standard deviations, and
• a change at least as large as the metric's minimum change, in the bad direction. The defaults are 5 points for score, 5 percentage points for pass rate, 10% for cost and 20% for latency.

The second condition stops a very steady agent from alerting on a trivial change. Cost and latency also have a fixed absolute floor under their percentage: a change must be at least $0.0005 per run for cost, and 50 ms for latency. A 20% jump from 5 ms to 6 ms is not an alert. The Manage page shows this floor next to the setting; you cannot change it.

A metric raises at most one alert per agent per 24 hours. If a pass-rate alert and a score alert fire together, they travel as one message: the score message, with the pass-rate alert's id in related_alert_ids.

Who is told

Alerts always appear in the app, on the Manage page and the Trends chart. Webhook deliveries and the daily digest email are sent to owners on a paid plan. On the Free plan, alerts are in-app only: there is no webhook and no digest email.

Badge checks after a scheduled screening finishes. You can also press Check for drift now on the Manage page, once per minute per agent; if you press it sooner, the page says how many seconds to wait. There is no background polling.

On the agent page

The Trends card shows the four series. It sits on the Improve tab of the agent page, which only people who can edit the agent see, and it has its own link (/agents/<id>?section=improve#drift) that webhook deliveries point to. A red marker sits on each day that raised an alert and links to the first run behind it.

A dashed line marks each day the agent's blueprint version changed, with a note under the chart such as "v3 → v4". A drop that starts at a dashed line usually points at the change you made. Version numbers are shown to the owner only.

For a public agent, the public API lists only the metric, the day and the direction of each alert. It never includes numbers, runs or blueprint versions. Drift that happened while the agent was private is never shown publicly, even after you make the agent public.

The chart is a labelled graphic with a text summary for screen readers, and every marker can be reached with the keyboard.

Manage alerts and thresholds

On the agent's Manage page, the Drift alerts card lists each alert with the observed value, the baseline it was compared with, the z-score, the blueprint version and links to the triggering runs. Acknowledge marks one as seen. The same card holds the thresholds form: an on/off switch for the whole agent, the sensitivity (a z-score threshold), the baseline window, and for each metric an on/off switch and a minimum change. Saving replaces all of these together.

Raise a minimum change or turn a metric off if an agent is naturally noisy. Lower it if you want to hear about smaller moves.

Webhooks

Subscribe a webhook to agent.drift.score, agent.drift.cost or agent.drift.latency in Settings. A pass-rate alert is delivered as agent.drift.score; the metric field says pass_rate. Deliveries carry the usual X-Badge-Signature header; see the REST API reference for the verification recipe.

{
  "event": "agent.drift.score",
  "timestamp": "2026-10-14T10:00:00+00:00",
  "data": {
    "alert_id": "6f1f6c1e-0000-4000-8000-000000000001",
    "agent": { "id": "00000000-0000-4000-8000-000000000001", "name": "Support Agent" },
    "metric": "score",
    "direction": "down",
    "bucket_date": "2026-10-14",
    "baseline": { "mean": 72.4, "std": 1.1, "window_buckets": 14 },
    "observed": 55.0,
    "z": -15.8,
    "delta": -17.4,
    "run_ids": ["run-id-1", "run-id-2"],
    "blueprint_version_id": "3d2c9a40-0000-4000-8000-000000000002",
    "blueprint_version_number": 4,
    "chart_url": "https://badgeia.com/agents/00000000-0000-4000-8000-000000000001?section=improve#drift",
    "related_alert_ids": []
  }
}

Limits

• An alert says a number moved. It does not say why. Use the run links and the version note to find the cause.
• Too little history means no alert. Until a metric has 5 days with runs there is no baseline to compare with.
• The baseline follows the agent. A slow decline over many weeks moves the baseline with it and may not trip an alert; the Trends card still shows it.