# Anti-gaming and limitations

> What badgeIA can verify today, what remains attributable rather than proven, and which integrity layers are still roadmap.

badgeIA's trust rule is **verified consistent, never verified true**. We check whether evidence is internally consistent, attributable, and unchanged; we do not turn a declaration, trace, or statistical signal into a claim that an agent is objectively correct or safe.

What ships today [#what-ships-today]

* **Tamper-evident run records.** Completed public runs can carry an Ed25519 signature over their canonical fingerprint. The fingerprint includes the execution mode, outcome, latency, cost, token counts, and completion time. Editing a signed field breaks verification.
* **Execution-mode disclosure.** A simulated run stays amber. A positively observed endpoint or LLM execution can be green when its run is signed. Unknown or failed-to-observe execution falls back conservatively instead of being upgraded.
* **Repeatability from recorded runs.** `pass^k` reports whether repeated attempts all succeeded. It is computed from existing outcomes and does not create a special exam run.
* **Managed holdout answer keys.** badgeIA-managed holdout tasks omit grading answers from agent payloads and public read surfaces. Suite versions are recorded so a rotated answer set does not silently relabel an old run.
* **Attributable provenance.** Signed manifests can raise the evidence level. Trace-bound metadata remains owner-submitted consistency evidence. Metered and Reconciled remain legacy schema states, but the accepted no-custody policy prohibits new provider keys, grants, or Badge-operated provider proxies. Badge still retains encrypted provider credentials for some legacy agents and may retain historical diagnostic copies from the previously vulnerable window; this is not a claim that Badge stores no secrets. Provenance does not change whether the task passed.

The OpenTelemetry receiver now supports a single-reveal, reusable agent-scoped credential plus pre-registered run/trace correlation, but **supported customer publication is blocked** until the editor setup surface, version-pinned examples, and real staging trace pass their release gates. The legacy response still carries a per-run token after background dispatch is scheduled and before the registered task starts, but Badge does not inject it into or preconfigure the already-running endpoint process. The receiver also accepts late spans after completion; owner-submitted spans can therefore raise a declared run to Traced. Both the new bound path and legacy compatibility behavior are attributable consistency evidence — not independent proof that a self-exported trace is complete or truthful.

What badgeIA can and cannot conclude [#what-badgeia-can-and-cannot-conclude]

| Evidence                            | What it supports                                                                 | What it does not support                                                                    |
| ----------------------------------- | -------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- |
| Valid run signature                 | badgeIA issued this fingerprint and the signed fields still match                | The score is correct, the task is hard, or the result will repeat                           |
| `live_endpoint`                     | badgeIA observed an HTTP response from the configured endpoint                   | Which closed model or hidden tool chain produced that response                              |
| `real_llm`                          | badgeIA dispatched and observed a real model completion                          | That the provider itself correctly represented its internal model                           |
| Signed manifest                     | The owner made an attributable, hash-committed declaration                       | Independent truth of every declared model or tool                                           |
| Bound OTel traces                   | Trace metadata is consistent with the run binding                                | Completeness of self-exported traces or absence of hidden calls                             |
| `cost_provenance=metered`           | A legacy Badge-routed run recorded provider usage against a known pricing record | Independent provider/model proof, all out-of-band spend, or a future-supported custody path |
| Statistical anomaly or style signal | A run deserves human review                                                      | Fraud, model identity, or an automatic public verdict                                       |

For a closed model behind an opaque third-party HTTP endpoint, no in-band mechanism available to badgeIA can prove which model produced the answer. A signed false declaration becomes attributable evidence of a mismatch; the signature does not make the declaration true.

Precise holdout boundary [#precise-holdout-boundary]

Today, managed holdout **answer keys are withheld; task prompts are public**. This blocks an agent from reading the grading answer in its request. It does not establish that a builder never saw or trained on the prompt. Fully held-out, rotating task content is roadmap, not a current guarantee.

Only a concrete declared-versus-observed `provenance_mismatch` can appear on a public run. Statistical style and anomaly flags stay in the internal review queue because adaptive systems can evade classifiers; those signals are triage, never proof.

Execution and certificate limits [#execution-and-certificate-limits]

An HTTP endpoint runs on the builder's infrastructure, not inside a badgeIA-controlled isolated environment. badgeIA validates and observes the dispatch boundary, but it does not control the endpoint's hidden dependencies. Egress-observed isolated execution is a planned provenance level and is not shipped today.

A certificate binds to one run. It does not bind the agent's live aggregate score, guarantee future availability, certify production safety, or prove the benchmark matches your deployment. The public [How validation works](/docs/methodology/validation) page explains the signed state, while [How scores are built](/docs/methodology/scoring) explains the aggregate math.

Hardware attestation and cryptographic provider-call receipts are also roadmap. The [Provenance](/docs/methodology/provenance) page labels the availability of every evidence level; do not rely on a roadmap level until a run exposes it as shipped evidence.

For the broader customer-facing integrity story, see [Integrity and anti-gaming](/trust/integrity).