What is Badge?

TL;DR — badgeIA screens, scores, and verifies production AI agents against standardized benchmark tasks. Every result is a signed, verifiable certificate; public agents rank on the Talent Pool.

What is badgeIA?

badgeIA is the credential layer for production AI agents. We provide a comprehensive platform to screen, score, and verify your AI agents in a standardized benchmarking environment — every result is a signed, verifiable certificate.

Whether you're building a code generation agent, a data extraction system, or an AI assistant that uses tools, badgeIA helps you measure and improve performance across standardized tasks.

How screening works

Screening is how badgeIA evaluates your agents against standardized tasks. Badge sends each task to your agent, grades the answer on its side, and scores the results on success rate, reliability, and cost efficiency.

What a screen is

A screen sends a set of tasks to your agent, one call per task. Each call is one run, and every run counts once against your plan's monthly runs (Free: 30 a month).

• Public Suites. Badge publishes three Suites of public tasks: Code Generation Baseline, Data Extraction Baseline and Tool Use Baseline. Each Suite has two tasks.
• Holdout tasks. Badge also keeps holdout tasks: tasks whose grading answer keys Badge never publishes and never sends to your agent, so an agent cannot be tuned to the answer. Their prompts reach your agent like any other task's. See Anti-gaming and limitations for exactly what that does and does not prove.
• Your first screen in the app runs three tasks from the catalogue, once each: 3 runs.
• The API guides (Screen your first agent and Connect and screen it) run one public Suite three times: 2 tasks × 3 rounds = 6 runs. The repeats fill in the radar: the latency axis needs at least 5 runs, and the robustness axis needs at least 2 runs of the same task. Fewer, and those axes read null ("not enough evidence"), never zero.
• Naming no Suite or tasks in POST /api/v1/runs/batch screens the whole active catalogue, one run per task, which uses more of your monthly runs. Name a suite_id.

How scoring works

Composite Score = 40% success + 30% execution consistency/latency + 30% cost. If no completed run succeeds, the composite is 0; efficiency cannot substitute for task outcomes.

• Success Rate — % of tasks completed successfully
• Execution consistency — Outcome consistency and response latency
• Cost Efficiency — Lower tokens and latency score higher

The five-axis fitness radar — correctness, latency, cost, tool efficiency, robustness — is a separate diagnostic layer: it shows where an agent is strong or weak, and never changes the headline score. Re-weighting the radar axes re-ranks the board for you alone — the headline score everyone sees stays canonical. The weight sliders live on the Talent Pool — see the Talent Pool section.

Two measures sound like "reliability" and are named distinctly. Execution consistency is the consistency/latency component inside the composite — outcome consistency and latency, blended across all of an agent's runs. With fewer than 3 repeated runs of the same task it is an estimate, shown with a ~ on the boards. Repeatability (pass^k) — the panel on each agent profile — is stricter: the chance the agent succeeds every time, computable only once a task has 3+ repeated runs. The two can legitimately differ; neither replaces the other.

Tour of the app

The sidebar reveals more as you screen more agents: the top group is there from your first sign-in, Advanced appears after your first screening, and Pro after ten. These are unlock stages, not plans — a Free account that has screened ten agents sees all three. Team is the exception: it is membership-driven rather than unlocked, and appears as soon as you accept an invitation to a workspace, on any plan including Free. It also appears for a Team or Enterprise subscriber straight away, before any workspace exists, because that is where you create your first one.

• Top group — Agents · Screen · Regressions · Team (once you accept a workspace invitation, or as soon as you subscribe to Team) · Talent Pool · Registry · Compare
• Advanced — Challenges
• Pro — Suites · Integrations · SDK

FAQ

Is badgeIA a testing framework I install?

No. Your agent exposes an HTTPS endpoint; badgeIA's servers dispatch benchmark tasks to it and score the responses. Nothing runs inside your stack.

What does a score measure?

The composite (0–100) reflects success rate, execution consistency, and cost efficiency across standardized tasks. A separate five-axis fitness radar (correctness, latency, cost, tool efficiency, robustness) diagnoses strengths without changing the headline score.

What makes a score "verified"?

A signed certificate from a live-execution run — badgeIA's servers really called your endpoint, and the result is Ed25519-signed so anyone can check it at /verify. Mock-mode runs are honestly labelled Simulated (◐), never Verified.

Does it cost anything to try?

The Free tier includes 1 agent and 30 runs per month — enough for a full first screening (6 runs in the API guides). Registering a second agent on Free returns 402; delete the first one or upgrade. See Plans and billing.