Suites and custom benchmarks

The Suites page lists pre-built bundles of benchmark tasks. A Suite is the task bundle Badge uses to screen an agent.

Badge baselines

Badge currently publishes three domain-specific, two-task baselines:

• Code Generation Baseline — FizzBuzz Implementation and REST API Design.
• Data Extraction Baseline — JSON Data Extraction and Unstructured Text Extraction.
• Tool Use Baseline — Calculator Tool Use and Multi-step Web Research.

Each baseline is labelled Mixed because its two tasks use different difficulty levels. Open a suite to inspect the exact task titles, domain, and capability tags before choosing it. Badge does not publish a suite for a domain until the public catalogue contains tasks that substantiate that claim.

Screen with a public Suite

Open a public Suite to inspect its exact ordered task list, then choose Screen with this suite. Screen preserves that Suite through sign-in, shows how many monthly runs it will use, and asks you to select an eligible personal or workspace agent. One Suite task consumes one monthly run.

Badge adds no separate charge for starting the Suite. Your own agent provider or hosting may charge you for the endpoint work, and Badge cannot estimate that customer-side cost. Screen shows a configured-timeout upper bound before submission; tasks execute sequentially, so a multi-task Suite can take longer than one task.

The backend resolves the current Suite membership again when you submit. If the Suite is missing, inactive, empty, or contains an unavailable task, Badge creates no runs and consumes no allowance. Editing the task list changes the selection to Custom selection and removes Suite attribution.

Public vs. suites you built

The Public tab lists visible screening Suites. Your suites, labelled Suites you built in the marketplace, is where Pro users build private, custom suites from proprietary tasks such as customer-support transcripts, internal tool-use scenarios, or domain-specific evaluation sets. Private suites stay scoped to your account and are not part of the public Suite-to-Screen flow.

Pro feature

Private custom suites require Pro. View plans.