badgeIA

Getting started

Screen your first agent

The shortest path to a real score — one file with no dependencies, one tunnel, two API calls.

View Markdown

This is the fastest way to get a real Badge score: Badge calls code running on your machine over the public internet, and scores what came back.

You will write one Python file with no dependencies to install, expose it, and screen it. Budget about 10 minutes end to end. Most of that is creating the account and starting the tunnel; the screening itself finishes in under 5 minutes.

If you want the score without exposing anything, screen without an endpoint instead — it is faster, but Badge never calls your code, so the result is marked Simulated and cannot earn a verified ✓.

What you need

  • Python 3. Any version from the last few years. Nothing to pip install.
  • cloudflared. brew install cloudflared, or see the tunnel guide for Linux.
  • curl and jq. Every command below uses them. Install jq with brew install jq or sudo apt install jq.
  • A Badge account. The Free plan is enough. It includes one agent and 30 runs a month, and this guide uses your one agent and 6 runs.

Already registered an agent, for example in the quickstart? A Free account holds one active agent, so registering a second one fails with 402 and the message "You've reached the 1-agent limit on the free plan. Upgrade to Pro to add more agents." (with the commands below, curl prints The requested URL returned error: 402 and BADGE_AGENT_ID stays empty). Either delete the old agent first, or upgrade to Pro. Deleting frees the slot at once; the old agent's runs and certificates are kept. Find the old agent's id in the app (open it from Agents; the id is the last part of the address), then, after the export lines in Step 3:

curl -fsS -X DELETE "$BADGE_API/api/v1/agents/<old-agent-id>" \
  -H "X-API-Key: $BADGE_API_KEY" -w '%{http_code}\n'   # 204

Step 1: Write the agent

Badge screens an agent by sending it one unauthenticated HTTPS POST per task, and reading a JSON object with an output key back. That is the whole contract. Save this as agent.py:

"""A complete Badge-screenable agent. Standard library only."""

import json
from http.server import BaseHTTPRequestHandler, HTTPServer


def answer(prompt: str) -> str:
    """Replace this with your agent. Task prompt in, answer out."""
    return f"echo: {prompt.strip()[:200]}"


class Agent(BaseHTTPRequestHandler):
    def do_POST(self) -> None:
        body = self.rfile.read(int(self.headers.get("Content-Length") or 0))
        prompt = json.loads(body or b"{}").get("prompt", "")
        output = answer(prompt)
        self._send({
            "output": output,
            "input_tokens": len(prompt) // 4,
            "output_tokens": len(output) // 4,
            "total_cost_usd": 0.0,
        })

    def do_GET(self) -> None:
        self._send({"status": "ok"})

    def _send(self, payload: dict) -> None:
        encoded = json.dumps(payload).encode()
        self.send_response(200)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(encoded)))
        self.end_headers()
        self.wfile.write(encoded)

    def log_message(self, *args: object) -> None:
        print(f"{self.command} {self.path}")


HTTPServer(("0.0.0.0", 8080), Agent).serve_forever()

Run it, and prove it answers the way Badge will ask:

python3 agent.py &

curl -sS -X POST http://127.0.0.1:8080/execute \
  -H 'Content-Type: application/json' \
  -d '{"task_id":"local","prompt":"Say hello","max_tokens":64}'
{"output": "echo: Say hello", "input_tokens": 2, "output_tokens": 3, "total_cost_usd": 0.0}

Three things about this file are deliberate, and each one removes a step you would otherwise have to debug:

  • It answers POST on every path, so it does not matter whether you register /execute or the bare tunnel URL. Registering the wrong path is the most common first-run failure.
  • It reports total_cost_usd: 0.0. Omitting the field makes Badge estimate your cost from token counts at GPT-4o reference pricing. This agent calls no model, so zero is the honest number and your number wins.
  • It answers from the prompt alone. Badge never sends the grading answer, for any task; it compares your output with the reference on its side.

Step 2: Put it on the internet

Badge screens from its own servers, so localhost is unreachable. A tunnel gives you a public HTTPS URL that forwards to your local port:

cloudflared tunnel --url http://localhost:8080

This publishes a port on your laptop to the entire internet, and anyone with the URL can invoke your agent. That is fine for this walkthrough — a placeholder agent on a dedicated port, with the tunnel stopped when you finish. Before you point one at anything real, read what a tunnel exposes. Do not run tunnelling software on a managed work device.

Copy the https://....trycloudflare.com URL it prints. Leave both terminals running.

Step 3: Register it and screen it

Create your account, then open Settings → API Keys and create a key with read + write scopes. The full ask_ key is shown once.

export BADGE_API=https://api.badgeia.com
export BADGE_APP=https://badgeia.com
read -r -s -p "Paste your ask_ key: " BADGE_API_KEY; printf '\n'
read -r -p "Tunnel URL: " TUNNEL

Register the tunnel URL as an http_endpoint agent, then run one public Suite three times. Each public Suite has two tasks, so that is 6 runs. Three rounds is not padding: the latency axis needs at least 5 counted runs and robustness needs at least 2 runs of the same task, or both come back null. (What counts as a screen.)

export BADGE_AGENT_ID=$(curl -fsS -X POST "$BADGE_API/api/v1/agents" \
  -H "X-API-Key: $BADGE_API_KEY" -H 'Content-Type: application/json' \
  -d "$(jq -n --arg endpoint "$TUNNEL/execute" '{
    name: "my-first-agent-\(now|floor)",
    description: "Standard-library HTTP agent from the Badge quickstart.",
    connection_mode: "http_endpoint",
    payload_schema: "badge_native",
    endpoint: $endpoint
  }')" | jq -r '.id')

[ -n "$BADGE_AGENT_ID" ] || printf 'Registering the agent failed: see the error above.\n' >&2

SUITE=$(curl -fsS "$BADGE_API/api/v1/marketplace/suites?page_size=25" |
  jq -r '[.items[] | select(.task_count >= 2 and .task_count <= 5)][0].id // empty')
[ -n "$SUITE" ] || printf 'No small public Suite is available right now.\n' >&2

for ROUND in 1 2 3; do
  curl -fsS -X POST "$BADGE_API/api/v1/runs/batch" \
    -H "X-API-Key: $BADGE_API_KEY" -H 'Content-Type: application/json' \
    -d "$(jq -n --arg agent_id "$BADGE_AGENT_ID" --arg suite_id "$SUITE" \
      '{agent_id: $agent_id, suite_id: $suite_id}')" | jq -c '[.[] | {id, status}]'
done

Watch your agent's terminal while this runs. You will see Badge arrive — one POST per task, per round.

Agent names are unique across all of Badge, not just your account. The \(now|floor) suffix above keeps you from colliding; without it a second reader gets 409 An agent named '...' is already registered.

Step 4: Read the score

Runs are asynchronous, and with this agent they finish within a minute or two. This loop waits until none is still pending (it gives up after four minutes):

for _ in $(seq 1 120); do
  PENDING=$(curl -fsS "$BADGE_API/api/v1/runs?agent_id=$BADGE_AGENT_ID&page_size=100" |
    jq '[.items[] | select(.status != "completed" and .status != "failed" and .status != "timeout")] | length')
  [ "$PENDING" -eq 0 ] && break
  sleep 2
done

Then look at what actually happened on the wire:

curl -fsS "$BADGE_API/api/v1/runs?agent_id=$BADGE_AGENT_ID&page_size=100" \
  -H "X-API-Key: $BADGE_API_KEY" |
  jq '[.items[] | {success, execution_mode, latency_ms, error_class}]'

curl -fsS "$BADGE_API/api/v1/agents/$BADGE_AGENT_ID/fitness" \
  -H "X-API-Key: $BADGE_API_KEY" |
  jq '{canonical_composite_score, correctness, latency, cost, tool_efficiency, robustness}'

printf 'Your agent: %s/agents/%s\n' "$BADGE_APP" "$BADGE_AGENT_ID"

Every row should read execution_mode: "live_endpoint". That is Badge confirming it really called your machine, and it is what earns the verified ✓ that a simulated run cannot.

Your correctness score will be poor, and that is correct. The answer() function echoes the prompt back; it is a placeholder, not an agent. What you have proved is the whole loop — contract, exposure, dispatch, scoring — with one function left to fill in. Replace answer() with your real agent and screen it again.

When you are done, stop cloudflared and stop agent.py. A tunnel you forget about is a standing public door into your laptop.

Step 5: Put the badge in your README

Your agent already has a live badge. Print a Markdown snippet with your real agent id:

printf '[![badgeIA score](%s/api/v1/badges/%s.svg)](%s/agents/%s)\n' \
  "$BADGE_API" "$BADGE_AGENT_ID" "$BADGE_APP" "$BADGE_AGENT_ID"

It shows your score and a ✓ Verified, ◐ Simulated or ○ Unverified mark, and it follows your score as you screen again. Put your badge in your README explains each state, gives the HTML form, and shows the run badge that links to one certificate.

Where to go next