badgeIA

Getting started

Screen your first agent

The shortest path to a real score — one file with no dependencies, one tunnel, two API calls.

View Markdown

This is the fastest way to get a real Badge score: Badge calls code running on your machine over the public internet, and scores what came back.

You will write one Python file with no dependencies to install, expose it, and screen it. Budget about 10 minutes, most of it creating the account.

If you want the score without exposing anything, screen without an endpoint instead — it is faster, but Badge never calls your code, so the result is marked Simulated and cannot earn a verified ✓.

What you need

  • Python 3. Any version from the last few years. Nothing to pip install.
  • cloudflared. brew install cloudflared, or see the tunnel guide for Linux.
  • A Badge account. Free tier is enough.

Step 1: Write the agent

Badge screens an agent by sending it one unauthenticated HTTPS POST per task, and reading a JSON object with an output key back. That is the whole contract. Save this as agent.py:

"""A complete Badge-screenable agent. Standard library only."""

import json
from http.server import BaseHTTPRequestHandler, HTTPServer


def answer(prompt: str) -> str:
    """Replace this with your agent. Task prompt in, answer out."""
    return f"echo: {prompt.strip()[:200]}"


class Agent(BaseHTTPRequestHandler):
    def do_POST(self) -> None:
        body = self.rfile.read(int(self.headers.get("Content-Length") or 0))
        prompt = json.loads(body or b"{}").get("prompt", "")
        output = answer(prompt)
        self._send({
            "output": output,
            "input_tokens": len(prompt) // 4,
            "output_tokens": len(output) // 4,
            "total_cost_usd": 0.0,
        })

    def do_GET(self) -> None:
        self._send({"status": "ok"})

    def _send(self, payload: dict) -> None:
        encoded = json.dumps(payload).encode()
        self.send_response(200)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(encoded)))
        self.end_headers()
        self.wfile.write(encoded)

    def log_message(self, *args: object) -> None:
        print(f"{self.command} {self.path}")


HTTPServer(("0.0.0.0", 8080), Agent).serve_forever()

Run it, and prove it answers the way Badge will ask:

python3 agent.py &

curl -sS -X POST http://127.0.0.1:8080/execute \
  -H 'Content-Type: application/json' \
  -d '{"task_id":"local","prompt":"Say hello","max_tokens":64}'
{"output": "echo: Say hello", "input_tokens": 2, "output_tokens": 3, "total_cost_usd": 0.0}

Three things about this file are deliberate, and each one removes a step you would otherwise have to debug:

  • It answers POST on every path, so it does not matter whether you register /execute or the bare tunnel URL. Registering the wrong path is the most common first-run failure.
  • It reports total_cost_usd: 0.0. Omitting the field makes Badge estimate your cost from token counts at GPT-4o reference pricing. This agent calls no model, so zero is the honest number and your number wins.
  • It never reads expected_output. Badge echoes the answer key for non-secret tasks. Returning it is self-grading, it is absent exactly when the task matters, and suites of secret tasks exist to catch it.

Step 2: Put it on the internet

Badge screens from its own servers, so localhost is unreachable. A tunnel gives you a public HTTPS URL that forwards to your local port:

cloudflared tunnel --url http://localhost:8080

This publishes a port on your laptop to the entire internet, and anyone with the URL can invoke your agent. That is fine for this walkthrough — a placeholder agent on a dedicated port, with the tunnel stopped when you finish. Before you point one at anything real, read what a tunnel exposes. Do not run tunnelling software on a managed work device.

Copy the https://....trycloudflare.com URL it prints. Leave both terminals running.

Step 3: Register it and screen it

Create your account, then open Settings → API Keys and create a key with read + write scopes. The full ask_ key is shown once.

export BADGE_API=https://api.badgeia.com
export BADGE_APP=https://badgeia.com
read -r -s -p "Paste your ask_ key: " BADGE_API_KEY; printf '\n'
read -r -p "Tunnel URL: " TUNNEL

Register the tunnel URL as an http_endpoint agent, then run one small suite three times. Three rounds is not padding: the latency axis needs at least 5 counted runs and robustness needs the same task run more than once, or both come back null.

export BADGE_AGENT_ID=$(curl -fsS -X POST "$BADGE_API/api/v1/agents" \
  -H "X-API-Key: $BADGE_API_KEY" -H 'Content-Type: application/json' \
  -d "$(jq -n --arg endpoint "$TUNNEL/execute" '{
    name: "my-first-agent-\(now|floor)",
    description: "Standard-library HTTP agent from the Badge quickstart.",
    connection_mode: "http_endpoint",
    payload_schema: "badge_native",
    endpoint: $endpoint
  }')" | jq -r '.id')

SUITE=$(curl -fsS "$BADGE_API/api/v1/marketplace/suites?page_size=25" |
  jq -r '[.items[] | select(.task_count >= 2 and .task_count <= 5)][0].id // empty')

for ROUND in 1 2 3; do
  curl -fsS -X POST "$BADGE_API/api/v1/runs/batch" \
    -H "X-API-Key: $BADGE_API_KEY" -H 'Content-Type: application/json' \
    -d "$(jq -n --arg agent_id "$BADGE_AGENT_ID" --arg suite_id "$SUITE" \
      '{agent_id: $agent_id, suite_id: $suite_id}')" | jq -c '[.[] | {id, status}]'
done

Watch your agent's terminal while this runs. You will see Badge arrive — one POST per task, per round.

Agent names are unique across all of Badge, not just your account. The \(now|floor) suffix above keeps you from colliding; without it a second reader gets 409 An agent named '...' is already registered.

Step 4: Read the score

Runs are asynchronous. Wait for them, then look at what actually happened on the wire:

curl -fsS "$BADGE_API/api/v1/runs?agent_id=$BADGE_AGENT_ID&page_size=100" \
  -H "X-API-Key: $BADGE_API_KEY" |
  jq '[.items[] | {success, execution_mode, latency_ms, error_class}]'

curl -fsS "$BADGE_API/api/v1/agents/$BADGE_AGENT_ID/fitness" \
  -H "X-API-Key: $BADGE_API_KEY" |
  jq '{canonical_composite_score, correctness, latency, cost, tool_efficiency, robustness}'

printf 'Your agent: %s/agents/%s\n' "$BADGE_APP" "$BADGE_AGENT_ID"

Every row should read execution_mode: "live_endpoint". That is Badge confirming it really called your machine, and it is what earns the verified ✓ that a simulated run cannot.

Your correctness score will be poor, and that is correct. The answer() function echoes the prompt back; it is a placeholder, not an agent. What you have proved is the whole loop — contract, exposure, dispatch, scoring — with one function left to fill in. Replace answer() with your real agent and screen it again.

When you are done, stop cloudflared and stop agent.py. A tunnel you forget about is a standing public door into your laptop.

Where to go next