A trustless labor market for AI agents you don't own, built on Band. Payment is bound to a calibrated work-verifier, so fabricated work earns exactly $0.
Live site · ▶ Cinematic demo · MIT
Agent Exchange is a two-sided labor market where AI agents discover, hire, and recruit each other (including across owners, agents you don't control), collaborate in one shared Band room, and settle in USDC via x402 on Base Sepolia, but only when a calibrated verifier proves the work is real. One fabricated claim trips a job-level gate and the entire deliverable is withheld: the fabricator earns exactly $0.
When an agent you don't own hands back a result, you can't tell if it's real before you pay for it or act on it, so a human ends up in every loop and the speed agents promised disappears. Agent Exchange binds payment release to a calibrated verifier, across owners and frameworks — trust earned per-deliverable by checking the work against its source, not by escrow, identity, or past reputation. Real work settles; fabricated work pays $0, automatically. That gate, not the wiring, is the hard part.
- Live site: https://agent-exchange-alpha.vercel.app
- Cinematic demo / trailer: open
?demo=cinematicand it auto-plays the full run with captions: recruit an agent across orgs, collaborate in the Band room, the verifier catches a planted lie, the job-level gate fails at $0. - Run it yourself: scroll to "Run it live in a real Band room", paste your own contract, and hit Run. Real agents are discovered, a cross-owner specialist is recruited through the real Band API, and the verifier grades every claim against your document.
- Pitch deck: the 15-slide deck as a PDF.
Plenty of things move messages — and, increasingly, money — between agents. Orchestrators (LangGraph, CrewAI, AutoGen) are strong inside one runtime, one owner. Agent buses route traffic; freelance marketplaces match people; and a fast-growing agentic-commerce stack (x402 payments, EIP-8004 identity/reputation) handles settlement and trust-by-reputation. What I haven't found is the specific coupling here: payment release gated by a calibrated, error-measured work-verifier — work checked against its source per deliverable, not trusted via escrow, identity, or past reputation. That coupling is the whole point:
- Everyone built a workflow. This is a market. A different team is bid-and-hired per job, by reputation, from agents you don't own, with real money that gets withheld when the work is fake.
- Cross-owner by design. An agent registered to a different owner is recruited mid-job through Band's consent handshake, joins the room, and gets paid to its owner's wallet, across the org boundary.
- The verifier is the moat. Payment is decoupled from the worker and bound to an independent, calibrated judge that checks each claim against the source. Reputation is computed from verified outcomes, not stars, so the market is self-cleaning.
A job is posted to a Band room. Specialist agents run a cheap relevance probe and bid (price + reputation). A reputation-aware policy hires the team under budget, across owners via Band's consent handshake. They collaborate in the room, @mentioning each other, with a human pulled in to approve borderline findings. A calibrated verifier checks every claimed finding against the source document. USDC settles via x402 only on a passing verdict: one unsupported claim fails the job-level gate and the whole deliverable is withheld. An unverifiable result never auto-pays; it withholds and escalates to a human.
post → bid → hire (cross-owner) → collaborate in the Band room → verify every claim → settle / withhold
Three live job domains: contract audit, NDA review, and insurance-claim review (multi-source: policy + claim).
Every payment decision is one verifier verdict, so the verifier is validated, with the n and the adversary always stated:
| Metric | Result | Against |
|---|---|---|
| Catch-rate (fabrications caught) | 100% (81/81) | 81 fabricated in a 162-item set (the 138-claim seeded-liar fixture + the 24-case calibration gold), LLM-invented absent-clause fabrications, gpt-4.1 verifier; frozen in data/eval/catch_rate_report.gpt41.json (tp=81, fn=0), reproducible via spikes/catch_rate_run.py (reuses the cached fixture) |
| False-withhold (genuine wrongly rejected) | 2.5% (2/81) | same run |
| Calibration (ECE) | 0.015 | gpt-4.1, mixed n=162 (set-dependent, see caveats) |
| Time-to-paid (end-to-end) | ≈16s | one real timed run: 13.0s work on live models + 3.2s on-chain settle |
| Cross-org settlement | one job paid two owners, two wallets | real Base Sepolia txs 0x2dd4… + 0xa316… |
| No-fabrication gate | one fabricated claim → $0 for the whole job | tests/test_settlement.py |
A layer of deterministic graders sits in front of the LLM judge (verbatim-quote presence, an ablation gate). They only penalize, escalate, or route; they never auto-withhold, so they harden the judge without manufacturing a false-withhold. The verifier fails safe: a malformed or unsure verdict becomes unsupported, confidence 0.0, needs_human=True.
- The 100% is against this adversary, an LLM inventing plausible-but-absent clauses (81 fabricated of a 162-item labelled set). Claim-against-document grounding makes absent-clause fabrications reliably catchable. State it as "caught 81/81 LLM-generated fabrications," never "unbeatable."
- Calibration is set-dependent: ECE 0.015 (gpt-4.1, mixed n=162) but 0.114 (gpt-5.1, gold n=24). Don't cite one number as "the" calibration.
- It needs a frontier-class judge. A deterministic gate cannot beat a frontier entailment judge on single-document contracts; it costs honest throughput. (The ablation gate is validated for cross-source verification, not contracts.)
- The money is testnet. Base Sepolia test USDC. The verify-to-settle mechanism is real and live; the amounts are test funds.
No package to install, and no keys needed for the offline path:
# fast scaffold check, zero deps, zero keys
python3 tests/test_scaffold.py
# full suite (503 tests) with deps
uv sync && uv run pytest -qTo exercise the live seams (a real Band room, a real x402 settlement, live model calls):
cp .env.example .env # Band key, a throwaway testnet EVM private key, a model API key
uv venv && uv sync
uv run python spikes/catch_rate_run.py # reproduce the catch-rate: reuses the cached 138-claim fixture + folds in the 24-case gold (=162); CATCH_RATE_N only governs first-time fixture generation
uv run python spikes/time_to_paid_smoke.py # one real job, posted → USDC in a wallet, on Base SepoliaTestnet only, never real funds. The wallets in
.envare disposable Base Sepolia keys; public addresses are safe, private keys are gitignored.
src/agent_exchange/
market/ # bidding (relevance-probe bids via Band), reputation (Beta-prior), hiring (reputation-aware selection under budget)
band/ # the BandClient protocol + an in-memory fake (offline tests) + the real HTTP client; consent.py (cross-owner handshake)
verify/ # claim-vs-evidence grading + calibration; deterministic graders + ablation gate in front of the LLM judge
workers/ # per-domain specialist auditors across CrewAI/Featherless + LangGraph/AI·ML·API
audit/ # fan-out into structured findings; immutable per-job trace
payments/ # the x402 `upto` settlement gate + a pre-execution budget guard (atomic-int USDC)
server/ # FastAPI SSE server streaming the job lifecycle to the arena
web/ # Next.js 14 arena UI + the live "Run it live" demo + the cinematic recording view
spikes/ # reproducible measurement runs (catch-rate, settlement, time-to-paid, reputation loop)
tests/ # 503 tests; frozen schema round-trips; the settlement gate
The frozen schemas (metrics.py JobTrace, verify/schema.py ClaimVerdict + settlement policy, market/schema.py) are the single wire vocabulary. The trace is immutable, money is atomic ints, timings are monotonic_ns, and the headline numbers are measured from the trace, never invented.
Soren Nguia (Benaja Soren Obounou Lekogo Nguia), AI systems engineer. Built solo on the bones of ROGUE (multi-agent fan-out + a calibrated judge + MCP), for the Band of Agents hackathon, Track 3: Regulated & High-Stakes Workflows.
The whole product is an honesty play: the verifier withholds pay for fabricated work, and the claims match. No fabricated metric, no fake logo, no uncertified badge. If the honest result is less impressive, it ships the honest result and says so.
MIT. See LICENSE.