Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ Versioning: [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

### Added

- **End-to-end fixture consumption for D.x verdict-synthesis paths** ([X402-47](https://vahdatfardin.atlassian.net/browse/X402-47)). New `tests/fixtures/bazaar/captured-responses/` directory with three hand-rolled, self-describing fixtures — each capturing the well-known + challenge + discovery responses needed to exercise one v0.3.2 D.x failure mode through `runBazaarCheck` end-to-end without live HTTP. Initial set: `d2-missing-propagation.json` (manifest correct, indexer surfaces blank resource → `metadata_propagation: missing` → `upstream_issue`; the @zev / TheRoosters / GM pattern), `d3-processing-stuck.json` (facilitator settled but CDP discovery returns 0 resources → `indexer_state: processing` → `upstream_stuck`; Max's polyodds.bet pattern, canonical #2207), `d5-body-discovery.json` (extensions.bazaar uses BodyDiscoveryExtension shape — was false-positive `implementation_issue` pre-v0.3.2, now passes cleanly with `looks_correct`; AsaiShota's test-echo-cdp + 0xdespot's hyperD pattern). Self-describing JSON schema (`{scenario, input, mocks, expected}`); new integration test `tests/integration/bazaar-check-captured-responses.test.ts` iterates over every fixture, builds a mock fetcher dispatching by URL pattern, runs the full pipeline, asserts verdict + per-check `detail` facets match via dotted-path extraction. Adding a new fixture is one JSON file — the harness picks it up automatically. **Scope cut vs original ticket:** the original X402-47 AC called for consuming 6 named contributor fixtures (TomSmart cdp-mature, AsaiShota test-echo-cdp, evanatpizzarobot tensorfeed, 0xdespot hyperd, hypeprinter007 anchor-x402 multi-rail). Only TomSmart's first fixture is in the repo today; the other 5 are pre-committed but not delivered. This PR ships the **fixture-consumption infrastructure** with synthetic captures so the v0.3.2 audit gate has D.x coverage NOW; real contributor fixtures wire in as they arrive (TomSmart Sunday drop next).
- **D.3 indexer-state probe + `upstream_stuck` composite verdict** ([X402-46](https://vahdatfardin.atlassian.net/browse/X402-46), ADR-004 Pillar 1 + Pillar 3). The existing `indexing` check now emits a `detail.indexer_state` facet with four values: `indexed` (≥1 resource in CDP discovery), `processing` (404 or empty resources — Max's polyodds.bet case + the canonical #2207 indexer-state-stuck cluster), `unknown` (HTTP error / non-JSON / network failure), and `not_applicable_non_cdp` (operator declared a non-CDP facilitator in their well-known manifest; CDP discovery is not the canonical indexer for this service per ADR-004 Pillar 3). When `indexer_state: processing` fires the verdict synthesizer returns a **new composite verdict `upstream_stuck`** — distinct from generic `upstream_issue` because the root cause is known (facilitator settled, indexer queue stalled). `upstream_stuck` rolls up to exit code 3 (preserves CI contract per ADR-004); verdict prose + JSON facets carry the granularity. New shared `src/bazaar/facilitator-detect.ts` helper — manifest-claim-based detection (`extensions.bazaar.facilitator` value), used by both D.2 (propagation) and D.3 (indexing). When facilitator is non-CDP, both D.2 and D.3 short-circuit to `not_applicable_non_cdp` BEFORE the network probe — fast, no HTTP cost, no false-positive. **v0.3.3+ deferred:** `processing_fresh` vs `processing_stale` distinction (requires settle-timestamp data we don't collect without driving live settles; future work via operator-supplied evidence flag).
- **D.2 metadata propagation diff** ([X402-45](https://vahdatfardin.atlassian.net/browse/X402-45)). New 5th check in `bazaar-check` results: `propagation`. Queries CDP discovery for the service's `payTo` and diffs the rendered fields (`name`, `description`) against what the operator declared in `/.well-known/x402`. Surfaces the @zev / TheRoosters / GM pain pattern: manifest is correctly shaped, indexer dropped fields, listing renders blank — your implementation is fine, the gating step is upstream. Emits `detail.metadata_propagation` with one of four states: `ok` (all diffed fields match — status pass), `partial` (some drift — status info + diff array showing per-field mismatches), `missing` (indexer surfaces none of the manifest's declared fields, matching the canonical #2207 indexer-state cluster — status info), `unknown` (defensive default when no manifest is available via `--endpoint` mode, no `payTo` is extractable, or the discovery query failed — status pass). New module `src/bazaar/propagation.ts`; pure-function `computePropagationStatus()` helper exported for downstream re-use. Diff scope intentionally narrow (`name`, `description`) — extends when new pain shapes surface. **Facilitator-aware semantics layered separately:** D.3 (X402-46) will add the explicit `not_applicable_non_cdp` state per ADR-004 Pillar 3; D.2 stays narrow and returns `unknown` for non-CDP services rather than asserting attribution it can't yet determine.
- **`bazaar-check --log json` is now a documented public API contract** ([X402-44](https://vahdatfardin.atlassian.net/browse/X402-44), ADR-004 Pillar 2). Three deliverables landed together: the contract doc at [`src/bazaar/json-api.md`](./src/bazaar/json-api.md) (envelope shape + per-check `detail` keys + verdict discriminator + stability rules + regeneration workflow), a frozen exemplar at [`tests/fixtures/bazaar/json-api-snapshot.json`](./tests/fixtures/bazaar/json-api-snapshot.json), and a snapshot test at [`tests/integration/bazaar-check-json-api.test.ts`](./tests/integration/bazaar-check-json-api.test.ts) (6 tests catch field renames, removals, additions, and reordering against the exemplar). **Versioning rule** committed: additive changes (new optional fields, new check names, new `verdict.kind` values, new `detail.*` keys) ship in MINOR versions with a `### JSON API` CHANGELOG entry; shape-breaking changes (renames, removals, type changes, fixed-position reordering) require a MAJOR version + integrator notice. Exit-code contract preserved across all minor versions (D.3's `upstream_stuck` will roll up to exit code 3, not a new code). README "JSON API" section added under the `bazaar-check` documentation; `CONTRIBUTING.md` "JSON API discipline" section added with a PR self-check. TomSmart_ai's mapper-integration is the named consumer this contract is committed to.
Expand Down
69 changes: 69 additions & 0 deletions tests/fixtures/bazaar/captured-responses/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Captured-response fixtures for bazaar-check end-to-end coverage

X402-47 — hand-rolled fixtures that exercise the D.2 / D.3 / D.5 verdict
facets end-to-end via `runBazaarCheck` without needing live HTTP.

Each `.json` file in this directory is **self-describing**: it carries
the input config + the three mock responses (well-known, challenge,
discovery) + the expected verdict + key facet values. The integration
test at [`tests/integration/bazaar-check-captured-responses.test.ts`](../../../integration/bazaar-check-captured-responses.test.ts)
iterates over every fixture and runs the full check pipeline against
the captured responses.

## Why captured responses, not live URLs

The pre-committed contributor fixtures (TomSmart's cdp-mature, AsaiShota's
test-echo-cdp, evanatpizzarobot's TensorFeed, 0xdespot's hyperD,
hypeprinter007's anchor-x402) are real production services. Running
bazaar-check against them live is slow, flaky, and gated on those
services being up. The fixture bed in `production-set/` captures their
URL lists + expected-at-capture verdicts as schema-only structural
contracts.

This directory takes the opposite approach: tiny **hand-rolled**
captured responses that exercise specific verdict-synthesis paths
deterministically. Each fixture isolates one D.x failure mode.

When live contributor fixtures arrive (Sunday 2026-05-24 TomSmart drop;
others rolling), they wire into `production-set/` as URL+verdict pairs
for structural assertion, while this `captured-responses/` directory
keeps the end-to-end behavioral coverage hermetic.

## Fixture schema

```jsonc
{
"$comment": "Short one-line description of what this fixture exercises",
"scenario": "<stable-slug>", // matches the file basename
"input": {
"serviceUrl": "https://...", // service URL passed to bazaar-check
"chain": "base-sepolia" | "base"
},
"mocks": {
"well-known": { "status": 200, "body": {...} }, // response from /.well-known/x402
"challenge": { "status": 402, "body": {...} }, // response from the service URL
"discovery": { "status": 200, "body": {...} } // response from CDP discovery
},
"expected": {
"verdict": "looks_correct" | "implementation_issue" | "upstream_issue" | "upstream_stuck",
"exitCode": 0 | 2 | 3,
"facets": { // optional; per-check detail facets to assert
"<check>.<facet>": <value>
}
}
}
```

## Existing fixtures

| File | Exercises |
|---|---|
| `d2-missing-propagation.json` | D.2 `metadata_propagation: missing` (the @zev / TheRoosters / GM pattern) |
| `d3-processing-stuck.json` | D.3 `indexer_state: processing` → `upstream_stuck` verdict (Max's polyodds.bet pattern) |
| `d5-body-discovery.json` | D.5 BodyDiscoveryExtension variant → no false-positive `implementation_issue` (AsaiShota's test-echo-cdp pattern) |

## Adding a new fixture

1. Drop a `<scenario-slug>.json` here following the schema above
2. Re-run `pnpm test tests/integration/bazaar-check-captured-responses.test.ts`; the new file is picked up automatically
3. If the new fixture exercises a verdict combination the harness doesn't know how to assert yet, extend the harness in `tests/integration/bazaar-check-captured-responses.test.ts`
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
{
"$comment": "D.2 — manifest correct, indexer surfaces blank resource → metadata_propagation: missing. Models @zev / TheRoosters / GM pattern.",
"scenario": "d2-missing-propagation",
"input": {
"serviceUrl": "https://api.fixture-d2.example.test",
"chain": "base-sepolia"
},
"mocks": {
"well-known": {
"status": 200,
"body": {
"name": "Fixture D.2 Service",
"description": "Real description in the manifest",
"accepts": [],
"extensions": {
"bazaar": {
"name": "Fixture D.2 Service",
"description": "Real description in the manifest"
}
}
}
},
"challenge": {
"status": 402,
"body": {
"x402Version": 1,
"accepts": [
{
"scheme": "exact",
"network": "base-sepolia",
"maxAmountRequired": "1000",
"resource": "https://api.fixture-d2.example.test",
"payTo": "0xd2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2",
"asset": "0x036CbD53842c5426634e7929541eC2318f3dCF7e",
"maxTimeoutSeconds": 300
}
],
"extensions": {
"bazaar": {
"name": "Fixture D.2 Service",
"description": "Real description in the manifest"
}
}
}
},
"discovery": {
"status": 200,
"body": {
"resources": [{ "id": "blank-card-resource" }]
}
}
},
"expected": {
"verdict": "upstream_issue",
"exitCode": 3,
"facets": {
"propagation.metadata_propagation": "missing",
"indexing.indexer_state": "indexed"
}
}
}
60 changes: 60 additions & 0 deletions tests/fixtures/bazaar/captured-responses/d3-processing-stuck.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
{
"$comment": "D.3 — facilitator settled but CDP discovery returns 0 resources for the payTo → indexer_state: processing → upstream_stuck verdict. Models Max's polyodds.bet pattern (canonical #2207 cluster).",
"scenario": "d3-processing-stuck",
"input": {
"serviceUrl": "https://api.fixture-d3.example.test",
"chain": "base-sepolia"
},
"mocks": {
"well-known": {
"status": 200,
"body": {
"name": "Fixture D.3 Service",
"description": "Service whose indexer queue is stuck on processing",
"accepts": [],
"extensions": {
"bazaar": {
"name": "Fixture D.3 Service",
"description": "Service whose indexer queue is stuck on processing"
}
}
}
},
"challenge": {
"status": 402,
"body": {
"x402Version": 1,
"accepts": [
{
"scheme": "exact",
"network": "base-sepolia",
"maxAmountRequired": "1000",
"resource": "https://api.fixture-d3.example.test",
"payTo": "0xd3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3",
"asset": "0x036CbD53842c5426634e7929541eC2318f3dCF7e",
"maxTimeoutSeconds": 300
}
],
"extensions": {
"bazaar": {
"name": "Fixture D.3 Service",
"description": "Service whose indexer queue is stuck on processing"
}
}
}
},
"discovery": {
"status": 200,
"body": {
"resources": []
}
}
},
"expected": {
"verdict": "upstream_stuck",
"exitCode": 3,
"facets": {
"indexing.indexer_state": "processing"
}
}
}
72 changes: 72 additions & 0 deletions tests/fixtures/bazaar/captured-responses/d5-body-discovery.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
{
"$comment": "D.5 — extensions.bazaar uses BodyDiscoveryExtension shape (info.input/output/schema). Pre-v0.3.2 this was false-positive `implementation_issue`; now it passes the challenge check and verdicts cleanly. Models AsaiShota's test-echo-cdp + 0xdespot's hyperD pattern.",
"scenario": "d5-body-discovery",
"input": {
"serviceUrl": "https://api.fixture-d5.example.test/api/v1/data",
"chain": "base-sepolia"
},
"mocks": {
"well-known": {
"status": 200,
"body": {
"name": "Fixture D.5 Body-Discovery Service",
"description": "API-style service declaring body-discovery via @x402/extensions",
"accepts": [],
"extensions": {
"bazaar": {
"info": {
"input": { "type": "object", "properties": { "q": { "type": "string" } } },
"output": { "type": "object", "properties": { "result": { "type": "string" } } }
},
"schema": { "version": "2.11.0" }
}
}
}
},
"challenge": {
"status": 402,
"body": {
"x402Version": 1,
"accepts": [
{
"scheme": "exact",
"network": "base-sepolia",
"maxAmountRequired": "1000",
"resource": "https://api.fixture-d5.example.test/api/v1/data",
"payTo": "0xd5d5d5d5d5d5d5d5d5d5d5d5d5d5d5d5d5d5d5d5",
"asset": "0x036CbD53842c5426634e7929541eC2318f3dCF7e",
"maxTimeoutSeconds": 300
}
],
"extensions": {
"bazaar": {
"info": {
"input": { "type": "object", "properties": { "q": { "type": "string" } } },
"output": { "type": "object", "properties": { "result": { "type": "string" } } }
},
"schema": { "version": "2.11.0" }
}
}
}
},
"discovery": {
"status": 200,
"body": {
"resources": [
{
"name": "Fixture D.5 Body-Discovery Service",
"description": "API-style service declaring body-discovery via @x402/extensions"
}
]
}
}
},
"expected": {
"verdict": "looks_correct",
"exitCode": 0,
"facets": {
"indexing.indexer_state": "indexed",
"propagation.metadata_propagation": "ok"
}
}
}
Loading