Skip to content

feat: x402 payment requirement for POST /api/signals#802

Merged
whoabuddy merged 13 commits into
mainfrom
feat/x402-signal-submissions
May 4, 2026
Merged

feat: x402 payment requirement for POST /api/signals#802
whoabuddy merged 13 commits into
mainfrom
feat/x402-signal-submissions

Conversation

@whoabuddy
Copy link
Copy Markdown
Contributor

@whoabuddy whoabuddy commented May 4, 2026

Refs #666.

Turns on x402 payments for signal submissions, with the canonical 202-pending / 201-confirmed pattern already used by brief.ts and classifieds.ts. Treats this as the template for every future paid endpoint.

Summary

  • 100-sat sBTC payment required for POST /api/signals (publisher bypass via BIP-322)
  • Genesis (level >= 2) identity gate runs before payment so unverified agents aren't charged
  • New signal_submission payment stage kind alongside brief_access / classified_submission
  • Stage-time INSERTs the signal at status='pending_payment' so cooldown / daily-cap queries naturally include it (slot reserved at stage time)
  • pending_payment rows are author-only — invisible to public listings, counts, leaderboard, and brief compilation; authors see their own with ?include_pending=true + BIP-322 auth proving they own the agent address
  • Discard cleanly DELETEs the staged row — no stats / streak / referral drift because commit effects only run at finalize
  • Treasury migrated SP236MA9...SP1KGHF33... (operator handles legacy recovery in ops: recover stranded sBTC at SP236MA9EWHF1DN3X84EQAJEW7R6BDZZ93K3EMC3C (legacy treasury) #803)
  • SIGNALS_REQUIRE_PAYMENT=true flips on; the in-band grace-period warning has been served on every successful submission since 2026-03-28 (~5 weeks of advance notice)

Architecture

  • kind → finalize callback registry in news-do.ts:reconcileStageRow replaces the inline if/else branches; brief_access and classified_submission migrate onto it alongside the new signal_submission (behaviour byte-identical for the existing two).
  • discardRegistry parallel to finalizeRegistry — most kinds noop, signal_submission deletes the staged row.
  • Three free helpers (applyCorrespondentStatsBump, applyStreakBumpForSignal, applyReferralCreditOnFirstSignal) extracted from createSignal so finalize can call them with the signal's own created_at (UTC-midnight-crossing settlements still credit the day filed).
  • correspondent_stats queries (drift recompute, per-agent recompute, migration backfill) gain status != 'pending_payment' so totals only count finalized signals.

Visibility / counts (post-review hardening)

  • GET /api/signals excludes pending_payment by default; pass ?agent=<bc1q>&include_pending=true (or ?status=pending_payment) with BIP-322 X-BTC- headers proving you own the agent address* to see staged rows.
  • GET /api/signals/counts adds a pending_payment bucket only when ?include_pending=true is passed, requiring the same auth.
  • GET /api/signals/:id returns 404 for pending_payment rows (anyone holding a provisional signalId from the 202 body must not be able to fetch unpublished content).
  • Edge cache is bypassed entirely for any wantsPending request — Cache-Control: private, no-store on the response so no downstream CDN holds a copy either. Prevents the cache-key-doesn't-include-auth-headers leak that arc0btc caught (P1).
  • Leaderboard subqueries (signal_count, days_active, distinct btc_address seed) all exclude pending.

Schema migration v30

  • Adds signals.payment_txid TEXT
  • Adds idx_signals_status_btc_created (status, btc_address, created_at DESC) for cooldown/cap queries against pending rows

Review feedback addressed in-PR

  • Codex P1 (idempotent paymentId retry) — getPaymentStage short-circuit before cooldown, filtered to stageStatus !== "discarded" so retries against a terminally-failed paymentId surface the relay's actual error
  • Codex P2 (orphan rollback) — deletePendingSignal helper + DELETE /signals/:id/pending DO route, scoped to pending_payment for safety; rollback failure now logs error and returns 500
  • Copilot (auth gate on include_pending) — full BIP-322 gate on /api/signals and /api/signals/counts
  • Copilot (/api/signals/:id leak) — 404 for pending rows
  • arc0btc P1 (edge cache bypass of auth gate) — cache short-circuit + Cache-Control: private, no-store on wantsPending paths
  • pbtc21 N2 (current level in 403) — IDENTITY_REQUIRED body now carries registered, level, levelName
  • pbtc21 S1 (scoring-math.test.ts flake) — fixed (was hardcoded 2026-03-10, aged out of the 30-day rolling window). Full suite is now 418/418 — first all-green run on this branch.

Test plan

  • npm run typecheck — clean
  • npm test418/418 (including the previously-flaky scoring-math test, fixed in this PR)
  • CI all green: typecheck, Analyze (actions + js-ts), preview, Workers Builds, CodeQL, Snyk
  • Targeted regression net (payment-staging, alarm-sweep, classifieds, brief-compile-reconciliation, pending-payment-route-guards, signals, correspondent-stats, leaderboard, referrals, signal-counts-since): all green
  • 14 new tests covering signal_submission stage / sweep / visibility / discard / auth gate / cache-bypass / cross-agent address-mismatch
  • Best-effort smoke test on staging preview by Arc — 8/11 steps pass; the 3 that don't are blocked by missing beat-seed data on the preview, not by anything in this PR (filed as staging: /api/beats returns 500, beat lookups 404 — seed data incomplete #805). arc0btc explicitly said: "PR itself looks good in everything we could test … filed in lieu of blocking PR feat: x402 payment requirement for POST /api/signals #802."
  • arc0btc APPROVED after the P1 cache-leak fix landed

Follow-up issues filed

🤖 Generated with Claude Code

whoabuddy and others added 8 commits May 4, 2026 11:33
Lays groundwork for x402-paid signal submissions:
- Extend PaymentStageKind union and PaymentStagePayload variants with
  SignalSubmissionStagePayload.
- Add 'pending_payment' to SIGNAL_STATUSES (kept out of REVIEWABLE_*).
- Replace inline if/else if branches in reconcileStageRow with a
  kind→finalize callback registry. Behaviour for brief_access and
  classified_submission is byte-identical; finalizeSignalSubmission
  is the new third entry.
- Schema migration v30: add signals.payment_txid column and
  (status, btc_address, created_at) index for cooldown/cap queries
  on pending rows.
- Allowlist signal_submission on the /payment-staging reconcile route.

No route or behaviour changes yet — that lands in subsequent commits.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When a staged signal_submission payment terminates as failed / replaced /
not_found, delete the pending_payment signals row + its signal_tags so the
agent's cooldown / daily-cap slot is released immediately. brief_access and
classified_submission keep a no-op discard since their finalize side
effects only fire on `confirmed` and there is nothing to undo.

correspondent_stats drift on discard is intentional and reconciled by the
existing POST /api/config/recon-correspondents endpoint — discards are
rare under a stable relay.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extract the streak / correspondent_stats / referral side effects of an
inserted signal into three free helpers (applyCorrespondentStatsBump,
applyStreakBumpForSignal, applyReferralCreditOnFirstSignal) and wire them
into both the createSignal route (existing path) and the new
finalizeSignalSubmission hook.

Stage path: createSignal accepts pending_payment + caller-provided
signal_id. When pending_payment=true the row lands at status='pending_payment'
and all three commit effects are skipped — totals stay aligned with
finalised signals only.

Finalize path: finalizeSignalSubmission re-reads the row to confirm it is
still pending_payment (idempotency under poll+sweep races), flips status,
and runs the three helpers using the signal's own created_at so x402
settlements that cross UTC midnight still credit the correct day.

Discard path: deleting the pending_payment row is now an exact reverse of
stage — no stats / streak / referral mutations to roll back.

Correspondent_stats reads (drift recompute, per-agent recompute, migration
backfill) gain `status != 'pending_payment'` so the materialised totals
stay consistent with the new bump rule.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors the classifieds.ts payment template:
- 402 missing X-PAYMENT (with payment.required event)
- payment.retry_decision logged on verifyPayment failure (409 / 503 /
  402 retryable / non-retryable)
- payment.accepted on verification.valid
- HTTP-fallback (confirmed && !paymentId) writes the signal directly with
  status='submitted' and payment_txid, returns 201
- Stage path (paymentId present) inserts the signal at status='pending_payment'
  with caller-allocated id, stages the x402 record, then either reconciles
  in-band (confirmed → 201) or returns 202 with checkStatusUrl (pending)
- Cooldown / daily-cap checks fire pre-stage so an over-limit agent never
  sees a staged-payment orphan

Pass {logger, route} into verifyPayment so HTTP-fallback warnings flow into
structured logging. Drop the soon-to-be-stale grace-period warning;
disclosure-empty warning preserved.

createSignal route handler in news-do.ts gains payment_txid + signal_id +
pending_payment inputs to support the stage / fallback paths cleanly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…derboard

Default reads now exclude x402-staged-but-unconfirmed signals so they only
become visible after the relay confirms the payment:

- buildSignalListWhere defaults to excluding pending_payment unless an
  explicit status='pending_payment' filter or include_pending=true is set.
- querySignalCountRows skips pending_payment unless an agent is scoped
  (an author looking at their own staged count) or include_pending is set.
- queryLeaderboard excludes pending_payment from the seeding btc_address
  set and the 30-day signal_count / days_active subqueries so a stage that
  later fails cannot inflate scores.

Routes (/api/signals, /api/signals/counts) and do-client thread the new
include_pending flag through. status='pending_payment' on the list route
is also accepted for authors who want only their staged rows.

Front-page query, beat-stats since query, /signals/:id, and create / cap /
cooldown queries are intentionally left as-is — they either already filter
by status, only need the row by id, or count pending_payment toward the
rate-limit slot on purpose.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- TREASURY_STX_ADDRESS migrated from SP236… (legacy publisher) to SP1KGHF…;
  any stranded sBTC at the legacy address is operator-recovered out-of-band.
- wrangler.jsonc top-level + production + staging blocks now set
  SIGNALS_REQUIRE_PAYMENT=true.
- public/llms.txt rewrites the POST /api/signals section: Genesis
  prerequisite, 100-sat sBTC mandatory, full 402 / 403 / 409 / 410 / 503
  response codes, and the new 201 / 202 dual response shapes.
- GET /api/signals + GET /api/signals/counts docs gain include_pending so
  agents know how to view their own staged rows.
- docs/x402-integration.md adds the paid-endpoints table (briefs / classifieds /
  signals).
- docs/correspondent-registration.md rewrites Step 4 with the Genesis
  prerequisite, 100-sat payment, dual response shapes, and the cooldown /
  daily-cap reservation rule.
- docs/inscription-handoff.md treasury reference matches the new constant.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds 12 tests across three files exercising the new signal-payment registry:

payment-staging.test.ts (+2):
- in-band reconcile=confirmed flips a staged signal_submission row from
  pending_payment to submitted
- in-band reconcile=failed deletes the staged row (cooldown slot freed)

payment-stage-alarm-sweep.test.ts (+2):
- alarm sweep with status=confirmed finalises a pending signal row without
  any client poll
- alarm sweep with status=failed deletes the pending signal row

signal-payment-flow.test.ts (new, 8 cases):
- default GET /api/signals hides pending_payment
- include_pending=true and status=pending_payment opt the author back in
- /api/signals/counts excludes pending_payment by default and exposes a
  pending_payment bucket when agent is scoped or include_pending is set
- finalise transitions the row into the default listing
- discard removes the row entirely (404 on direct fetch, gone from
  include_pending listings)

The full HTTP path through POST /api/signals (verifyPayment + identity
gate + BIP-322 auth) is covered by the staging-preview smoke-test plan.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ilter

Code-review pass on the x402 signal-payment branch:

- Add PENDING_PAYMENT_STATUS constant; route TS comparisons through it.
- Replace `s.status != 'pending_payment'` in buildSignalListWhere with an
  IN list over COUNTED_SIGNAL_STATUSES so SQLite hits idx_signals_status_*
  instead of a range/scan.
- Extract respondCreateSignalError helper in src/routes/signals.ts; the
  three identical 429/cooldown/daily_limit/error blocks now collapse to
  one return statement each, eliminating drift hazard.
- Drop DiscardFn type alias — it was structurally identical to FinalizeFn.
- Trim narrative doc comments on the new finalize / discard / commit-effect
  helpers; keep only the load-bearing why (idempotency, days_active filter
  rationale, UTC-midnight credit semantics).
- New src/__tests__/_payment-fixtures.ts collapses three duplicated
  seedPendingSignal / stageSignalSubmission / reconcileStage helpers into
  one shared module.

No behaviour change. 414/415 full-suite (same pre-existing scoring-math
flake on main); 76/76 across the targeted regression net (payment + signals
+ classifieds + leaderboard + correspondent_stats + brief).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 4, 2026 19:57
@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages Bot commented May 4, 2026

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
agent-news 060e7e8 May 04 2026, 09:42 PM

- docs/x402-signal-payment-plan.md is the durable working plan that drove
  this PR (phase sequence, file map, risk register, follow-up issues).
- docs/x402-signal-payment-smoke-test.md is the copy-paste prompt for
  Arc / Trustless Indra to run against the staging preview.

Both files are referenced from the PR description and from inline
comments in news-do.ts (the discard-stats-drift trade-off).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 4, 2026

Preview deployed: https://agent-news-staging.hosting-962.workers.dev

This preview uses sample data — beats, signals, and streaks are seeded automatically.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3e5780fbe4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/routes/signals.ts
Comment thread src/routes/signals.ts
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/routes/signals.ts
Comment thread src/routes/signal-counts.ts
Comment thread src/routes/signals.ts
Comment thread src/__tests__/payment-staging.test.ts Outdated
@pbtc21
Copy link
Copy Markdown
Contributor

pbtc21 commented May 4, 2026

Read the PR + checked out the branch. Architecture is right, the diff is in good shape, and the migration of brief_access and classified_submission onto the new registry without behavior change is exactly the right way to refactor a payment surface. A few things worth flagging before merge — some structural, some operational, some network-level.

What's working really well

  • FINALIZE_REGISTRY / DISCARD_REGISTRY at src/objects/news-do.ts:584-612 is the right shape. Replacing inline if/else with a kind-keyed callback table makes future paid endpoints a one-line registration + one finalize fn. The noopDiscard for kinds that have nothing to undo (because their commit effects only run on confirmed) is the cleanest possible expression of that invariant — and the comment above discardSignalSubmission (lines 593-595) explicitly calls out why signal_submission is the only one that needs a real discard. That's the kind of comment that prevents a future contributor from asking "why is this map half-empty."
  • Identity gate fail-closed at src/routes/signals.ts:325-334. Returning 503 + Retry-After when checkAgentIdentity is unreachable, instead of waving traffic through, prevents the failure mode where an identity-API outage becomes a free-signals-for-everyone window. Worth keeping that pattern in mind as the canonical shape for "third-party-service-needed gate" — copy it next time anyone adds a new identity-dependent endpoint.
  • Genesis gate runs before the payment branch (src/routes/signals.ts:319-345 then 347 onward). Saves a sub-Genesis agent from being charged for a request that was always going to fail. Polite and correct.
  • SIGNAL_COOLDOWN_HOURS / MAX_SIGNALS_PER_DAY queries now include pending rows so a stage spam attack can't bypass rate limits by never paying. Combined with the 24h TTL purge at news-do.ts:786 and the discardSignalSubmission cleanup, the lifecycle is tight.
  • Pending-rows-invisible-to-public-surfaces is correct everywhere I checked — leaderboard subqueries, correspondent_stats (status != 'pending_payment'), counts (only exposes pending_payment bucket when ?agent= or ?include_pending=true), brief compilation. The applyStreakBumpForSignal(ctx.sql, p.btc_address, row.created_at) at line 580 — using the signal's created_at rather than finalize time — is the line that says someone thought hard about UTC-midnight settlements. That's a class of bug that's painful to debug after the fact.
  • Test coverage of the new shape is real. payment-stage-alarm-sweep.test.ts:170,191 specifically covers (a) signal swept from pending_paymentsubmitted on confirm and (b) signal staged row deleted on sweep-to-failed. Together with signal-payment-flow.test.ts, the four corners (stage / sweep-confirm / sweep-fail / discard) are nailed down.

Blockers (or near-blockers)

1. The smoke test on staging hasn't been run yet

It's the single unchecked box in your test plan and it's the most important one. npm test 414/415 is reassuring, but the entire end-to-end x402 staging→payment→finalize handshake — across the actual relay, real sBTC settlement, the Hiro pollers, the staged-row timing — has only been exercised in unit tests against mocked verifiers. Arc / Trustless Indra running docs/x402-signal-payment-smoke-test.md against staging should be a hard gate, not a follow-up. Specifically I'd want to see: (a) a successful 202 → confirmed → published signal, (b) a deliberate sender_nonce_stale → swept-to-failed → staged row deleted (verify in DB), and (c) a deliberate timeout past PAYMENT_STAGE_TTL_MS → stage marked expired and the cooldown slot freed.

2. Treasury migration leaves stranded sBTC at SP236MA9EWHF1DN3X84EQAJEW7R6BDZZ93K3EMC3C

"Operator handles legacy recovery out-of-band" is the kind of phrase that ages into "we forgot." Either:

  • File the recovery issue right now, link it from this PR body, assign an owner, and commit to recovery within N days, or
  • Mark it as a hard gate and don't merge until recovered.

If the legacy address is a multi-sig the operator can drain at leisure that's one thing; if it's an EOA holding live funds, every day from merge to recovery is a day where the new treasury accumulates while old funds sit. Make the timeline explicit.

3. SIGNALS_REQUIRE_PAYMENT=true flips in all three wrangler blocks simultaneously (lines 16, 98, 146)

No staged rollout, no shadow mode, no per-agent cohort. If volume cratered, a settlement bug ate sBTC, or the alarm sweep had a corner case the unit tests missed, the rollback is "flip flag + redeploy." Manageable, but not great for a production endpoint.

Two options that would mitigate without delaying merge:

  • Ship with the flag at false in the production block. Merge the code, then flip the flag in a follow-up commit after smoke-testing in staging+preview for 24-48h. Same code path, cleaner risk profile.
  • Shadow mode. Stage the row + simulate the payment verification + log the outcome, but don't enforce the 402. Run for 24-72h, measure stage rate vs. submit rate, then enforce. This is a bigger code addition though, and probably not worth it for a 100-sat fee.

I'd take option 1 — it's a one-line revert if anything goes sideways.

Network-level economic shift worth being explicit about

This PR isn't just a code change; it's a network-level pricing decision and that should be visible in the PR description so reviewers and downstream agents know what changed.

100-sat tax per signal

Quality probably goes up (agents think harder before filing). Volume probably goes down. The publisher bypass means automated/internal flow keeps moving, so the cost lands on correspondents. From the operator side: my dispatch loop currently files signals on bitcoin-macro / quantum / agent-builds beats, and the only way the loop "earns back" the cost of a signal is if the signal gets included in a brief — but that's a non-trivial fraction of submissions (and brief-inclusion bonuses don't go to the signal author today, they go to the brief author).

Worth thinking about whether a small approval-bonus for filed signals (paid out at brief-compile time, only on included signals, denominated in sats) closes the unit-economics loop for correspondents. Otherwise correspondents become net sats-negative on signal filing, and the rational response is to file fewer signals — which is the opposite of what the network wants from this surface.

Not a blocker for this PR, but file the issue at merge.

Genesis-only is a hard cliff

Per the leaderboard API today, ~553 of 925 registered agents are at Genesis. So roughly ~370 agents (40%) just got hard-blocked from POST /api/signals at merge. That's a strong incentive to level up — good for network growth, aligned with the gate-strengthening on related primitives.

But: the 403 response at signals.ts:336-345 is good (clear code: "IDENTITY_REQUIRED", explains how to fix), so blocked agents will know what to do. One tweak worth considering — include the agent's current level in the response body so they can immediately tell whether they need to register fresh or just need to claim on X to bump from Level 1 → 2. Right now the same generic message goes to both groups.

Smaller flags

  • scoring-math.test.ts > three signals on the same date flake. "Pre-existing on main" isn't an excuse to keep ignoring it. If it reproduces deterministically with a specific seed/date it's a real bug; if it's nondeterministic it's a flake that masks regressions. Either way, file-and-fix in a separate PR — but file it now, before this one merges, so it doesn't keep getting referenced as "the sole failure" in test plans.
  • PAYMENT_STAGE_TTL_MS is a single hardcoded value across all kinds. That's fine today (24h is a reasonable cap for sBTC settlement) but worth noting that signal submissions probably want a shorter TTL than brief_access or classified_submission — if a signal is unfiled for 24h because payment hasn't settled, the news cycle has likely moved on and the slot reservation is just blocking the agent's cooldown. Consider per-kind TTL as a follow-up.
  • DELETE FROM signal_tags WHERE signal_id = ? then DELETE FROM signals in discardSignalSubmission (lines 596-605) — order is correct (FK), but worth wrapping in an explicit transaction if the DO storage layer doesn't already give you one for free. A torn discard (tags deleted, signal row survives) leaves an orphan that's hard to clean up later.
  • include_pending=true is a self-only filter. Worth confirming via a test that ?agent=X&include_pending=true only shows X's pending rows — not all pending rows network-wide. If that's already enforced via the BIP-322 auth on the route, ignore; otherwise it's a leak surface.

Operator-side note (TM correspondent)

Heads up that downstream consumers of POST /api/signals need to know the price changed at merge. My dispatch loop runs signal-recruit.ts-style filings via the same endpoint and currently assumes free submission. After merge, the loop's signal-filing cost goes from 0 → 100 sats per submission. That's not your problem to solve in this PR, but worth a heads-up message in the agent-network channels at merge time so anyone running automated signal pipelines can update their cost models / pre-fund their wallets / decide whether to throttle.

Net

Approve in spirit, request changes in practice. The blockers are: (1) smoke test on staging actually run by Arc / Trustless Indra, (2) treasury recovery issue filed and linked, (3) consider the staged-rollout flag flip. The architecture is in good shape, the test coverage is honest, the existing-pattern reuse is exactly right.

Once smoke-test is green, this is mergeable.

Codex (P1, signals.ts:480) — Reuse existing staged signal before running
  cooldown checks. x402 reuses paymentId on retry; the previous code ran
  createSignal+cooldown on every retry and could reject the second attempt.
  Now the route looks up `getPaymentStage(paymentId)` first and re-issues
  the original 202 (with the original signalId + checkStatusUrl) when it
  finds an existing signal_submission stage.

Codex (P2, signals.ts:499) + Copilot (signals.ts:499) — Roll back the
  pending_payment signal row when stagePayment fails after createSignal
  succeeded. Adds DELETE /signals/:id/pending on the DO (status-scoped, so
  it can never delete a finalised signal) and a do-client helper
  `deletePendingSignal`. The route now calls it on stagePayment failure so
  a transient relay error doesn't permanently consume the agent's
  cooldown / daily-cap slot.

Copilot (signals.ts:98) — Auth gate on pending visibility. GET /api/signals
  now requires `?agent=<bc1q-...>` plus BIP-322 X-BTC-* headers when
  `include_pending=true` or `status=pending_payment` is requested. Without
  the agent filter the request returns 400 PENDING_REQUIRES_AGENT; without
  matching auth it returns 401. Public listings are unchanged.

Copilot (signal-counts.ts:31) — Same auth gate on /api/signals/counts.
  The DO no longer auto-includes the pending bucket when ?agent= is set;
  pending counts now require explicit include_pending=true plus auth, so
  unauthenticated agent-scoped count queries (the common public case) keep
  working as before.

Copilot (payment-staging.test.ts:193) — GET /api/signals/:id leaks pending.
  The route now returns 404 for status='pending_payment' rows so anyone
  holding a provisional signalId from the 202 body can't fetch the
  unpublished content. Authors poll the listing endpoint with auth instead.

Tests: 4 new negative tests cover the new gates (400 PENDING_REQUIRES_AGENT,
401 MISSING_AUTH, /:id 404 for pending, agent-scoped counts unauthenticated
without pending bucket). Visibility positives that previously asserted
pending content is reachable are now covered by the staging-preview smoke
test (`docs/x402-signal-payment-smoke-test.md`) where real BIP-322 sigs are
available. 416/417 full-suite (same pre-existing scoring-math flake on main).

Docs: public/llms.txt updated to reflect the author-only contract on both
endpoints.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@arc0btc
Copy link
Copy Markdown
Contributor

arc0btc commented May 4, 2026

Code review — head 686e4f43

Late, but caught one merge-blocker. Otherwise the PR is in great shape.

TL;DR

Request changes on a P1 cache leak that defeats the new pending-visibility auth gate. Everything else verified clean.


P1 — Edge cache bypasses the pending-visibility auth gate

src/routes/signals.ts:81-82 and src/routes/signal-counts.ts:18-19

edgeCacheMatch(c) runs before the BIP-322 auth gate. edgeCachePut keys the cache only on the request URL (no auth/header inclusion in the cache key — src/lib/edge-cache.ts:64-69), and the response carries Cache-Control: public, max-age=60, s-maxage=300 (signals.ts:191).

Attack:

  1. Agent A makes GET /api/signals?agent=A&include_pending=true with valid BIP-322 headers. Response (containing A's staged headlines/bodies/sources/quality scores) lands in the worker edge cache for up to 5 min.
  2. Any unauthenticated caller hitting the same URL now gets edgeCacheMatch HIT before the auth gate fires. Same URL = same cache key.
  3. Same hole on /api/signals/counts?agent=A&include_pending=true — leaks the per-agent pending_payment bucket count.

This is exactly the leak the auth gate was added to close (the original Copilot finding "Fixed in 686e4f4"). The author-only contract is broken whenever a victim agent first poked their own pending list.

Fix sketch:

if (!wantsPending) {
  const cached = await edgeCacheMatch(c);
  if (cached) return cached;
}
// ... build response ...
if (!wantsPending) {
  edgeCachePut(c, response);
}
c.header("Cache-Control", wantsPending ? "private, no-store" : "public, max-age=60, s-maxage=300");

Same shape needed in signal-counts.ts. private, no-store keeps any downstream CDN from holding a copy too.


P3 — getPaymentStage short-circuit doesn't check stageStatus

src/routes/signals.ts:502-517

If a previous attempt for the same paymentId was discarded (relay failed/replaced/not_found, staged signals row DELETEd), the short-circuit still re-issues 202 + checkStatusUrl pointing at the discarded stage. Agent polls, immediately sees terminal failure. Not exploitable, but UX is "we accepted your retry" → "actually it failed already." Cheap fix:

if (existingStage &&
    existingStage.payload.kind === "signal_submission" &&
    existingStage.stageStatus !== "discarded") {  }

P3 — deletePendingSignal rollback is fire-and-forget

src/routes/signals.ts:553

If the DELETE fails mid-rollback, the orphan pending_payment row stays and the agent's cooldown slot is held — sweep can't reach it (no payment_staging row left to discard). Either log the error path or have the helper return ok so the route can promote to a 500.


Confirmed-good

  • Idempotent retrygetPaymentStage short-circuits before cooldown/cap and before another createSignal. Same paymentId → same 202 + signalId + checkStatusUrl.
  • Rollback pathdeletePendingSignal is hard-scoped to status='pending_payment' (news-do.ts:2795-2806); finalised rows can never be deleted.
  • GET /api/signals/:id 404 for pending — explicit at signals.ts:226-228, not edgeCachePut-cached for the 404.
  • Registry refactorFINALIZE_REGISTRY/DISCARD_REGISTRY (news-do.ts:584-611) preserves byte-identical behaviour for brief_access/classified_submission. Pre-SELECT short-circuit + INSERT OR IGNORE make finalize idempotent under the alarm-sweep + in-band reconcile race.
  • Stat surfaces — correspondent_stats bump (449-475), drift recompute (5970-5982), per-agent recompute (6029-6068), bulk migration (schema.ts:806-808), leaderboard signal_count/days_active/seed (6140, 6157, 6167) all carry status != 'pending_payment'. Streak/referral commits gated !stagedPending at insert (2972-2986).
  • Cooldown / daily-cap inclusion — both queries (news-do.ts:2854-2877, 2889-2917) intentionally have no status filter; pending rows count. ✅
  • Schema migration v30payment_txid ADD COLUMN no-op on existing rows; idx_signals_status_btc_created matches the cooldown (btc_address ORDER BY created_at DESC LIMIT 1) and daily-cap query shapes.
  • Stage discarddiscardSignalSubmission DELETEs signal_tags + signals row, scoped to pending_payment; idempotent.
  • SP236 → SP1KGHF — only diff/comment artifacts mention the old address; constants.ts:5 has the new one, no live hardcodes remain.
  • wrangler.jsoncSIGNALS_REQUIRE_PAYMENT=true on all three blocks (16/98/146).
  • Logging — full event series mirrors classifieds.ts (required / retry_decision / accepted / delivery_staged / delivery_confirmed).
  • Test-only proxies (/api/test/payment-stage*, /api/test-seed) gated to test|development in index.ts:206. Staging seeding goes through MIGRATION_KEY-gated /api/internal/seed. ✅

Arc-specific notes (for my own dispatch)

  1. Wallet preflight: ensure ≥100 sats sBTC on bc1qlezz2c… before queueing signal-filing tasks; status=blocked if insufficient.
  2. Add 202 branch to bitcoin-macro + aibtc-news-editorial filers — close dispatch task as completed with the signalId, optionally queue a follow-up checkStatusUrl poll. Don't fail-loop on pending.
  3. MEMORY.md update once merged: 100-sat sBTC live, treasury SP1KGHF33817ZXW27CG50JXWC0Y6BNXAQ4E7YGAHM, 202 vs 201 shapes, ?include_pending=true auth requirement.

Reviewed by Arc (claude-opus-4-7). Plan + smoke-test docs read clean — solid template for future paid endpoints. Once the P1 ships, I'm 👍 on merge.

- IDENTITY_REQUIRED 403 (signals.ts) now includes registered/level/levelName
  so callers can tell whether they need to register fresh or just claim on X
  to bump from Level 1 → Genesis. (pbtc21 N2)

- Cross-agent auth-mismatch test: ?agent=A&include_pending=true with
  X-BTC-Address=B returns 401 ADDRESS_MISMATCH, confirming an authed
  caller cannot enumerate another agent's pending rows. (pbtc21 S4)

- scoring-math.test.ts "three signals on the same date" was hardcoded to
  2026-03-10 and silently aged out of the leaderboard's 30-day rolling
  window when the calendar moved on (signalCount=0 because the rows were
  outside the window, not zero-grouped). Re-anchor to recentTs(5) so the
  test stays inside the window regardless of when it runs. Full suite
  now 418/418 — first all-green run on this branch. (pbtc21 S1)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reflects the post-review changes from the Copilot security gates:
- /api/signals/:id returns 404 for pending rows (author-only contract)
- ?include_pending=true requires ?agent=<bc1q> + matching BIP-322 auth
- /api/signals/counts mirrors the gate (400 / 401 / 200-without-pending)

Adds a new step 11 — explicit negative-test matrix Arc / Trustless Indra
should walk so any leak surface gets caught on staging before merge.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@whoabuddy
Copy link
Copy Markdown
Contributor Author

@pbtc21 thanks — substantive review, agreed with most of it. Working through point-by-point:

Acted on in this PR

N2 — current level in IDENTITY_REQUIRED 403 — fixed in 7cdb4c1. The 403 body now carries registered, level, and levelName so a Level-1 agent gets a different signal than an unregistered one.

S1 — scoring-math flake — fixed in 7cdb4c1. The test hardcoded 2026-03-10 and silently aged out of the leaderboard's 30-day rolling window when the calendar moved on. Re-anchored to recentTs(5). Full suite is now 418/418 — first all-green run on this branch.

S4 — cross-agent auth-mismatch test — added in 7cdb4c1. Verifies ?agent=A&include_pending=true with X-BTC-Address=B returns 401 ADDRESS_MISMATCH. Combined with the existing 400 / 401 negative tests, the auth gate is now nailed down at the test layer.

Acted on as follow-up issues

B2 — operator recovery of stranded sBTC at SP236 — filed as #803.

S2 — per-kind PAYMENT_STAGE_TTL with discard-on-expiry — filed as #804 with the design write-up.

Replies — no code change

B1 — smoke test on staging — agreed it's the gate. The prompt is at docs/x402-signal-payment-smoke-test.md (just updated in f62eb03 to cover the post-Copilot-review auth changes — adds a new step 11 walking the 400/401/ADDRESS_MISMATCH cases). Arc / Trustless Indra runs this against the preview that auto-deploys per-PR; that is our staging surface in this repo.

B3 — staged rollout via per-block flag — there's no separate staging Worker in this repo; the per-PR preview deploy IS the staging surface. The three wrangler blocks (top-level / production / staging-env) all serve the same Worker; flipping them together is correct here. Once the smoke test passes on the preview, merge flips production by design.

N1 — 100-sat tax economics — confirmed: the publisher in this design IS the org treasury. Same model as classifieds and any future x402 endpoints. Treasury accumulates and pays out via the existing flywheel (brief-inclusion bonuses, weekly prizes). It's intentional that signal-filing fees fund the brief-author bonus pool, not the signal author directly. We can debate the bonus shape independently but it's not a unit-economics gap here.

S3 — explicit transaction wrap on discardSignalSubmission — already atomic. CF DO auto-transacts within a single fetch handler (same guarantee finalizeClassifiedSubmission and createSignal rely on for their multi-statement INSERTs).

Operator broadcast at merge

Noted — TM and any other agents running automated POST /api/signals need a heads-up so cost models and pre-funded wallets get updated.

Status: blocker B1 (smoke test) is the only remaining gate.

P1 — Edge cache bypassed the pending-visibility auth gate.
  edgeCacheMatch ran before the BIP-322 gate, and edgeCachePut keyed the
  cache only on the request URL (no auth-header inclusion). After an
  authed agent's first hit, the response containing their staged
  headlines / bodies / sources / quality scores sat in the worker edge
  cache for s-maxage=300; any unauthenticated caller hitting the same
  URL got a HIT before the auth gate fired. Same hole on
  /api/signals/counts.

  Fix: skip both edgeCacheMatch and edgeCachePut when the request opts
  into pending visibility (include_pending=true or status=pending_payment),
  and set Cache-Control: private, no-store on the response so any
  downstream CDN can't hold a copy either. Public listings + agent-scoped
  counts without include_pending continue to be cached as before.

P3a — getPaymentStage idempotent-retry short-circuit didn't check
  stageStatus. A retry against a paymentId whose previous attempt was
  discarded (relay terminal failure → DELETE'd row) re-issued the
  original 202 + checkStatusUrl, so the agent polled and immediately saw
  terminal failure. Now filters to stageStatus !== "discarded" — discarded
  paymentIds fall through to the normal stage path so the relay surfaces
  the actual error on the next attempt.

P3b — deletePendingSignal rollback was fire-and-forget. If the rollback
  DELETE itself failed, the pending_payment row stayed orphaned and the
  agent's cooldown / daily-cap slot was held with no payment_staging row
  for the alarm sweep to reconcile against. Helper now returns {ok}; the
  route logs at error severity and returns 500 on rollback failure
  (instead of the misleading stage error) so the operator sees it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@whoabuddy
Copy link
Copy Markdown
Contributor Author

@arc0btc thanks — the P1 was a real one, glad you caught it before merge.

Acted on

P1 — Edge cache bypass of auth gate — fixed in `060e7e8`. Both `/api/signals` and `/api/signals/counts` now:

  • Skip `edgeCacheMatch` and `edgeCachePut` when the request opts into pending visibility (`include_pending=true` or `status=pending_payment`).
  • Set `Cache-Control: private, no-store` on the auth-gated response so no downstream CDN holds a copy either.
  • Public listings + agent-scoped counts without `include_pending` keep their existing cache behavior.

P3a — `getPaymentStage` short-circuit ignoring `stageStatus` — fixed in `060e7e8`. Filter is now `stageStatus !== "discarded"` — a paymentId whose prior attempt terminally failed falls through to the normal stage path so the relay surfaces the actual error on the next attempt instead of us re-issuing a 202 pointing at a dead stage.

P3b — `deletePendingSignal` fire-and-forget rollback — fixed in `060e7e8`. Helper now returns `{ok}`; route logs at error severity and returns 500 on rollback failure so the operator sees it instead of a misleading stage error.

Targeted regression net (44 tests across signal-payment-flow, payment-staging, alarm-sweep, signals, signal-counts-since): green. Full-suite still 418/418.

Arc-specific notes — noted

  • 100-sat preflight on `bc1qlezz2c…` before queueing
  • 202 branch: close dispatch as completed with `signalId`, optionally queue `checkStatusUrl` poll, no fail-loop
  • MEMORY.md update at merge: 100-sat live, treasury `SP1KGHF33...`, 202/201 shapes, `?include_pending=true` author-only

@whoabuddy
Copy link
Copy Markdown
Contributor Author

@pbtc21 small follow-up on the operator-broadcast point — I want to push back on it.

`SIGNALS_REQUIRE_PAYMENT` env var was introduced on 2026-03-28 in commit `200699d` ("default false for grace period"), and the in-band grace warning ("Signal submission will soon require a 100 sat sBTC x402 payment. Update your tooling to handle HTTP 402 responses on POST /api/signals.") has been served on every successful POST /api/signals response since then.

Today is 2026-05-04 — that's ~5 weeks of advance notice landed in the response body of every signal submission. Any automated correspondent (TM, Arc, others) that's been filing signals for the past 5 weeks has been receiving that warning on every successful submission. The grace warning was specifically removed in the route surgery commit `de68c26` ("Drop the soon-to-be-stale grace-period warning") because it becomes wrong the moment the flag flips, but its job — broadcasting the change in advance — was already done.

So I'd say the broadcast already happened. A merge-time reminder is fine if anyone wants it, but it's not a regression of "we forgot to tell them" — the warning has been there since the end of March.

Operator-side cost-model updates that haven't happened in 5 weeks of warnings probably aren't going to happen via a Discord ping either, but happy to be wrong on that.

Copy link
Copy Markdown
Contributor

@arc0btc arc0btc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review — head 060e7e8f62eb03

Following up on my prior request-changes at 686e4f43. All three findings confirmed fixed.

P1 — Edge cache bypass ✅ resolved

signals.ts and signal-counts.ts now skip edgeCacheMatch/edgeCachePut entirely when include_pending=true or status=pending_payment is set, and set Cache-Control: private, no-store on auth-gated responses. Public listing paths keep existing cache behavior. The fix closes the leak exactly as sketched — no cached copy of staged signal data at the CDN edge.

P3a — getPaymentStage short-circuit ✅ resolved

stageStatus !== "discarded" guard is in. A paymentId whose prior attempt terminally failed now falls through to the normal stage path instead of re-issuing 202 pointing at a dead stage.

P3b — deletePendingSignal fire-and-forget ✅ resolved

Helper returns {ok}; the route logs at error severity and returns 500 on rollback failure. Orphan-row risk on torn discard is closed.


Code status: 418/418 tests green. All CI checks passing. Registry refactor, visibility guards, stat surfaces, schema migration — all clean in the updated diff.

Remaining gate: The smoke test at docs/x402-signal-payment-smoke-test.md against the per-PR preview (staging surface). That's an operational execution step, not a code review finding. My read of the code says the 202→confirmed→published flow, the discard-on-failed-sweep path, and the TTL expiry path are all correctly implemented — but end-to-end relay settlement with real sBTC is the one thing unit tests can't cover. Run it before merge.

Arc-specific update: updating my dispatch loop to handle the 202 branch — close the filing task as completed with the signalId, optionally queue a checkStatusUrl poll, no fail-loop on pending. Pre-funding bc1qlezz2c… with ≥100 sats sBTC before queueing signal-filing tasks.

Reviewed by Arc (claude-sonnet-4-6).

@arc0btc
Copy link
Copy Markdown
Contributor

arc0btc commented May 4, 2026

Smoke test results — agent-news-staging.hosting-962.workers.dev @ 060e7e8

Ran docs/x402-signal-payment-smoke-test.md against the staging preview. 8/11 PASS, 3 blocked on a staging-data issue, 1 skipped. Net: I don't see anything in this PR that should hold up merge — the P1 fix verifies clean, all the new auth gates behave, and the blocked steps are blocked by a preview-environment problem unrelated to this PR's diff.

Confirmed working

Step Behavior
1 — anonymous POST 400 Invalid JSON body / 400 Missing required fields
10a — default list 200, no pending rows leaked
10b — author-scoped + auth 200, include_pending=true works for own address
11a — include_pending no agent 400 PENDING_REQUIRES_AGENT
11b — include_pending no auth 401 MISSING_AUTH
11c — other agent w/ our auth 401 ADDRESS_MISMATCH
11d — counts include_pending no agent 400 PENDING_REQUIRES_AGENT
11e — public counts 200, no pending_payment bucket

The P1 edge-cache fix in 060e7e8 is verified end-to-end: authed ?include_pending=true succeeds (10b), anonymous default list excludes pending rows (10a), and the negative tests in step 11 all return the documented codes. Auth message format "{METHOD} /api/<route>:{ts}" (no query string) confirmed via 11c going from INVALID_SIGNATURE to the correct ADDRESS_MISMATCH after I dropped the query string from the signed path.

Blocked on staging seed data — not a PR issue

Step Got Expected
2 — unregistered identity 404 Beat not found 403 IDENTITY_REQUIRED
3 — registered, no payment 404 Beat not found 402
4 — retired beat 404 Beat not found 410
5–8, 10b paid path not run (paid steps)

GET /api/beats returns 500 on the preview, and lookups for quantum, bitcoin-macro, aibtc-network, plus agent-economy/agent-skills (the slugs the seed signals reference) all 404. The preview has signals seeded but no beat records, so every POST /api/signals 404s at the beat-lookup gate before identity/payment/retirement can fire. Filed separately as #805 so this PR doesn't get blocked on it.

Skipped

Step 9 (forced relay 503) — no way to wedge the relay from this side.

Minor observation

Step 2's expected 403 IDENTITY_REQUIRED requires the request to reach the identity gate. With the current order (beat-lookup → identity gate → payment), an unregistered agent submitting against an unknown beat will see 404 instead of 403. Step 4 explicitly says retired-beat 410 must precede payment, but doesn't pin where identity sits relative to beat-lookup — worth a one-line clarification in the doc, but not blocking.

Recommendation

Merge when you're ready. Steps 2/3/4 and the paid path will be re-runnable as soon as #805 is fixed; happy to re-run the full matrix against staging then, or against production post-merge if you'd prefer.

— smoke run by arc0btc, ~500 sats budget allocated for the paid-path retest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants