fix(lurker): PSS bin selection — base is min(PO(peer, target), L), not PO(peer, target)

[flotob]

Credit where due

This correction comes from Viktor Trón, the inventor of the Swarm protocol, who reviewed the light-node receiver design and asked the one question that mattered: which PO bins is the receiver subscribing to? — and then pointed out that our answer was wrong in exactly the deep-peer regime.

The bug

For a trojan mined to an L-bit prefix of target t, on a covering peer p with b_p = PO(p, t), the trie geometry gives two regimes:

Regime	Where the trojan sits on p	Old selection (b_p..=b_p+3)
b_p < L	Exactly bin b_p (p diverges from t at b_p; the trojan still agrees there)	Base right by accident; the +3 window pulled bins that cannot contain it
b_p ≥ L	L + Geom(½) — independent of b_p, i.e. around bin L, shallower than b_p	Missed entirely once b_p > L+3

Why every mainnet run passed anyway: the demos mined 2-byte (16-bit) targets while covering storers sat at b_p ≈ 11–14 < L — the accidental-correct regime — and 5-peer covering redundancy masked individual deep-peer misses (the symmetric-rendezvous co-resident node at 18 shared bits was squarely in the broken regime; its neighbors covered for it). Working, but not for the stated reason.

The fix

PSS_MINED_PREFIX_BITS = 16 becomes a receiver-side convention (bee's de-facto 2-byte targets, what ant's own gateway demos send), and bin selection moves into a pure covering_bins(b_p, want_gsoc, want_pss):

• GSOC: exactly bin b_p — the watch target is the chunk address; never affected.
• PSS, b_p < L: exactly bin b_p (was: b_p..=b_p+3 — 3 dead bins dropped).
• PSS, b_p ≥ L: bins L..=L+3 (geometric tail: 15/16 of the mass per peer; the covering-set redundancy handles the rest).
• Mixed watch: deduplicated union (the desired-set/handover machinery already handles non-contiguous bin sets per peer — only the range plumbing changed).

The convention does double duty, per Viktor's own accounting: bins ≥ L carry only ~2^−(L−d) of a depth-d storer's ingest (candidate try-unwrap traffic stays small), and L must exceed the storage depth or no storer keeps the trojan at all — the "1–3 bytes, your choice" freedom in the API was a trap at both ends.

Follow-up worth its own discussion: at L = 24, the entire trojan-bearing backlog of a 4M-chunk reserve is ~1–2K chunks (~8 MB) — small enough to sweep completely on subscribe. That turns PSS reception from a live tail into an offline mailbox. Not in this PR; noted in the constant's docs.

Verification

• 5 new regression tests: both PSS regimes (including b_p > L+3, the total-miss case), GSOC exactness, mixed-watch union/dedup, empty watch.
• Gate: cargo clippy --workspace --all-targets -- -D warnings = 0, fmt clean, full workspace green (34 suites; 110 in ant-p2p).
• Mainnet e2e note: the lab batches from the PSS + GSOC messaging on a light node (send + receive, bee-interoperable) #50 sweep have expired; the standing demo suite (companion of PSS + GSOC messaging on a light node (send + receive, bee-interoperable) #50) covers the shallow regime, and a deep-regime e2e (receiver co-resident beyond the mined prefix, single covering peer) is the right test to add once a fresh batch is funded — happy to run it on request.

🤖 Generated with Claude Code

[flotob]

Deep-regime mainnet verification — the fix observed live

Ran the scenario this PR exists for, on mainnet at 010e903, with a freshly funded depth-20 batch:

Setup: rendezvous target = a connected mainnet storer's own overlay (c7aa1937…), so that storer covers the subscription at b_p = 31 — maximally deep in the geometric regime. Subscribe /pss/subscribe/{topic}?neighborhood=c7aa…, then 4 sends mined to the 16-bit prefix c7aa (4 independent mining draws, so the per-message 1/16 tail miss can't fake a failure).

Puller placement (the fix, visible): the deep peer got pullers at bins 16, 17, 18, 19 — the L..L+3 window:

lurker pulling neighborhood bin peer=QmXcxNMDH… bin=16 …
lurker pulling neighborhood bin peer=QmXcxNMDH… bin=17 …
lurker pulling neighborhood bin peer=QmXcxNMDH… bin=18 …
lurker pulling neighborhood bin peer=QmXcxNMDH… bin=19 …

The pre-fix selection (b_p..=b_p+3) would have placed a single bin-31 puller on this peer — bins that a 16-bit-mined trojan cannot reach.

Delivery attribution (the regime, observed): all 4 messages delivered, every one through the deep peer, at bins matching L + Geom(½):

pull round peer=QmXcxNMDH… bin=18 … delivered=1   (msg #1)
pull round peer=QmXcxNMDH… bin=16 … delivered=1   (msg #2)
pull round peer=QmXcxNMDH… bin=16 … delivered=1   (msg #3)
pull round peer=QmXcxNMDH… bin=16 … delivered=1   (msg #4)

Observed bins {16,16,16,18} against predicted mass (½ at 16, ¼ at 17, ⅛ at 18). Attribution is single-source: the other covering peers sit at b_p ≈ 5–6 and don't store this neighborhood at all, so the deep peer was the only possible path — precisely the path the old code couldn't use.

One additional live observation matching the geometry: an initial single-send run produced no delivery — consistent with the 1/16 tail (PO(c, storer) > L+3) the window deliberately accepts, and the reason the covering-set redundancy (and, in this maximally-adversarial single-source setup, multi-send) exists. Four draws: all four in-window.

No-regression: gsoc_e2e ✅ and pss_e2e ✅ (shallow regime, own-neighborhood) re-passed on the same build.

With the unit tests covering both regimes and the deep regime now demonstrated live against mainnet storers, this is proven end to end.

🤖 Generated with Claude Code

[flotob]

Viktor Trón ruled on the mining-prefix convention: L = 24 (3-byte targets). Pushed as 341e696 — PSS_MINED_PREFIX_BITS 16 → 24, tests made L-relative. (The stacked mailbox #55 was rebased on top and its docs updated to match.)

Re-verified end to end on mainnet at the L=24 build (release, 3-byte mined targets, directed PSS to the node's own overlay so residency is guaranteed):

• Live: 4/4 — send 4 messages mined to a 3-byte prefix, all received.
• Mailbox: 4/4 — send 4 with nobody subscribed, then ?history=true recovers all 4 (history pullers swept bins 9–14 — the covering peers' proximities — from cursor − 4096).

One real finding worth flagging (and now in the SWIP as an informative note): 24-bit mining is ~256× a 2-byte mine. On the debug lab build it exceeds the 60s /pss/send timeout outright; even on release the geometric variance of the search occasionally trips 60s, so the mailbox run needed swarm-chunk-retrieval-timeout: 180s. Server-side it's a few seconds; on a constrained mobile sender (Freedom) it's material — L=24 makes reception ~256× cheaper at the cost of ~256× more sender work. The right fix is to mine off the request's critical path (background task) rather than block the send; noted as follow-up, not blocking.

Net: the deeper convention is the right call — it's what makes the mailbox a complete backlog sweep rather than a recent window — and it's now proven on mainnet. Sender-side mining ergonomics are the one thing it costs, and that's a UX problem, not a protocol one.

🤖 Generated with Claude Code

[flotob]

Reverted to L = 16 (2-byte targets) — after more discussion, we're keeping the convention at 16, not 24. 341e696 (the L=24 bump) is dropped; the branch is back at the reviewed L=16 state plus a doc commit recording why 16 is the deliberate choice.

The deciding insight — and a correction to my earlier "L=24 is 256× cheaper reception" claim: that benefit only accrues to a receiver deeply resident in the target neighborhood (covering peers at b_p ≥ L). A light node's covering peers sit at b_p ≈ 9–14, so covering_bins pulls bin b_p regardless of L — I measured it identical at 16 and 24 on mainnet (same bins, same candidates). So for a light-node deployment, L=24 buys the receiver nothing and costs the sender 256× (~2^24 hashes — seconds on a phone, and it tripped the /pss/send timeout in the earlier verify). PSS already has an economic spam gate via the postage stamp, so the extra proof-of-work is largely redundant.

Net: 16 is the right default for a mobile light node — cheap to mine, identical receive. The mailbox (#55) is a recent-history window at any L (its docs are updated to say so honestly, rather than claiming complete-backlog which also needs deep residency).

Whether the network should mandate a deeper prefix as deliberate anti-spam PoW is a genuine protocol-incentive question the Swarm designer raised and is still weighing — parked as an open question in the SWIP, since its home is the core protocol, not our light-node default.

Both branches force-pushed at L=16; gate green.

🤖 Generated with Claude Code