AntStream: camera capture + live publisher loop (#67 stage 2, #65) - #76
AntStream: camera capture + live publisher loop (#67 stage 2, #65)#76meinharrd wants to merge 10 commits into
Conversation
Stage 1 built the publisher loop minus the camera and measured what it
sustains. Stage 2 replaces the synthetic generator with the real capture
pipeline and adds the two things a viewer needs — a rolling HLS playlist
and a feed that points at it.
Rust (crates/ant-ffi/src/publisher.rs, new)
* The live loop, grown out of bench.rs rather than beside it: Target,
the loopback HTTP client, publish_bzz and data_chunk_count moved
here as product code and bench.rs now imports them, so the bench
keeps measuring exactly the call a broadcast makes.
* Per segment: POST /bzz -> rebuild the media playlist over a sliding
window -> POST /bzz the playlist -> publish its reference as a
sequence-feed update with POST /soc (bee-js shape: id =
keccak256(topic || index_be8), payload timestamp_be8 || reference).
POST /feeds at start yields the channel manifest a viewer resolves.
* Live-edge discipline from the stage-1 findings: in-flight window 4
(8 collapsed the connection layer), drop-oldest backlog rather than
unbounded lag, and a gap marked #EXT-X-DISCONTINUITY instead of a
silent hole. Default rendition 360p/900 kbit/s/2 s segments.
* Segments commit in capture order even though uploads finish out of
order, so the playlist is never reordered; a media segment whose
initialization segment did not land is never listed.
* C ABI: ant_publisher_start/_push_segment/_progress/_stop, documented
in include/ant.h.
Swift (examples/ios-stream)
* CaptureEngine.swift (#65): AVCaptureSession -> H.264 ->
AVAssetWriter in .mpeg4AppleHLS emitting fMP4 segments, runtime
bitrate/segment duration, on-disk segment ring, and interruption
handling (call, backgrounding, camera flip, orientation, thermal
downshift) that ends the current segment cleanly and starts a new
one flagged discontinuous.
* LiveBroadcast.swift joins capture to the publisher and holds the
foreground keep-alive (idle timer, audio session).
* LiveView.swift is the going-live screen: preview, the publish-lag
indicator, and the dropped/failed counters.
* Camera + microphone usage descriptions; Go live now opens LiveView.
ant-side gaps found during integration
* A saturated postage batch was reported as 502 Bad Gateway on
POST /bzz and POST /soc, which reads as retryable — a long
broadcast would retry forever against a permanently full bucket.
One shared mapping now answers bee's 402 "batch is overissued"
everywhere, matching what /chunks/stream and /stewardship already
did.
* PushSoc had none of PushChunk's collision-bucket guard, so on an
immutable batch a feed update surfaced as a bare "stamp issue
failed: bucket full" and on a mutable one wrapped a bucket —
evicting somebody's chunk — with no log. It now takes the same path.
Part of #67 (stage 2), implements #65.
- ant_publisher_stop's grace was 75 s but stopping publishes two rounds of the 60 s per-segment deadline (the in-flight window, then the bounded backlog behind it). Raised to 130 s and corrected on every doc surface the call has: lib.rs and ant.h. - PublisherReport::kept_up is a conjunction whose first clause is 'at least one segment published', so a broadcast that published nothing reported false — and the summary card rendered that as 'fell behind', blaming the uplink for a broadcast that never started. The card is now three-state, the same treatment BenchReport.hasMeasurement gives the throughput verdict, with one predicate both the label and the tint read.
Found by a regression test for the kept_up predicate: when a *later* initialization segment failed to upload (a writer restart after an interruption), the media segments that followed were still listed under the previous writer's #EXT-X-MAP — an initialization segment that does not describe them, i.e. a playlist a player cannot start. They also counted as published, so kept_up could report a broadcast as keeping up while the segments after the failure were unplayable. Committed::Failed now carries the segment kind, a failed init clears the map in force (nothing is listed again until a fresh one lands), and kept_up counts segments_listed — segments that reached a published playlist — rather than segments uploaded. segments_listed is surfaced on the snapshot/report (shown as PLAYABLE on the live screen) so the Swift verdict reads the same first clause the Rust predicate does.
POST /feeds is a real upload and can sit on the 60 s publish deadline while the peer set warms up. Awaiting it before spawning the pump spent the first minute of a broadcast waiting for a reference nobody had been handed yet, while the capture backlog shed segments behind it. It now runs alongside the loop, with an in-flight guard so the start-time attempt and the committer's retry can't both be on the wire at once.
ByteCountFormatter's default nonnumeric formatting rendered 0 as 'Zero
KB' — visible in the broadcast checklist ('Zero KB free') and now next
to the live counters ('SENT Zero KB'), where every neighbouring cell is
a numeral. Both surfaces are in the simulator evidence for this PR.
Simulator evidence —
|
|
Re-captured after
Both workflows are green on the head commit: |
|
[alan-review R1] Blocking —
Failure scenario — and it's the one this PR itself identified as real (gap 2: SOC writes hitting a full collision bucket, now surfaced as a hard 402/ Empirically confirmed with a temporary Both asserts of honest behavior ( Suggested direction: compute the effective lag at read time as something like |
|
[alan-review R1] Minor findings (combined, non-blocking):
|
|
[alan-verify R1] R1-F1 ( Reproduced independently against the real publisher with a temporary
So both clauses hold as the reviewer described: Broader trigger, also reproduced: with all uploads failing after the first update ( UI impact is direct, not inferred: Visual note: the |
`keeping_up` and `kept_up` were computed from the latency of the last feed update that *landed*. That figure only moves on success, so once `POST /soc` starts failing it freezes at its last good value while the segments go on uploading and listing — `segments_listed == media_pushed` still holds, `feed_updates` is still > 0 — and the badge stays green with a frozen lag while no viewer sees anything past the last playlist that landed. Measure the live edge instead: keep the capture instant of the newest segment a landed feed update made playable, and report its *age*. That grows for exactly as long as nothing new reaches a viewer, so a stuck feed clears the three-segment budget and both the badge and the verdict turn. Bootstrapped with the first committed segment so a broadcast whose very first update never lands measures from the content nobody saw, and frozen at run finish so a report read later still describes the broadcast rather than how long the host waited to ask. Docs updated on every surface that describes the field: `ant.h`, the snapshot/report doc comments, and the mirrored Swift models.
91b8bbc to
1ddea92
Compare
|
[alan-fix R1] — commit R1-F1 —
|
|
[alan-review R2] Blocking — the capture → publisher hand-off is not order-preserving, but every playlist-correctness property this PR pins rests on push order.
On the Swift side nothing guarantees that order:
Failure scenario — every writer restart, a first-class feature of this PR (camera flip, orientation change, thermal downshift, interruption recovery): the old writer's final media segment and the new writer's initialization segment are delivered back-to-back ( Same root cause weakens Suggested direction: make the hand-off single-file. Verification note: no Swift toolchain on this box, so this is confirmed-from-source on both sides: the Rust side's order dependence is direct from |
|
[alan-review R2] Minor findings (combined, non-blocking):
|
|
[alan-verify R2] R2-F1 — CONFIRMED. The hand-off is genuinely unordered, and the consequence lands in the playlist. Why the hop is unordered. The capture side is fine — Why it matters. ( The existing Caveat, for fairness: the finding's second clause — that the same unordered hop makes No screenshot evidence: the defect is invisible in the app by construction — every on-screen counter and the |
The publisher assigns each segment its commit sequence by the order the push_segment calls arrive, and that sequence is both the playlist order and the #EXT-X-MAP a media segment is listed under. The capture hand-off spawned an unstructured Task per segment, each of which then awaited a detached FFI task, so arrival order was not the writer's order. At a writer restart (camera flip, orientation, thermal downshift, interruption recovery) the old writer's final media segment and the new writer's initialization segment are emitted milliseconds apart; an inversion lists the old segment under the new map -- undecodable after a dimension or bitrate change -- with the discontinuity tag on the wrong entry, while segments_listed and kept_up stay clean. onSegment now enqueues into an AsyncStream from the writer's delivery queue (synchronous, FIFO, still never blocking capture) and a single pump task awaits one push at a time. stop() closes the feed and waits for the pump to drain rather than polling segmentsPushed against capturedSegments, which could not converge on a failed push. Also state the ordering contract on every doc surface of the call (publisher.rs, lib.rs, ant.h) and pin it in the restart playlist test.
|
[alan-fix R2] — fixed in R2-F1 — capture→publisher hand-off is now order-preserving (blocking)Root cause. I reproduced the consequence against the real playlist assembler with a temporary
Fix. The hand-off is now a single ordered queue instead of a task per segment:
Contract, on every doc surface (per the repo lesson about shared FFI contracts): Verification
Not covered by evidence: the writer-restart inversion itself can't be shown end-to-end on the runner — the simulator has no camera to flip and the fake batch means no playlist is ever assembled to inspect. The failure mode is proven at the playlist layer (above) and the ordering guarantee is now structural in the hand-off; a restart on hardware stays a manual check. |
|
[alan-review R3] Minor findings (combined, non-blocking). No blocking findings this round: I re-drove the R1/R2 fix areas (live-edge age semantics, the ordered Swift hand-off, the pump/committer drain protocol, the shared
|
meinharrd
added
alan:clean
- EXT-X-TARGETDURATION is a run-max that only ratchets up: RFC 8216 $6.2.1 forbids it changing across reloads, and keyframe-aligned cuts routinely overrun the configured target (R3 #1, with a test pinning the slide-out case). - A push racing cancel() can no longer orphan a segment: push refuses under the state lock once pump_finished is set, and the pump's final sweep (same critical section) drops any segment that slipped in before it — every push now ends as published, failed or dropped (R3 #2). - duration_s / sustained_mbit_s / sustained_chunks_s freeze with the run via ended_at, the same principle as lag_ms_final; the happy-path test now pins all four (R3 #3). - The channel-manifest retry is spawned instead of awaited on the committer, so a hanging POST /feeds cannot stall playlist and feed updates for landed segments (R1 #4). - Derived topics seed from milliseconds plus a process-wide nonce, so stop + restart of a channel inside one clock tick cannot overwrite the old feed's head at index 0 (R1 #5, with a same-tick test).
initialSegmentStartTime was pinned to .zero while the camera path starts its session at the first sample's capture-clock PTS (hours since boot) — and Apple's fMP4 authoring contract anchors segmentation boundaries at initialSegmentStartTime, so the mismatch risked a wrong first-segment duration or an immediate cut on exactly the path CI cannot exercise (review R2 #1). startWriting is now deferred to the first video sample: beginSession() sets initialSegmentStartTime to that sample's own timestamp and starts the session at the same instant, on the camera and test-pattern paths alike — which also covers a restarted writer, whose first sample is nowhere near time zero (the test-pattern frame counter is deliberately not reset).
- The stop-drain worst case is three rounds of the 60 s deadline, not two: the pump pops a segment before acquiring a window permit, so at cancel time up to max_backlog + 1 segments can sit behind a full window. The grace stays at 130 s — a drain that slow means every upload is timing out and the tail is lost regardless — but lib.rs and ant.h now say so instead of claiming nothing can outlive it (R2 #2). - ant_publisher_stop's docs no longer claim a second call returns the same report: that only holds for a broadcast that finished on its own; once a stop has returned and released the slot, another call fails with "not broadcasting" (R1 #2). lib.rs now matches ant.h. - AntNode.stopPublisher's comment said ~75 s where the bound is the 130 s stop grace (R1 #1).
|
Review minors — all 9 open findings addressed in
|
1 participant
Part of #67 (stage 2), implements #65.
Stage 1 (#72) built the publisher loop minus the camera and measured what it
sustains. Stage 2 swaps the synthetic generator for the real capture pipeline
and adds the two things a viewer needs: a rolling HLS playlist and a feed that
points at it.
What changed
Rust —
crates/ant-ffi/src/publisher.rs(new)The live loop, grown out of
bench.rsrather than beside it.Target, theloopback HTTP client,
publish_bzzanddata_chunk_countmoved intopublisher.rsas product code andbench.rsnow imports them, so the benchkeeps measuring exactly the call a broadcast makes. If the two ever drift, the
stage-1 numbers stop describing the product.
Per segment:
POST /bzzthe segment (and the fMP4 initialization segment whenever thewriter restarts).
POST /bzzthe playlist, then publish its reference as a sequence-feedupdate with
POST /soc— the bee-js shape (id = keccak256(topic ‖ index_be8), payloadtimestamp_be8 ‖ reference), so any bee gatewayresolves the channel.
POST /feedsat start yields the feed manifest thatis the one reference worth sharing.
Live-edge discipline, straight from the stage-1 findings:
DEFAULT_MAX_IN_FLIGHT = 4, documented as measured, not exposed as a UI knobDEFAULT_BITRATE_KBPS = 900,DEFAULT_SEGMENT_MS = 2000PublisherReport::kept_up#EXT-X-DISCONTINUITYTwo correctness properties the tests pin: segments commit in capture order
even though four uploads finish out of order (so the playlist is never
reordered), and a media segment whose initialization segment did not land is
never listed (a playlist no player can start is worse than a shorter one).
Direct gateway POSTs only — deliberately not
UploadManagerjobs: that isresume-oriented VOD tooling and a resumed segment is a segment nobody will ever
play. It becomes the right tool in stage 3.
C ABI:
ant_publisher_start/_push_segment/_progress/_stop,documented in
include/ant.h.Swift —
examples/ios-streamCaptureEngine.swift(AntStream: camera capture → HLS fMP4 segments (pure AVFoundation) #65) —AVCaptureSession(camera + mic) →VideoToolbox H.264 →
AVAssetWriterin.mpeg4AppleHLSmode emitting fMP4segments + playlist data. Bitrate and segment duration are runtime
parameters; a local ring buffer keeps the last two minutes of segments on
disk. Interruptions — incoming call, backgrounding, camera flip, orientation,
thermal downshift — finish the current writer so its last segment is complete
and playable, and recovery starts a fresh writer whose first media segment is
flagged discontinuous.
LiveBroadcast.swift— joins capture toant_publisher_*and owns theforeground keep-alive (idle timer held off, audio session configured for
recording, both released on every exit path). Stopping waits for the writer's
final segment to reach the publisher before closing its queue.
LiveView.swift— the going-live screen: preview, the publish-lagindicator (capture → the feed update that makes the segment playable) and
the dropped/failed counters. The badge's colour is the node's own
keeping_up, so the UI can't disagree with the report.LiveView.ant-side gaps found during integration
Both are on the exact write path a broadcast hammers, and both were found by
running the publisher against the real gateway:
502 Bad GatewayonPOST /bzzandPOST /soc— the retryable class. A broadcast walking abatch's collision buckets for an hour would retry forever against a
permanently full bucket and the UI would report a network error. One shared
upload_error_statusnow answers bee's402 "batch is overissued"everywhere, matching what
/chunks/streamand/stewardshipalready did.PushSochad none ofPushChunk's collision-bucket guard. SOCs are nota rare write — every redundant upload mints dispersed replicas as SOCs, and
a live feed writes one per playlist update — so on an immutable batch a feed
update surfaced as a bare
stamp issue failed: bucket full, and on amutable one it wrapped a bucket (evicting somebody's chunk) with no log at
all. It now takes the same guarded path, with the same actionable message.
Verification
cargo test --workspace— 35 test binaries, 0 failures.cargo clippy --workspace --all-targetsclean,cargo fmt --checkclean.publisher.rs): playlist assembly in capture order behindone
#EXT-X-MAP; a dropped segment becoming a discontinuity rather than asilent gap; a new initialization segment starting a new map and a
discontinuity; the window sliding with a correct
EXT-X-MEDIA-SEQUENCE/EXT-X-DISCONTINUITY-SEQUENCE; media before its initsegment never listed;
#EXT-X-ENDLISTon close; per-broadcast topicderivation; config validation.
a broadcast publishes segments, playlist and a feed update whose SOC ant's
own
soc_validaccepts, at exactlykeccak256(topic ‖ index_be8)forindex 0, with the bee v1 40-byte
ts ‖ refpayload; a full backlog sheds itsoldest segment and marks the gap; a rejecting gateway is reported rather than
counted as published (and no bogus playlist is published without an init
segment); pushing after stop is refused.
ant-gatewayrouter(
crates/ant-ffi/tests/live_publisher_gateway.rs): the production router inlight_modeon a loopback port, backed by a stub node loop, driven by thepublisher. Asserts every feed update reached the node at its sequence
address, none landed past the reported head, and the gateway never dispatched
an invalid SOC. This is what surfaced gap 1 above.
string the node actually emits maps to 402, and the other four error classes
keep their status.
antstream-visual.Not verifiable here, stated plainly: the camera path itself is device-only.
The simulator has no capture device, so CI runs the same encoder, segmenter and
publisher from a generated test pattern (labelled as such on screen), and its
uploads stop at the gateway's batch check because the runner has no real
postage batch — the same wall the stage-1 publish rows hit. A real
device broadcast (and the owed #67 device rows) still needs hardware and a
funded plan.