arq+mfsk: make the MFSK robust rung actually carry a transfer by rafael2k · Pull Request #160 · Rhizomatica/mercury

rafael2k · 2026-08-05T23:27:56Z

Supersedes #125. Same goal — the MFSK weak-signal rung as the ladder floor — but the data plane now actually works on a real-time audio path.

It works

Pinned to the MFSK floor (MERCURY_PIN_LADDER=0), two mercury processes over a PulseAudio bench:

TEST HOOK: ladder pinned to level 0 (mode 100)
B rx 90 -> 180 -> 270 -> 360 -> 450 -> 512/512
=== RESULT: 512/512 bytes in 98.1s (42 bps) match=True ===
MFSK frames decoded: 6      backlog flushes: 0

Six consecutive 13.5 s MFSK bursts, every one decoded, byte-exact, clean ACK round-trips. On the full ladder: 512/512 in 47.2 s (87 bps).

Seven bugs, each with a test that fails without the fix

commit	bug
`2058c43`	`ARQ_ACCEPT_RX_WINDOW_MS` was fixed at 9000 ms, sized for a DATAC15 first frame. The ladder floor is MFSK at 13.5 s, so the IRS gave up mid-burst and retransmitted ACCEPT on top of the ISS's data, going deaf for 3.7 s of every burst. Now derived from the ladder.
`c33e353`	`mfsk_downmix()` malloc'd and rebuilt the whole window per search — 1.7 MB, 215k trig calls, a 63-tap FIR over 107k samples, ~6x/s. Now incremental, carrier phase from the absolute sample index.
`eb31722`	The modulator clipped 45% of its own payload: `MFSK_TXAMP` 6000 against a 32767 rail gave a true peak of 42426 and PAPR 1.9 dB. Round-trip tests could not see it — both ends saw the same distortion.
`eb31722`	RX window sized for two bursts; with one, a burst was decodable for only ~0.48 s of slide.
`b1c527e`	Sync search trimmed to offsets that can actually decode; postamble fallback gated.
`c386102`	Backlog cap and ACCEPT window both derived from `arq_protocol_longest_burst_s()`. The cap was recomputed from the live payload mode and shrank to ~7 s mid-session, flushing bursts that were still arriving.
`eae2459`	The one that fixed the stall. `expect_pattern_ack` was true for all of CONNECTED, and modem.c uses it to gate a ~3-burst correlation run on every capture chunk — so a detector for a signal that can only arrive in one state ran all session, eating half the receive budget.

The last one is worth reading closely. Measured with the payload mode pinned away from MFSK, so the waveform was not involved:

branch:  consumed 3457-3587 samp/s   ring grew to 396,960 B   22/512 bytes
trunk:   consumed 7977-7992 samp/s   ring 480-840 B           512/512 in 39.1 s

Trunk has no pattern ACK, which is the entire difference. The receiver could not keep real time, the capture ring grew without bound, and the backlog guard then destroyed bursts mid-arrival — the "decodes one frame then stalls forever" behaviour on every continuously-fed transport.

Test additions

tests/modem/test_mfsk_modem.c now feeds the way the live funnel does (880-sample capture chunks, carrying the remainder — 880 is not a multiple of nin=320, and a loop that ignores that silently drops 240 samples of every chunk) and sweeps prior audio 0.08–25.6 s, trailing audio to 16 s, and a second burst after the window wraps. Plus an anti-clipping assertion on the TX samples themselves, which is the only way to catch a defect that a round trip cannot see.

tests/datalink_arq/test_arq_ladder.c pins the two ladder-derived guards against every rung.

Not done yet

Fading not yet characterised — everything above is a clean channel. A Watterson sweep is next; that is the actual point of a robust mode.
32-bit ARM unmeasured. Headroom on a laptop is 6.4–8.0k samp/s against 8k needed. If it is tight on a Pi, the lever is per-tone Goertzel detection (see modem73/phy/mfsk_modem.hh) instead of downmix + FIR + correlation.
42 bps at the floor vs 105 bps for trunk's ladder — expected for the robust rung, but it is the number to beat.
We key hard on/off; modem73's raised-cosine edge ramp is worth taking regardless of anything else.

Full unit suite green.

🤖 Generated with Claude Code

rafael2k · 2026-08-06T00:40:30Z

The fringe floor was the sync accept threshold, not the code

Pushed 453d509. Following up on the sensitivity question — the answer turned out not to be the LDPC, the LLRs, or anything in the demodulator.

mfsk_sync_search ended with a magic constant:

return (best_fine_metric < 0.5) ? -1 : best_fine;

The per-symbol statistic is |corr|²/(E_tmpl·E_rx) = SNR_sym/(1+SNR_sym), so a 0.5 gate demands SNR_sym ≥ 0 dB. That one number was the weak-signal limit of the entire mode — everything below it was thrown away before the demodulator ran.

The proof: the code doesn't matter

Sweeping all five ported LDPC rates gave byte-identical FER curves (2/20 and 17/20 at the same SNRs):

rate	K	payload	FER @ −1.1 dB	FER @ −0.1 dB
8/16	800	100 B	0.90	0.15
5/16	500	62 B	0.90	0.15
3/16	300	37 B	0.90	0.15
2/16	200	25 B	0.90	0.15
1/16	100	12 B	0.90	0.15

A 16× stronger code changing nothing means the limiter is upstream. Instrumenting the search confirmed it: at SNR3k −3.1 dB it returns off=-1, metric=0.344, payload never attempted.

What the detector actually sees

input	metric
noise only (600 searches, flat vs SNR)	0.037 – 0.049
signal @ −5.1 dB	0.219
signal @ −9.1 dB	0.091

The signal stood ~10× above the noise floor while the gate sat above the signal. 0.08 is 1.6× the measured noise maximum — 0 false syncs in 600 noise-only searches.

Results

	before	after
AWGN 50% FER	−0.6 dB	−11.2 dB (~10.6 dB)
Watterson 2 ms/1 Hz 50% FER	+5.0 dB	−3.7 dB (~8.7 dB)
Watterson error floor	0.25–0.30, never cleared	gone (0.00 at +3.9 dB)

That floor was sync failing during fades, not the decoder. DATAC15 measures −10.3 dB on the same harness, so the ladder floor now sits ~0.9 dB below the rung above it — which is the whole point of having a fringe rung.

The hazard that had to be fixed with it

A lower threshold admits more bad anchors, and they were not survivable: modem_mfsk caches the located anchor and marks it tried, so a false peak whose payload failed CRC left the decoder blind until it slid out of the window — a missed burst, not just wasted CPU. A resident payload that fails CRC now releases the anchor and the search restarts one symbol past it (safe: the search returns the earliest peak above threshold, so nothing decodable can hide before it).

Tests

Both new tests were verified to fail without their fix — the weak-preamble test fails when rebuilt with -DMFSK_SYNC_ACCEPT=0.5, and the failed-anchor test fails when the anchor release is removed. Neither burst in the latter carries a postamble; with one, the postamble fallback masks the defect completely (my first version of that test passed against unfixed code for exactly this reason).

Full C suite green, make integration-test green (bidirectional ARQ 112/112 both directions).

What I measured and deliberately did not ship

Rewriting the LLR derivation to codec2's non-coherent model (log-I₀ of amplitude + max* combining, replacing an energy difference with a ±5 clamp) moved the cliff by ~0 dB at both the old and the new threshold — byte-identical FER over 30 trials/point — while costing ~6% more decode time, which this mode cannot spare on 32-bit ARM.

Where the next dB are

Below ~−12 dB the signal metric reaches the detector's own noise floor, so no threshold goes further. That needs a detector that combines the preamble symbols instead of averaging per-symbol normalised ratios — averaging gains nothing from preamble length. Rate 1/8 buys only ~0.5 dB there, so the rung stays at rate 1/2 (all five codes share N=1600: rate costs payload, not airtime).

rafael2k · 2026-08-06T01:24:32Z

Second pass: the detector is not the bottleneck — measured, so I stopped there

Following up on "improve the detector too". I built the detector improvement, measured it, and it does not earn its place. Reverted; 453d509 remains the only shipped change. Here is the evidence, because the negative results are the useful part.

The oracle settles it

Bypassing the search entirely and decoding at the known true preamble offset:

	search-based	oracle (perfect timing)
rate 1/16 @ −13.1 dB	15/20	16/20
rate 1/16 @ −14.1 dB	1/20	2/20

Perfect knowledge of the burst position buys essentially nothing. Removing the accept gate altogether (CRC as the only detector) buys ~0.5 dB at −11 dB and does not move the −12 dB wall at all. Below ~−12 dB the limiter is the demod/decode chain, not acquisition.

I did write a ranked top-K candidate API (mfsk_sync_search_candidates) plus speculative decoding, since at the fringe the true peak still ranks first even when it no longer stands above the noise maximum. The oracle shows there is nothing to win, so it went in the bin rather than into the modem — it would have cost LDPC decodes on noise while idle, and MFSK RX CPU is already the live-path constraint.

The design curve — tone count beats code rate

AWGN, 50% FER, gate removed:

config	burst	goodput	50% FER
M=32, rate 1/2 (current)	13.1 s	59.8 bps	−11.4 dB
M=16, rate 1/2	16.3 s	48.0 bps	−12.2 dB
M=8, rate 1/2	22.0 s	35.6 bps	−12.9 dB
M=4, rate 1/2	32.6 s	24.0 bps	−13.5 dB
M=8, rate 3/16	22.0 s	12.7 bps	−14.1 dB
M=32, rate 1/16	13.1 s	6.1 bps	−13.5 dB

Two things fall out. 9 dB of code-rate reduction (1/2 → 1/16) buys only 2.1 dB — nowhere near theory. And M=4/rate-1/2 reaches the same −13.5 dB as M=32/rate-1/16 with 4× the goodput, so the code rate is strictly the worse lever. M=8/rate-3/16 is the first config that actually reaches the −14 dB figure.

LLR quality is not the floor either: log-I₀ + max*, and widening the ±5 clamp to ±40, both change nothing even at rate 1/16.

…but fading inverts the ranking, so I am not proposing the change

Watterson 2 ms/1 Hz, rate 1/2, shipping gate:

	50% FER	error floor
M=32	−4.5 dB	0.00
M=8	−5.8 dB	0.10
M=4	−7.3 dB	0.30–0.40, non-monotonic

A 22–33 s burst spans many 1 Hz fade cycles, so a deep fade kills the frame at any SNR — M=4 sits at 30–40% FER even on a good channel. M=8 would trade a 10% error floor and 40% of the goodput for ~1.3 dB. That is a bad deal for an ARQ rung, so M=32 stays.

Where the next real gain is

There is no time-interleaving across the burst. That is precisely why the lower-rate and lower-M configurations cannot cash in their coding gain on a fading channel — the code sees a contiguous wipe-out rather than spread errors. Interleaving is the change that would make the deeper configs viable; the detector and the code rate are both exhausted.

Worth noting the shipped fix already removed the sync-caused fading error floor (0.25–0.30 → 0.00 at M=32). What remains is genuine fade loss.

rafael2k · 2026-08-06T10:40:04Z

Time-interleaving implemented — `4579a69`

An LDPC block code corrects scattered errors far better than a contiguous run, and a fading HF channel delivers exactly the contiguous case: the burst is 13.5 s, so a deep fade wipes a stretch of consecutive symbols and, with the coded bits laid down in order, a stretch of consecutive codeword bits with it. The decoder was being handed the one error pattern it is worst at.

Transmitted slot t now carries coded bit ilv[t] — Fisher-Yates from a fixed xorshift seed, integer arithmetic only, so both ends derive an identical table on any platform with nothing sent over the air. Pseudo-random rather than a fixed stride: these are quasi-cyclic codes (N=1600, circulant 100) and an arithmetic progression risks landing the damage on the same check nodes.

Measured — 60 trials/point, M=32 rate 1/2, builds verified to differ only in the interleaver

channel	SNR3k	without	with
Watterson 2 ms / 1 Hz	−3.1 dB	0.35	0.20
Watterson 2 ms / 1 Hz	−0.1 dB	0.08	0.07
Watterson 2 ms / 0.2 Hz	−4.4 dB	0.58	0.47
Watterson 2 ms / 0.2 Hz	−1.6 dB	0.25	0.18
AWGN	all	0.00	0.00

Better or equal at every point, never worse. At the 1 Hz fringe that is 21 failures out of 60 down to 12 — expected transmissions per frame 1.54 → 1.25, about 19% more ARQ throughput where it is needed, for zero airtime and a table lookup per bit. AWGN is unchanged, as it should be: interleaving buys nothing against white noise.

The gain is smaller than one might hope, for a reason worth writing down: at 1 Hz Doppler the fade coherence time is ~0.5 s, so a 13.5 s burst already spans ~26 independent fades and carries intrinsic time diversity. Interleaving helps most when a single fade covers a large fraction of the burst — which is why the 0.2 Hz slow-fade case gains at least as much, and that is the realistic NVIS condition.

What it did not do

It does not rescue the deeper-M configurations. M=4 keeps its ~0.40 error floor at −0.2 dB with interleaving in place, so my earlier hypothesis that interleaving would make M=4/M=8 viable is wrong — M=32 stays the rung. The M=4 floor is non-monotonic in SNR, so it is not noise-limited; I have not chased it further since the config is not a candidate.

Two defects found while wiring it up

h->llr was allocated as code->N floats, but mfsk_demod writes one LLR per transmitted bit slot (NPAY*bps). For M=32 both are 1600 so nothing overflowed — M=8 gives 1602 slots and a two-float overrun. Buffers are now sized by nb.
mfsk_be_close never freed h->bb: rxcap complex doubles, ~3.4 MB leaked on every open/close cycle, and the mode pool opens/closes per session.

Unit test covers permutation validity, cross-platform determinism, and the spread property. Full C suite green; go test -count=1 green (the earlier "ok (cached)" was Go reusing a result — the test binary hadn't changed even though mercury had).

Note this is an on-air bit-ordering change — both ends must run it. Pre-2.0 has no compatibility constraint and MFSK has not shipped in a release.

rafael2k · 2026-08-07T14:32:17Z

Frequency search — `ba1dce9`. Plus two ideas measured and rejected.

Read modem73 (public domain) looking for things to borrow. Its MFSK is almost the same waveform as ours — 32 tones, 1000 Hz, 5 bits/symbol, rate 1/2 — but with a weaker code (K=7 Viterbi against our LDPC N=1600), so a sensitivity bake-off isn't worth running. Three ideas looked worth having. One was, two weren't.

1. Frequency search — landed, and it fixes a real defect

We had no frequency tolerance whatsoever. Measured on the pre-fix decoder, clean channel:

dial offset	before	after
12 Hz (0.38 bins)	10/10	10/10
16 Hz (0.51 bins)	0/10	10/10
47 Hz (1.50 bins)	0/10	10/10
94 Hz (3.01 bins)	0/10	10/10
110 Hz (3.52 bins)	0/10	0/10 (past ±3-bin design range)

16 Hz — half a subcarrier — and the mode is stone deaf. That is well inside what two HF radios drift apart by. Every sensitivity number in this PR was measured on a perfectly tuned simulator, so the entire test suite was blind to it; on air it would have presented as "the robust mode just doesn't work", with nothing in the logs to explain why. Tuning tolerance is now ±12 Hz → ±100 Hz.

Two requirements that weren't obvious:

Half-bin steps, not whole. A whole-bin grid leaves the worst case exactly between two hypotheses, where tone energy splits across two bins and neither template captures it. Measured: 31/62/94 Hz recovered to 10/10, but 16/47/110 Hz stayed at 0/10.
Argmax, not first-past-the-gate. At half-bin spacing a neighbour 0.5 bin off still clears the threshold, so first-past latches the wrong offset — acquisition "succeeds" and every frame then fails CRC. The tell was 94 Hz going 10/10 → 0/10 when the grid got finer, which is backwards.

Cost is contained by latching: a CRC pass confirms the offset and later searches try that one hypothesis alone, and a resident payload failing CRC drops the lock so a wrong latch can't wedge the decoder. +21% decode CPU with signal present, none idle. No regression: AWGN cliff identical point-for-point at zero offset (20/20, 18/20, 11/20, 0/20 over −9.1..−12.1 dB SNR3k), false alarms still 0/200 despite 13× the hypotheses.

2. Goertzel detection — not worth doing

Profiled the decoder: 0.23% of real time. 0.18 s CPU per 78.7 s of audio — downmix 0.067 s, sync 0.064 s over 78 calls, payload 0.049 s over 6 bursts. On idle noise sync is 0.002 s over 156 calls, because the per-symbol floor rejects instantly.

The old "MFSK RX runs at half real time" was real, but it is already fixed — incremental downmix, search trim, and above all the pattern-ACK gate. Even 20× slower on a Pi that is ~5%. There is nothing left to optimise, so I'm not adding a second tone detector to maintain.

3. Smaller frames — rejected, it would make fading worse

modem73 uses 4.42 s frames against our 13.12 s, which looked like the obvious answer to our fade behaviour. It isn't.

First, arithmetic: a shorter frame needs a shorter codeword, and all five of our codes are N=1600. H_256_512_4 in the freedv tree isn't IRA/staircase, so our {C,Enc} format would need an encoder derived — real work. Before doing it I tested the premise. FER vs Doppler at fixed SNR (No=−12, 2 ms, 30 trials):

Doppler	independent fades per burst	FER
0.1 Hz	2.6	1.00
0.2 Hz	5.2	0.30
0.5 Hz	13.1	0.23
1.0 Hz	26.2	0.17
2.0 Hz	52.5	0.20

The burst relies on spanning many independent fades. Cutting it 3× divides that by 3: at 1 Hz, 26 → 9 fades (FER ~0.17 → ~0.27); at 0.2 Hz, 5.2 → 1.7, which the curve puts at a dead link. Slow NVIS fading is the case that matters, and cheaper retransmits don't come close to offsetting that.

It also retro-explains the interleaving result earlier in this PR: interleaving helped more at 0.2 Hz than at 1 Hz precisely because the slow case has so little intrinsic diversity to start with.

modem73 can afford short frames because their MFSK is a backup mode behind an OFDM/ROBUST ladder, and their published NVIS data sits at 0–16 dB SNR where margin is ample.

Where this leaves the rung

AWGN 50% FER −11.2 dB, fading cliff ~−4.9 dB with the sync-caused error floor gone, ~19% less retransmission under fade from interleaving, and now usable on a radio that isn't perfectly zero-beat. The remaining ~13 dB to capacity is not reachable by tuning this waveform — that needs coherent, pilot-aided modulation (modem73's ROBUST family territory), which is a separate piece of work.

Still OTA-pending, and note the interleaver and this change are both on-air format changes: Pedro's run needs matched builds on both stations.

…ison Pure-C port of Mercury v1's cl_mfsk (non-coherent M-FSK over OFDM subcarriers, originally Fadi Jerji): class -> struct+functions, std::complex<double> -> a plain {double re,im} struct, std::isfinite -> C99 isfinite; all tone tables (preamble, Welch-Costas ACK/BREAK, Sidelnikov NB, directed-HAIL FNV-1a suffix) carried over. Compiles clean (-Wall -Wextra); tests/modem/test_mfsk.c proves mod->demod is lossless (M=4/8/16/32) and that higher M is more robust. Measured (uncoded non-coherent BER vs Eb/N0, freq-domain AWGN/Rayleigh): v1's 32-MFSK buys ~6 dB over 2-FSK on AWGN/steady channels but only ~1.5-2 dB under fast fading. Combined with the earlier finding (2-FSK FSK_LDPC already beats DATAC15 on MPP), the fringe lever is going non-coherent FSK; 32-MFSK adds real gain on calm-NVIS/steady channels. This C port makes v1's 32-MFSK available to wire into a v2 weak-signal ARQ-ladder bottom rung (framing/sync/LDPC glue TBD). See docs/MFSK-PORT.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The real question for a weak-signal mode is acquisition, not just modulation. Record what is confidently known: v1's MFSK carries its own non-coherent preamble detector (not OFDM coherent sync), and the codec2 FSK_LDPC end-to-end figures (acquisition included) already beat DATAC15 at the fringe — so the non-coherent-FSK class is not acquisition-limited the way OFDM is. Also record the limitation: an isolated MFSK acquisition sim confirmed detector mechanics but could not be calibrated to the SNR3k axis; a trustworthy MFSK-specific floor needs end-to-end integration (framing + time_sync_mfsk_corr + LDPC in ch), the next step. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The MFSK-relevant slice of v1's cl_ofdm: self-contained radix-2 FFT/IFFT (no FFT-library dependency), subcarrier zero-pad/de-pad (DC-centred v1 layout), and cyclic-prefix add/remove. fft is 1/N-normalized, ifft unnormalized (v1 convention) so the pair is identity. This carries mfsk.c's tone bins to/from OFDM time-domain symbols — the framing layer needed to run the MFSK mode through a real channel end-to-end. Validated by a round-trip unit test (bins -> pad -> ifft -> +CP -> -CP -> fft -> depad recovers bins at machine precision). Next: port time_sync_mfsk_corr (non-coherent acquisition) + baseband channel + LDPC to measure the MFSK acquire+decode floor vs DATAC15. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Pure-C port of v1 cl_ofdm::time_sync_mfsk_corr: non-coherent (envelope) matched-filter preamble detection — two-phase search (coarse 4x-oversampled + fine), per-symbol normalized correlation averaged over the preamble symbols, per-symbol floor 0.05, threshold 0.5, earliest-strong-preamble early exit. Needs no phase/frequency lock, so it acquires below the coherent-OFDM floor — this is the acquisition path the MFSK weak-signal mode uses (not OFDM coherent sync). mfsk_sync_build_template() builds the preamble time template via the mfsk + ofdm framing. Unit test plants the template in a noise buffer: detected at the right offset (high SNR), rejected on pure noise. With mfsk.c (codec) + mfsk_ofdm.c (framing) + mfsk_sync.c (acquisition), the MFSK mode's TX/RX signal-processing core is now in C. Remaining for an SNR3k-comparable end-to-end floor vs DATAC15: baseband<->passband + carrier + interpolation chain through ch (large; the SNR3k calibration lives there). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…s the concern) Wired the ported MFSK core (mfsk.c + mfsk_ofdm.c + mfsk_sync.c) into a real passband pipeline through codec2 `ch` (SNR3k measured exactly as for DATAC15). Result: v1's non-coherent MFSK acquisition holds (metric 0.6-0.9) far below DATAC15's ~-7 dB acquisition wall — to ~-13 dB SNR3k on AWGN (matching v1's claim) and past -11 dB on MPP fading (uncoded BER <=3%, where DATAC15 delivers 22%), before any LDPC. Acquisition is NOT the bottleneck; the mode carries its own. Replaces the earlier "couldn't measure trustworthily" caveat with the ch-calibrated numbers. Remaining work is wiring it as an actual ARQ-ladder mode (LDPC + mode-pool + OLLA), not whether it can acquire. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…floor) Port v1's ROBUST_0 FEC — rate-1/16 LDPC (N=1600, K=100): systematic IRA encoder (mfsk_ldpc_encode) + normalized min-sum BP decoder (mfsk_ldpc_decode) over the v1 quasi-cyclic Tanner graph, matrix embedded in mfsk_ldpc_1_16.c. Verified: encoder produces valid codewords (H*c=0), noiseless encode->decode lossless (unit test). Wired end-to-end through the passband/ch pipeline (TX info->encode ->32-MFSK->passband->ch->RX acquire->demod->LDPC decode). Coded floor (15 frames/point): delivered == acquired everywhere — once acquired the code always decodes, so the mode is acquisition-limited, not decode-limited. Coded MFSK delivers ~2-3 dB deeper than DATAC15 on fading (47% at -10.8 dB SNR3k vs DATAC15 22%; 33% at -12.8 where DATAC15 ~0). Next lever is acquisition (postamble / longer preamble), not the code. See docs/MFSK-PORT.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…ison Since the coded MFSK mode is acquisition-limited, add a postamble: a second known-tone sequence (distinct tones, +2 offset) after the payload, so RX syncs on preamble OR postamble. mfsk_generate_postamble + mfsk_sync_build_postamble_ template; harness does dual-ended search. Measured (MPP, coded, same rx file): dual-ended buys ~2 dB — 87% vs 60% delivery at -10.8 dB SNR3k, 100% vs 80% at -8.8. Full mode comparison (MPP, delivered %): dual-ended coded MFSK 100/87/67/20 at SNR3k -8.8/-10.8/-12.8/-14.8 vs DATAC16 63/40/20, DATAC15 50/33/3, DATAC1/DATAC3 0. The MFSK weak-signal mode extends the floor ~3-4 dB below the most robust OFDM modes, at ~8 bps (deep-fringe bottom rung). Unit test added for the postamble. See docs/MFSK-PORT.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

… was wrong Generalise mfsk_ldpc to a code-descriptor + registry and port v1's full rate ladder: 1/16, 2/16, 3/16, 5/16, 8/16 (rate 1/2), all N=1600 (same airtime), payload K/8 = 12.5..100 bytes. Generic systematic IRA encoder + min-sum decoder take a mfsk_ldpc_code_t; unit test validates encode->H*c=0->noiseless-decode for all five rates. Measured (32-MFSK, dual-ended, MPP through ch): all rates deliver identically (delivered==acquired; even rate 1/2 decodes everything acquired down to the acquisition floor, since the 32-MFSK demod hands the code very clean LLRs). So the LDPC rate barely affects the floor — it only sets payload. 1/16 (12.5 B) was the wrong default: it can't even fit the 14-byte CONNECT frame, whereas 8/16 (100 B, ~60 bps, ~DATAC15 throughput at ~3-4 dB more robustness) does. Recommend shipping a small ladder (5/16, 8/16) for OLLA. Caveat: "rate is free" holds in idealised sim (no freq offset / drift over the ~13 s frame); real HF may favour shorter frames — set the ceiling OTA. See docs/MFSK-PORT.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Port cl_ofdm::detect_ack_pattern (Phase-1 peak-bin match count) to pure C as mfsk_detect_pattern in mfsk_sync.c, over the depadded-bins framing (same path as mfsk_demod) rather than v1's raw-FFT-bin + carrier-image handling, which this pipeline's LPF+depad already covers. Measured pattern-ACK vs a codec2 DATAC16 ACK through ch/Watterson: an ACK burst is 0.64s (vs 3.74s DATAC16, 5.8x less airtime) and survives ~10-12 dB deeper on fading -- 100% to ~-9 dB / >75% to ~-13 dB on moderate+poor, where DATAC16 is dead by ~-5 dB. Pure-noise false-alarm gate: 0/30 down to -26 dB. This directly addresses the reverse-path ACK-survival blocker; confirms v1's split of coded MFSK frames (data + CONNECT) vs Welch-Costas patterns (ACK/BREAK/keepalive). Unit test: planted ACK detected, noise rejected, BREAK not confused for ACK. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Mercury's datalink is codec-agnostic (opaque frame bytes + CRC in/out), but the modem layer historically called codec2/FreeDV directly. Introduce a modem_backend_t vtable (modem_backend.h) so more than one modem architecture can coexist behind one seam -- the original mercuryv2 design intent. FreeDV becomes a first-class backend (modem_freedv.c), a 1:1 adapter over the freedv_api. - generic_modem_t and rx_decoder_state_t now hold a modem_codec_t {backend, ctx} instead of a struct freedv *. - The per-mode pool is a generic slot array keyed by mode (was a hand-enumerated freedv-only struct); clear/open/lookup dispatch through the backend. - Both DSP funnels (send_modulated_data TX, rx_decoder_consume_chunk RX) plus bind/geometry/bitrate/spectrum route through the vtable; no freedv special-case. - CRC16 (freedv_gen_crc16) stays codec-independent in the TX funnel. - mod_out_short sized to max(pre/frame/postamble) so a burst backend can't overflow it. No behaviour change: full unit suite green; TX output byte-identical to the former direct-freedv path across DATAC15/16/1/3/4/17/QAM16C2 (parity harness). This is the foundation for the MFSK weak-signal backend (Stage 2). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Add modem_mfsk.c: the non-coherent 32-MFSK burst codec behind the modem_backend_t vtable, mode MERCURY_MODE_MFSK (100, clear of the freedv enum). - TX: preamble_tx/rawdata_tx/postamble_tx build baseband OFDM (rate-1/2 LDPC, 100-byte frame) and mix to the 2 kHz passband as int16, phase-continuous across the burst. Same frame contract as freedv (98 payload + 2 CRC16). - RX: keeps its own sliding sample window and presents the freedv-style nin/rawdata_rx interface -- accumulates symbols, downmix+LPF, non-coherent preamble correlation (mfsk_sync_search), energy demod, LDPC decode, and returns bytes ONLY on CRC16-valid (freedv's contract). Detection is rate- limited to a few attempts per burst. - backend_for_mode() routes MERCURY_MODE_MFSK to it; the mfsk sources are now linked into the mercury binary and the UI core lib. Not yet a selectable mode (no pool/ladder entry) so default behaviour is unchanged -- that is Stage 3. Deterministic round-trip unit test: TX a frame -> passband -> feed the RX in nin-sized chunks -> exact bytes back, CRC-gated; pure noise never false-decodes. Full suite green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Make MERCURY_MODE_MFSK a selectable payload mode reached only when the OLLA- corrected SNR drops below the DATAC15 floor (~-11 dB) -- deep fade only, so default behaviour is unchanged. modem.c: MFSK is now a supported + payload split mode (RX payload decoder binds to it when the peer TXes MFSK), registered in the pool, named, and given a bitrate level. arq: arq_mode_table gets the MFSK row (unique 98-byte payload so DATA-frame mode inference stays unambiguous; ~13.5s frame, generous ack/retry). New ARQ_SNR_MIN_DATAC15_DB (-11) sets the DATAC15->MFSK drop point; select_best_mode terminal now returns DATAC15 above it and MFSK below, with the standard +5 dB upgrade hysteresis so the link won't oscillate into the long MFSK frame. mode_rank(MFSK) = -1 (below DATAC15); MFSK entry is OLLA-driven, not retry-count (the hard-loss floor stays DATAC15). Session still starts at DATAC15. Deterministic test (test_arq_olla): DATAC15 held to -11, drops to MFSK at -12, and climbs back only above -6 (threshold+hysteresis). Full suite green; binary builds. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Replace the SNR/OLLA-negotiated, go-back-N, restage-buffered data-flow FSM with a simple delivery-driven stop-and-wait protocol that starts at the most robust rung (MFSK) and climbs. arq_fsm.c drops from 2466 to ~1470 LOC. - Data-flow FSM: 15 -> 5 states (IDLE_ISS/DATA_TX/WAIT_ACK/IDLE_IRS/ACK_TX); connection FSM (connect/disconnect handshake) unchanged. 21 -> 11 states. - Control ACK is now a Welch-Costas PATTERN (0.64s, ~10dB more robust than the 3.74s DATAC16 ACK it replaces) with a 2-symbol alphabet: ACK / ACK+TURN (== HAS_DATA piggyback). Emitted via ARQ_ACTION_TX_PATTERN + send_pattern_ack in modem.c; detected by a third RX consumer (mfsk_pattern_detect) that synthesizes ARQ_EV_RX_ACK. mfsk_pattern_tx/detect added to modem_mfsk. - Mode control: delivery-driven ladder MFSK->DATAC15->..->QAM16C2 indexed by speed_level (fast initial ramp, then N-clean-per-step; any retry steps down). NO SNR, NO OLLA offset, NO reverse-hold gate. Host SNR display is unaffected (it comes from decoding received DATA frames, not the ACK). - TX buffering: one immutable retained frame (raw user bytes, fixed seq<->content) instead of the 5-slot window + restage. A mode drop re-frames the same bytes at the smallest mode that still fits (mode_that_fits) -- never resized, so a duplicate is idempotent on the peer. - Deleted: MODE_REQ/ACK + TURN_REQ/ACK + KEEPALIVE states/handlers/builders, select_best_mode/maybe_upgrade_mode/OLLA/restage; wire flags TURN_REQ/ CTRL_ACKSEQ/BURST_END; SNR/OLLA/hard-loss/mode-hold constants. IRS liveness now via the no-progress net (replaces keepalive). Turn handoff is piggyback-only, with a CALLER-keeps-floor role tiebreak on simultaneous bids. Asymmetric links: each station adapts its own forward mode from its own delivery outcomes; the robust pattern ACK survives the weak reverse path, so the healthy forward mode is not spuriously downgraded (the reason the old reverse-hold gate existed) -- validated by test_sim_asymmetric_strong_forward. Tests: full suite green (172 tests). test_arq_fsm rewritten for the 5-state flow; test_arq_olla deleted -> test_arq_ladder (deterministic climb/drop, MFSK start); new test_pattern_ack_detection (DSP: 0 false ACKs on noise, ack/break discrimination); test_arq_sim gains pattern roundtrip, fade->MFSK+recover with byte-integrity, bidirectional piggyback, lost-ACK idempotency, asymmetric strong/weak. Sim models the pattern ACK as a short-airtime outframe with per-direction channel SNR. Bugs the sim caught and fixed in logic (not hidden): mode-drop duplicate-byte corruption (immutable frame), bidirectional double-ISS deadlock (role tiebreak), IRS-never-disconnects (inactivity net). Flat-erasure fuzz ceiling lowered 0.40-> 0.25: at flat high erasure 'drop-on-retry' crawls (no stall/corruption -- probed to completion); the accepted slower-at-high-loss tradeoff, tuned at OTA. OTA-gated (Pedro, Sao Roque <-> Belo Horizonte) before merge to mercuryv2. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The ARQ rethink passed the deterministic sims but failed the Go integration harness (TestMercuryARQTransfer, real codec path over the ch bridge): the session never connected, and once connect was fixed the MFSK-floor data delivered 0 bytes. Two independent root causes, both in the live RX loop that the abstract sims don't exercise: 1) Connect handshake starved by the pattern-ACK detector. The new Welch-Costas pattern detector (3rd RX consumer) ran its sliding- window correlation on *every* chunk, gated on arq_policy_ready — which is always true once the engine is up. That per-chunk cost slowed the RX loop enough to miss the DATAC16 ACCEPT on the timing-marginal CALL/ACCEPT turnaround, so the caller retried CALL forever and never connected. A pattern ACK can only arrive in two states (answerer awaiting the connect- confirm = ACCEPTING, or a live session awaiting a data ACK = CONNECTED), so gate the detector on a new snapshot flag expect_pattern_ack and skip it during CALLING/LISTENING/idle. (mercuryv2 has no such consumer and connects fine — this was a pure regression of the rethink.) 2) MFSK data burst chopped by the RX backlog flush. The capture-backlog flush used a fixed 2 s cap (sized for fast FreeDV frames). An MFSK payload frame is a single ~13.5 s burst; the 2 s cap flushed it ~6x mid-burst, so the MFSK sliding-window sync never saw a contiguous burst and decoded 0. Size the cap to the active payload frame duration + 3 s guard so an in-flight burst is never dropped, while fast modes keep the tight 2 s latency bound (preserves the issue 81 protection). With both fixes the harness passes end-to-end starting at the MFSK floor (full 102 B delivered, 0 backlog flushes); DATAC15-start also passes. Full unit+sim suite green; backtoback/control/bidir/large_queue integration green. Debugging note: mercury's own -v (DEBUG logging) is heavy enough to perturb the marginal connect timing (a Heisenbug) — validate with exit codes + a single promoted INFO log, not full -v. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…ean checkout test_mfsk_modem links ../modem/freedv/libfreedvdata.a, but that library is produced by the freedv sub-build, not by tests/Makefile — so a clean checkout (CI's UT job runs `make test` with no prior build) failed to link it ("ld returned 1 exit status"). This predates the connect/MFSK fixes; it only passed locally because a prior full build had left the .a in place. Add libfreedvdata.a as a prerequisite of test_mfsk_modem with a rule that runs the freedv sub-build, so `make test` is self-contained from clean. Verified: rm the .a + full `make -C tests test` from clean → all tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Follow-up to the dialed-SSID callback change (notify_connected/notify_ pending now carry the local call). test_arq_ladder.c is created on this branch by the data-plane rewrite, so the mercuryv2 SSID fix could not update its FFF fakes during the rebase; bring them to the 2-arg signature so the branch builds. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Rebased mfsk-arq-integration onto mercuryv2 which now carries PR #127's HARQ combined-decode parity gate (freedv_700.c + mpdecode_core.h come in via the base cleanly). Re-add PR #127's test_freedv_harq to this branch's (divergent) tests/Makefile — TEST_BINS entry + build rule — which the rebase dropped while resolving the Makefile toward each MFSK commit. Full suite green incl. test_freedv_harq 3/3. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The MFSK RX synced only on the preamble at the burst head, so a burst whose head was clipped -- the fragile part of a half-duplex burst (far end still keyed, PTT releasing, AGC/T-R turnaround settling) -- was unrecoverable and cost a full ACK-timeout retransmit. Over the -x sock virtual clock this stalled the transfer: the ISS starts its first ~13.5 s data burst while the IRS is finishing its connect turnaround, so the IRS drains the burst's preamble and never decodes it (observed: burst 1 best_metric=0.078 NO preamble; a later clean retransmit metric=0.885). The postamble is the same P known symbols emitted after the data; its template was already built (h->pstT/pstE/pstN) but unused. When the preamble path fails, search for the postamble and anchor the payload from the tail (payoff = poff - NPAY*Nofdm). Recovers head-clipped bursts; a general HF robustness win, not just for -x sock. Test: test_mfsk_modem_preamble_clipped_recovers_via_postamble drops the preamble entirely and requires the exact frame back. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The IRS payload decoder tracked peer_tx_mode = the last-DECODED mode (arq_modem_preferred_rx_mode() is hardcoded to DATAC16, so select_payload_rx_mode's first branch is dead). When the ISS climbed the ladder (e.g. MFSK->DATAC15) the IRS's dual decoder had no slot for the new mode, missed the first burst of every climb, never ACKed it, and the ISS retried and fell back -- the transfer oscillated at the MFSK floor and crawled. The IRS observes the same per-frame outcomes the sender climbs on, so it now mirrors the same delivery-driven ladder: a clean new frame climbs, a duplicate (sender retried and stepped down) steps down, and a full IDLE_IRS idle-hold with no RX steps down toward the floor (reset-on-miss, so a lost ACK that left us climbed above the sender re-rendezvous at the floor). Keeps the payload decoder on the mode the peer's NEXT burst will use, with no on-wire mode negotiation. ladder_step() is factored out of record_tx_outcome so both ends apply identical rules. Tests: test_irs_mirror_climbs_with_peer / _steps_down_on_duplicate / _resets_toward_floor_on_silence. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The data-plane rethink rewrote arq_fsm.c against a base that predates the LISTEN OFF fix (#141), so rebasing onto mercuryv2 would silently drop it and merging this branch would reintroduce the BPQ32 interlock bug. Re-applied against the new state machine: - CALLING / ACCEPTING: fall through to the APP_DISCONNECT teardown. - CONNECTED: abort semantics — no backlog drain, no air-side DISCONNECT frame; the peer times out. Deliberately unlike APP_DISCONNECT, which defers to drain. - DISCONNECTING: stop retransmitting DISCONNECT. Same three tests as on trunk, rewritten against the new fixtures. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…n start) Test hook only, off unless the env var is set. Pins the delivery ladder to one rung so a chosen payload mode is exercised on every run instead of whenever the channel happens to take the link there. Also pins the SESSION START, not just apply_speed_level(): the initial payload_mode/peer_tx_mode are assigned straight from arq_mode_ladder[0], so a pinned run still opened on MFSK and answered nothing. With that closed, pinning to DATAC15 gave the result this branch exists to get: the transfer still fails with MFSK entirely out of the picture.

…residency The existing round-trip put the burst 2 symbols into the receiver's window and fed it in nin-sized pieces. Neither matches the air: the IRS has been listening for seconds before the ISS keys, and modem.c hands the backend 880-sample capture chunks, not nin. Adds a feeder that mirrors rx_decoder_consume_chunk (accumulate, drain in nin-sized pieces, CARRY THE REMAINDER -- 880 is not a multiple of 320, and a loop that walks the chunk in nin steps silently drops 240 samples of every 880, which is a corrupted waveform, not a test), plus a sweep of how much prior audio precedes the burst: 2 to 640 symbols, i.e. 0.08 s to 25.6 s, spanning and exceeding one RX window. All points decode. That is the useful result: it clears the decoder and the window bookkeeping of the loopsim 0-byte failure, which had looked like a residency bug -- the preamble was found (metric 0.889 vs 0.168 noise) and the payload never fit. It is not residency. The real cause is a collision: the IRS retries ACCEPT every ~12.7 s while the ISS's MFSK data burst lasts 13.53 s, so the IRS keys up inside every burst, goes deaf for 3.7 s of it, and destroys the very frame that would tell it to stop. Trunk is unaffected because a DATAC15 burst is ~4 s and fits between retries; -x sock is unaffected because lockstep serialises the transmissions. These tests do not cover that -- it is an FSM timing invariant (ACCEPT retry cadence must exceed the ladder floor's burst, 13.50 s per arq_protocol.c) and belongs in the FSM tests. They do lock down the DSP side so the next person does not re-suspect it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

ARQ_ACCEPT_RX_WINDOW_MS was fixed at 9000, and its own comment says how it was arrived at: "ISS_guard(900)+DATAC15(4400)+margin(3700)". That was right when DATAC15 was the first data frame an IRS could expect. The ladder floor is now MFSK at 13.50 s, so the window was SHORTER than the burst it exists to wait for. Consequence, measured on loopsim: the IRS gave up mid-burst and retransmitted ACCEPT on top of the ISS's data every 12.7 s (9 s window + 3.71 s ACCEPT TX) against a 13.53 s burst. While transmitting it is deaf -- capture is drained, not decoded -- so it destroyed the very frame that would have told it to stop, then did the same to the retransmission. The session never left ACCEPTING and the transfer delivered 0 bytes. Trunk never saw it because a DATAC15 burst is ~4 s and fits between retries; -x sock never saw it because lockstep serialises the transmissions. Compute the window from the ladder instead: guard + longest rung + margin. Derived rather than hand-picked, so a future rung slower than some constant nobody remembered to bump cannot bring this back. For today's ladder that is 900 + 13500 + 3700 = 18.1 s, against 9.0 s before. test_accept_rx_window_outlasts_longest_ladder_burst pins the invariant against the ladder table rather than a literal, and fails on the old value with the diagnosis in the message. Verified it does: with 9000 restored it reports "ACCEPT RX window is shorter than the slowest ladder burst". On loopsim the IRS now reaches CONNECTED (+32.3 s) instead of falling back to LISTENING, and keys exactly once for its ACCEPT instead of inside every burst. NOT yet fixed: MFSK data frames still do not decode on the receiver even with a clean uncontended burst (rx_bytes=0), so this closes the collision and not the transfer. The decoder itself is cleared by the residency sweep in tests/modem/test_mfsk_modem.c. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…e window mfsk_downmix() malloc'd a buffer the size of the entire RX window and rebuilt it end to end on every sync search: 107520 samples, a 1.7 MB allocation, 215k trig calls, and a 63-tap FIR over the lot -- 6.8M complex MACs, roughly six times a second. The RX thread could not keep up. Measured on loopsim, the window slid at about 3400 samples/s against 8000 arriving, so the capture ring dropped audio and a 13.5 s MFSK burst was never contiguous: the preamble was found every time (metric 0.889 against a 0.168 noise floor) and the payload could never fit behind it, whatever the buffer size. That is why -x sock passed -- a virtual clock has no real-time pressure to fall behind. Downmix and filter only the samples that arrived since the last call. Carrier phase now comes from the ABSOLUTE sample index, so sliding the buffer no longer rotates already-computed samples, and bb/bf are slid alongside rxbuf to stay aligned. bf[i] is only final once bb has reached i + TAPS/2, so the search and the payload bounds work against bf_len rather than rxlen. Cost per search drops from O(window) to O(new samples), ~84x less work here. That is what makes this mode viable on 32-bit ARM at all, where it has to run faster than real time on a Pi. Correctness is covered offline by the residency sweep in tests/modem/test_mfsk_modem.c -- an incremental filter that is subtly wrong still "runs", it just stops decoding -- and all points from 0.08 s to 25.6 s of prior audio decode, plus noise still never false-decodes. On loopsim the burst now slides to within ~860 samples of fitting (off 4669 against the 3809 it needs) before the decoder's window is reset, so this is a necessary step and not yet the whole story: the payload decoder is losing its accumulated buffer mid-flight. Transfer still delivers 0 bytes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…ow for two Two findings from replaying a live burst through the backend offline. 1. MFSK_TXAMP was 6000, applied straight to the IFFT output of a 32-carrier OFDM symbol. Measured, the true peak was 42426 against a 32767 rail, so mfsk_emit() hard-clipped 45% of every payload it produced and PAPR collapsed to 1.9 dB -- the burst was closer to a square wave than to OFDM. This never showed up in the round trip because TX and RX both saw the same deterministic distortion; on the air the clipped subcarriers lost orthogonality and the LDPC payload never passed CRC, while sync still peaked at 0.889 because correlation shrugs clipping off. 2200 puts the peak at ~15.5k, about -6.5 dBFS, leaving room for the operator's TX gain. Found by dumping the receiver's resident window at the moment the burst was fully present but failed, then decoding those exact samples offline: they would not decode there either, which moved the search from the decoder to the waveform. Sample statistics then showed 1.7% of the received window pinned to the rail. 2. The RX window held one burst plus 3840 samples, so a burst was decodable only during the ~0.48 s between the data ending and the preamble sliding out the front. The search, running every 4 symbols, kept missing it: `off` never even reached 8000. Two bursts of window make it decodable for a whole burst-length of slide, which is the difference between "sometimes" and "reliably". test_mfsk_modem_tx_does_not_clip asserts headroom on the samples themselves -- the thing that was actually wrong -- since a round-trip test cannot see this. On loopsim this produces the first MFSK data frame ever decoded over the live audio path (100 bytes at +62 s), where every previous run in this session decoded none. The transfer still does not complete: only one burst in about six decodes and the sender never advances past seq=0, so ACK return and the remaining margin are the next thing to chase. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Two searches per attempt were doing work with no chance of yielding a frame. The preamble search scanned the whole window, but a preamble found beyond bf_len - (P+NPAY)*Nofdm has no resident payload behind it and cannot be demodulated; with the two-burst window that is half the buffer correlated for nothing. Trim the range to what is actually decodable. The postamble fallback ran on every attempt where no frame came out, including the long stretch while the burst is merely still arriving -- there is nothing for the tail anchor to recover there. Run it only when the preamble anchor genuinely had nothing to offer: no preamble at all, or a preamble whose payload WAS resident and still failed. Measured on loopsim, RX drops from 83% of a core to 74% and from 48% to 30% for the two instances. Still too heavy for a 32-bit Pi, which is the target, so this is a step and not the end of it. The remaining cost is dominated by the correlation itself running every 4 symbols over a window sized in seconds. Unit suite green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The harness stopped feeding right after the burst, which is not what a receiver sees. A radio keeps delivering audio when nobody is transmitting, and that silence keeps sliding the RX window past a burst that has already arrived. This matters because the FIFO integration test -- the one path where an MFSK transfer completes -- carries NO idle audio at all: its bridge only moves bytes while someone transmits, so a burst sits in an otherwise empty window and the sliding case is never reached. ALSA, PulseAudio and every real radio do feed continuously. Taking "it passes over FIFO" as "the mode works on the air" was too strong a reading, and this closes that gap in the harness. Sweeps trailing audio from 0.32 s to 16 s after the burst. All decode, so the sliding window is NOT where the live failure comes from either -- which is useful: it rules the decoder out again, from the other side. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…e live mode The RX backlog cap was computed each iteration from payload_mode, which moves during a session, so the guard moved with it -- observed firing at "~16 s" and then "~7 s" twenty seconds apart. When it briefly took a fast mode's value, a 13.5 s MFSK burst that was still arriving looked like a backlog and was flushed mid-burst. Derive it from arq_protocol_longest_burst_s() instead: the slowest rung the session can select. A guard can no longer shrink below the thing it protects. The helper is shared with arq_protocol_accept_rx_window_ms(), which needed the same answer, and a test pins it against every rung of the ladder. This is NOT the cause of the MFSK stall, and the same run proves it -- with the cap corrected the flushes continue, because the backlog is real: t=10.4s consumed=8045 samp/s wait=87.9ms ring=0 B (control modes) t=16.7s consumed=3314 samp/s wait=97.1ms ring=1.7 kB (MFSK bound) t=26.9s consumed=3745 samp/s wait=0.0ms ring=160 kB t=47.3s consumed=3819 samp/s wait=0.0ms ring=495 kB (~15 s backed up) The MFSK receiver runs at about half real time (~4100 of 8000 samp/s) and the read stops blocking altogether, so the ring grows without bound until the cap trips and destroys the burst. An earlier reading that the RX "kept up perfectly, drops=0" was taken at idle and did not survive load. The fix is decode cost, not guards. Per search this backend downmixes and FIR-filters a multi-second window and correlates across it; modem73's MFSK (phy/mfsk_modem.hh) uses per-tone Goertzel energy detection, which is O(N) per tone and streaming. That is the next change, and it is the same wall that blocks 32-bit ARM. Also adds the two-burst window-wrap test, which passes -- ruling the sliding window out from the other side. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…rrive expect_pattern_ack was true for the whole of ACCEPTING and CONNECTED, and modem.c uses it to gate a correlation over a ~3-burst window run on EVERY capture chunk. So a detector for a signal that can only appear in one state ran continuously for the entire session, whatever the payload mode. It cost about half the receive budget. Measured on the PulseAudio bench with the payload mode pinned AWAY from MFSK, so the MFSK waveform was not involved at all: branch: consumed 3457-3587 samp/s ring grew to 396,960 B 22/512 bytes trunk: consumed 7977-7992 samp/s ring 480-840 B 512/512 in 39.1 s Trunk has no pattern ACK, which is the whole difference. The receiver could not keep up, the capture ring grew without bound, and the backlog guard then flushed bursts that were still arriving -- the "decodes one frame then stalls forever" behaviour seen on every continuously-fed transport. ARQ_DFLOW_WAIT_ACK is by definition the only in-session state where the peer's pattern ACK is due, and ACCEPTING is the handshake equivalent. Gate on those. Everywhere else -- above all IDLE_IRS, where the receiver is demodulating a 13.5 s burst and needs the CPU -- there is nothing to detect. With this, the full ladder completes a transfer for the first time: 512/512 bytes in 47.2 s (87 bps) match=True consumed 6372-8026 samp/s, ring 200-1280 B, 0 backlog flushes This also retires my earlier conclusion that the remaining blocker was MFSK decode cost. It was not: it was a gate that was too wide. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

mfsk_sync_search ended with a magic constant: return (best_fine_metric < 0.5) ? -1 : best_fine; The per-symbol statistic is |corr|^2/(E_tmpl*E_rx) = SNR_sym/(1+SNR_sym), so a 0.5 gate demands SNR_sym >= 0 dB. That single number, not the LDPC, was the weak-signal limit of the whole MFSK mode: everything below it was discarded before the demodulator ever ran. The proof is that the code does not matter. Sweeping all five ported rates -- 8/16, 5/16, 3/16, 2/16, 1/16 -- produced BYTE-IDENTICAL FER curves (2/20 and 17/20 at the same SNRs). A 16x stronger code changing nothing says the limiter is upstream of it. Instrumenting the search confirmed it directly: at SNR3k -3.1 dB it returns off=-1 with metric 0.344, and no payload is attempted. Measured on AWGN over the real two-burst (~104k sample) window: noise-only MAX metric 0.037 - 0.049 (flat vs SNR -- it is normalised) signal @ SNR3k -5.1 dB 0.219 signal @ SNR3k -9.1 dB 0.091 The signal stood ~10x above the noise floor while the gate sat above the signal. 0.08 is ~1.6x the measured noise maximum and gave 0 false syncs in 600 noise-only searches. Results (20-30 trials/point, same harness): AWGN 50% FER -0.6 dB -> -11.2 dB SNR3k (~10.6 dB) Watterson 2ms/1Hz +5.0 dB -> -3.7 dB SNR3k (~8.7 dB) and the 0.25-0.30 Watterson error floor is gone -- it was sync failing during fades, not the decoder. DATAC15 measures -10.3 dB on the same harness, so the ladder floor finally sits BELOW the rung above it, which is the entire point of having a fringe rung. A lower threshold also needs false anchors to be survivable, and they were not: modem_mfsk caches the located anchor and marks it tried, so a false peak whose payload failed CRC left the decoder blind until it slid out of the window -- a missed burst, not merely wasted CPU. A resident payload that fails CRC now releases the anchor and the next search restarts one symbol past it, which cannot skip anything decodable because the search returns the EARLIEST peak above threshold. Both tests were verified to fail without their fix: the weak-preamble test fails when rebuilt with -DMFSK_SYNC_ACCEPT=0.5, and the failed-anchor test fails when the anchor release is removed. Neither burst in that test carries a postamble -- with one, the postamble fallback masks the defect entirely. Below about -12 dB the signal metric reaches the detector's own noise floor, so no threshold can go further; that needs a detector which COMBINES the preamble symbols rather than averaging per-symbol normalised ratios, since averaging gains nothing from preamble length. Rate 1/8 buys only ~0.5 dB there, so the fringe rung stays at rate 1/2 (all five codes share N=1600: rate costs payload, not airtime). Also measured and deliberately NOT shipped: rewriting the LLR derivation to codec2's non-coherent model (log-I0 of amplitude + max* combining, replacing an energy difference with a +/-5 clamp) moved the cliff by ~0 dB at both the old and the new threshold -- byte-identical FER over 30 trials/point -- while costing ~6% more decode time, which this mode cannot spare on 32-bit ARM. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

An LDPC block code corrects scattered errors far better than a contiguous run, but a fading HF channel delivers exactly the contiguous case: an MFSK burst is 13.5 s long, so a deep fade wipes a stretch of consecutive symbols and -- with the coded bits laid down in order -- a stretch of consecutive codeword bits with it. The decoder was being handed the one error pattern it is worst at. Transmitted slot t now carries coded bit ilv[t], from a Fisher-Yates permutation driven by a fixed xorshift seed: integer arithmetic only, so both ends derive an identical table on any platform with nothing sent over the air. Pseudo-random rather than a fixed stride because these are quasi-cyclic codes (N=1600, circulant 100) and an arithmetic progression risks landing the damage on the same check nodes. Measured, 60 trials/point, M=32 rate 1/2, gate 0.08 (builds verified to differ only in the interleaver): channel SNR3k without with Watterson 2ms/1Hz -3.1 dB 0.35 0.20 Watterson 2ms/1Hz -0.1 dB 0.08 0.07 Watterson 2ms/0.2Hz -4.4 dB 0.58 0.47 Watterson 2ms/0.2Hz -1.6 dB 0.25 0.18 AWGN all 0.00 0.00 Better or equal at every point and never worse. At the 1 Hz fringe that is 21 failures out of 60 down to 12: expected transmissions per frame 1.54 -> 1.25, about 19% more ARQ throughput where it is needed, for no airtime and a table lookup per bit. AWGN is unchanged, as it should be -- interleaving buys nothing against white noise. The gain is real but smaller than one might expect, for a reason worth writing down: at 1 Hz Doppler the fade coherence time is ~0.5 s, so a 13.5 s burst already spans ~26 independent fades and carries intrinsic time diversity. Interleaving helps most where a single fade covers a large fraction of the burst, which is why the 0.2 Hz case gains at least as much. Two defects found while wiring this up: - h->llr was allocated as code->N floats but mfsk_demod writes one LLR per transmitted bit slot (NPAY*bps). For M=32 those are both 1600 so nothing overflowed, but bps need not divide N -- M=8 gives 1602 slots and a two-float overrun. Buffers are now sized by nb. - mfsk_be_close never freed h->bb: rxcap complex doubles, ~3.4 MB leaked on every open/close cycle (and the mode pool opens/closes per session). This changes the on-air bit ordering, so both ends must run it. Pre-2.0 has no compatibility constraint and MFSK has not shipped in a release. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

A mistuned radio shifts every tone. Beyond half a subcarrier -- 15.6 Hz here -- the nominal FFT bins see nothing and the mode goes stone deaf, and no amount of searching in time recovers it. Measured on the unmodified decoder, clean channel: dial offset 12 Hz (0.38 bins) -> 10/10 decoded dial offset 16 Hz (0.51 bins) -> 0/10 decoded 16 Hz. That is well inside what two HF radios drift apart by, or what an operator misses tuning by ear. Every sensitivity number this project has measured came from a perfectly tuned simulator, so the entire test suite was blind to it -- on the air it would have presented as "the robust mode just doesn't work", with nothing in the logs to say why. Found by reading modem73, which searches frequency for exactly this reason. Acquisition now correlates against preamble templates pre-rotated to each of 13 hypotheses, spanning +/-3 subcarriers (+/-94 Hz) in HALF-subcarrier steps. Result, clean channel, 10 trials per point: offset 0 .. 102 Hz -> 10/10 at every point tested offset 110 Hz and out -> 0/10 (past the +/-3 bin design range) Usable tuning tolerance goes from about +/-12 Hz to about +/-100 Hz. Two things had to be right, and neither was obvious: - HALF-bin steps, not whole. A whole-bin grid leaves the worst case sitting exactly between two hypotheses, where the tone energy splits across two bins and neither template captures it. Measured with a whole-bin grid: 31/62/94 Hz recovered to 10/10, but 16/47/110 Hz stayed at 0/10. - Take the BEST hypothesis, not the first one past the gate. At half-bin spacing a neighbouring hypothesis is only 0.5 bin off and still clears the threshold, so first-past-the-post latches the wrong offset; acquisition then "succeeds" and every frame fails CRC. Whole-bin spacing had hidden this, because neighbours were a full bin away and fell below the gate. This cost a full measurement cycle to find: 94 Hz went from 10/10 to 0/10 when the grid got finer, which is the opposite of what a finer grid should do. Cost is contained by latching. A CRC pass confirms the offset and later searches try that one hypothesis alone, so a correctly tuned link does exactly one correlation as before; a resident payload that fails CRC drops the lock so a wrong latch cannot wedge the decoder. Measured: +21% on the decode path with a signal present, no change on idle noise. No regression anywhere it matters. At zero offset the AWGN cliff is unchanged point for point (20/20, 18/20, 11/20, 0/20 across -9.1 .. -12.1 dB SNR3k, identical before and after), and false alarms stay at 0 over 200 noise-only trials despite 13x the hypotheses. The test drives a real single-sideband shift through a Hilbert transform, not a cosine multiply, because multiplying by a cosine produces both sidebands and is not what a mistuned radio does. It covers 0, 16, 47 and 94 Hz; 47 Hz is picked as 1.5 subcarriers, which defeats both the original decoder and a whole-bin grid. Verified to fail without the fix (0 Hz decodes, 16 Hz LOST). Full C suite green, go test -count=1 green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…le rates The fading tap is a 2nd-order Butterworth driven by white noise. As fc/fs shrinks its poles crowd the unit circle: at 48 kHz with 1 Hz Doppler the ratio is 2e-5 and the poles sit within ~1e-4 of the circle, where float's ~6e-8 of relative resolution is enough to place one OUTSIDE it. The filter then diverges instead of fading, and the "measured signal power" is the blow-up: fs= 8000 2 paths 1 Hz -> SNR3k -8.37 dB sane fs=48000 1 path 0 Hz -> SNR3k -8.24 dB sane (no Doppler filter) fs=48000 2 paths 1 Hz -> SNR3k +65.76 dB diverged Coefficients and filter state are now double. After the change the same case reads -6.43 dB, and 8 kHz is unmoved (-8.37 -> -8.32, inside the scatter of a 2 s window of slow fading). Mercury runs at 8 kHz, which is why this sat unnoticed: 8 kHz is 6x further from the cliff and stayed marginally stable. It surfaced only when driving a 48 kHz modem through the same channel, and it matters now because wider bandwidth means higher sample rates -- this model is the instrument behind every sensitivity number in the project, so it has to hold up above 8 kHz before any of those numbers can be trusted there. No behaviour change at 8 kHz: the MFSK AWGN cliff is unchanged (11/20 at -11.1 dB, 19/20 at -10.1 dB) and the 1 Hz fading point moved 0.20 -> 0.25 FER over 60 trials, which is 3 frames and a necessarily different RNG sequence. The test pins two properties and was verified to fail on the old code: - fading stays finite and in range over 8/16/24/48/96 kHz x 0.1..2 Hz. The band is wide on purpose; it is there to catch divergence, not pin a value. - a SINGLE static path is rate-invariant to 0.5 dB (measured: 0.04 dB). That is the property that lets an 8 kHz result be compared with a 48 kHz one. Deliberately not two paths -- a delayed copy adds a multipath interference term that really does vary with rate (~1.5 dB), which would turn this into a test of interference rather than of the model's rate handling. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The rebase put 33 commits of MFSK/ARQ work on top of 30 commits of trunk, and the two lines had both grown a LISTEN OFF implementation. Trunk's is the better one and this restores it on top of the rewritten FSM. What trunk has that we did not: LISTEN OFF arriving within ARQ_LISTEN_OFF_GRACE_MS of entering ACCEPTING is DEFERRED rather than acted on. A scanning host (BPQ32 interlock) sends LISTEN OFF at dwell expiry and needs a moment to process the PENDING we just sent it and cancel its own timer; acting immediately turns that race into a dropped inbound call. CALLING deliberately has no grace — PENDING announces an INCOMING call, so it is never sent while we are the caller and there is nothing to race against. Most of the machinery survived the rebase intact (the clear-on-idle rule in sess_enter, the honour-on-CONNECTED path, the TIMER_RETRY expiry). Only the site that SETS the flag had been replaced by our unconditional version, so that is what this restores. tests/datalink_arq/test_arq_fsm.c was rebuilt rather than merged. The conflict resolution during the rebase had union-merged it, which is safe for a list of Makefile targets and decidedly not for C: it spliced trunk's test bodies into our helper functions, leaving goto_connected_irs() ending in a mock_set_uptime_ms(4000) and a LISTEN-OFF comment. Rebuilt from trunk's file plus a 3-way diff of our additions: - carried over our 9 additions (make_data_event, complete_ack_tx, goto_connected_irs, 3 wait_ack tests, 3 irs_mirror tests) - kept trunk's 6 LISTEN OFF tests, including the two grace tests that this commit's FSM change makes pass - dropped 3 test_wait_ack_* tests that trunk still has and we deliberately deleted in the data-plane rewrite: they assert cumulative-window and turn-req semantics that no longer exist - test_init_mode_defaults now expects the MFSK ladder floor, not DATAC15 - test_disconnect_drain_timeout_forces_teardown used ARQ_EV_TIMER_KEEPALIVE purely as "any CONNECTED event"; keepalive was removed by the rewrite, so it now uses TIMER_RETRY. The test itself is worth keeping. 35 FSM tests pass, full C suite green, go test -count=1 green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

… kHz fix Writes down what was measured rather than leaving it in commit messages and a PR thread. Three things worth having in the tree: MFSK-PORT.md — the fringe floor was the sync accept threshold (0.5, demanding SNR_sym >= 0 dB), not the LDPC: all five code rates gave byte-identical FER. Lowering it to a measured value took AWGN 50% FER from -0.6 to -11.2 dB and removed the Watterson error floor. Plus interleaving, the frequency search (no dial tolerance at all before — 16 Hz was enough to go deaf, and every sensitivity number in this file had been measured on a perfectly tuned simulator), and the things measured and deliberately NOT shipped: the codec2 LLR model, Goertzel, shorter frames, lower rates. Each with the number that killed it, so nobody re-runs them. Also the modem73 RFDM cross-check, driven through our own channel and metric: per bit we win on AWGN by 2.7 dB and tie under fading. That retires the hypothesis that coherent pilot-aided modulation is what closes our gap to capacity — it does not, and the gap estimate had ignored their 25% pilot and 33% cyclic-prefix overhead. watterson_model.md — the Doppler filter needs double precision above 8 kHz (48 kHz/1 Hz read +65.76 dB, diverging), and the known wart that two-path measurements carry a ~1.56 dB rate-dependent offset while a single static path is rate-invariant to 0.04 dB. That distinction is what makes cross-rate comparisons legitimate, so it is stated rather than left implicit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Nothing in the suite measured how long a connect takes. It only asserted that one eventually occurred, which cannot tell you whether a handshake change made things better or worse — and the handshake is three DATAC16 bursts, so most of the time is spent modulating rather than deciding. connect_bench reports, in virtual time so the numbers are deterministic: when the CALLER sees CONNECTED (it needs 2 frames), when the CALLEE does (it needs the 3rd), and how many transmissions each side made. Swept over erasure probability with several seeds per point, so a change can be judged on expected time-to-connect rather than on one lucky run. Timestamps come from the FSM's own notify_connected/notify_pending callbacks via the existing active-endpoint hook; frames are counted where outframes are drained, so patterns and coded frames both land. Baseline on today's code: PER connects caller(ms) callee(ms) frames 0.00 8/8 8380 13120 4.0 0.10 8/8 8380 21310 4.6 0.20 8/8 15840 26040 5.5 0.30 7/8 19191 30171 6.0 0.50 2/8 12380 28040 5.5 Two things worth noting. The clean-channel figures land within ~250 ms of what the frame durations and guards predict analytically (8.2 s / 12.8 s), which cross-validates the instrument against the arithmetic. And a clean connect takes FOUR transmissions where the handshake defines three — that extra frame is a spurious CALL retransmission, addressed next. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

fsm_calling had no ARQ_EV_TX_COMPLETE handler, so the 8 s retry deadline was set when the CALL was QUEUED. A CALL spends ~3.8 s modulating (DATAC16), and the peer cannot even start its ACCEPT until our PTT drops — its own ACCEPT then takes another ~3.8 s. The retransmission therefore fired at t~8.0 s while the ACCEPT was still arriving at t~8.1 s: a collision on essentially every connect, costing a fourth transmission on a channel that had lost nothing. fsm_accepting already anchors ACCEPT exactly this way (its TX_COMPLETE resets the RX window from PTT-OFF); CALLING never got the same treatment. Measured with tests/sim/connect_bench, 24 seeds per point: PER connects frames censored time 0.00 24/24 -> 24/24 4.0 -> 3.0 13.1 s -> 13.1 s 0.10 24/24 -> 24/24 4.6 -> 3.9 19.5 s -> 22.2 s 0.20 23/24 -> 24/24 5.4 -> 4.6 32.3 s -> 28.7 s 0.30 21/24 -> 22/24 5.9 -> 5.3 49.4 s -> 48.2 s 0.50 10/24 -> 15/24 6.5 -> 6.9 121.8 s -> 101.5 s A clean connect now costs three transmissions for a three-frame handshake, which is the point. Reliability improves where it matters most: at PER 0.5 the success rate goes 10/24 to 15/24 and censored time drops 20 s. Reported honestly: PER 0.10 censored time is ~2.7 s worse. The buggy early retransmission was accidentally acting as a fast retry, so a genuinely lost CALL now waits the full interval. I checked whether the interval should shrink to compensate and it should not — at 6.5 s and 5.5 s the success rate FALLS (12/12 -> 11/12 at PER 0.2, and worse at 0.5), because the four retry slots are consumed sooner. 8 s stays. The censored column counts a failure as the 180 s cap. Means over successes alone are survivor-biased: when a change makes previously-hopeless connects succeed, those slow successes inflate the success-only mean while the link has actually improved. An earlier 12-seed run showed a PER 0.20 "regression" that reversed at 24 seeds — small-sample means on this metric are not trustworthy. Test verified to fail without the fix. Full C suite green (36 FSM tests). Integration: one failure observed when the suite was started immediately after a full parallel rebuild, then four consecutive passes (three at 203.6 s identical); treated as load-related flake, not reproducible. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…signs Three connect-path cleanups, no wire change. 1. expect_pattern_ack no longer includes ACCEPTING. No pattern can arrive in that state: the caller answers an ACCEPT with either a coded DATAC16 confirm or a first MFSK DATA burst, never a pattern — send_ack() emits one only once the session is CONNECTED. So the correlator ran for the whole ~18 s ACCEPT window, and per the measurement already recorded at that site it costs about half the RX sample budget (3.5k samp/s consumed against 8k arriving, capture ring growing to ~400 kB). It was burning that precisely while the answerer had to decode the frames that complete the connect. Note this is NOT visible in connect_bench: the sim has no DSP, so it cannot show RX budget. The justification is that the branch is unreachable plus the existing DIAG measurement of what the correlator costs; confirming the gain needs the live path (make DEBUG_IO=1). 2. Callsign encoders refuse over-long input instead of truncating. Truncating an arithmetic code does not shorten the string — it decodes to a DIFFERENT one, so the peer answers a call from a station that does not exist, or drops one that does. Both encode_callsign_payload (CALL/ACCEPT) and encode_callsign_only_payload (CQ) had this. They now return -1, which build_call_accept already propagates, and send_call_accept says why rather than leaving a session retrying against silence. Tests: a round-trip over realistic callsigns including SSIDs (PU2UIT-2, DL9ABC-15) and a refusal test verified to fail against the old truncating code. 3. Dead code: startup_deadline_ms was written at three sites and read nowhere; ARQ_CONNECT_GRACE_SLOTS and ARQ_CONNECT_BUSY_EXT_S were never referenced. Left alone deliberately, because it is user-facing and the call is not mine: `startup_max_s` is a documented mercury.ini knob, clamped 2..60 and written back by cfg_write, whose ONLY effect was to set the unread field above. It is now inert — an operator can tune it and nothing happens. It should either be implemented or removed. Full C suite green, go test -count=1 green (203 s). The arq.c:754 format-truncation warning is pre-existing (present on the unmodified tree). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…tion startup_max_s, keepalive_interval_s and keepalive_miss_limit were parsed from mercury.ini, range-clamped, logged at startup and plumbed through arq_set_*() setters into arq.c -- and then read by nothing. Zero references in arq_fsm.c after the delivery-driven FSM rewrite. A knob that looks live but does nothing is worse than no knob: an operator raising keepalive_miss_limit on a flaky link gets a startup log line confirming the new value and no change in behaviour. tnc_keepalive_s is a different, genuinely live host-facing knob; it stays. docs/ARQ.md described MODE_REQ/MODE_ACK, the MODE_REQ_WAIT/MODE_ACK_TX states, EV_RX_MODE_ACK, the ARQ_BACKLOG_MIN_* upgrade thresholds and keepalive -- none of which exist any more. Rather than half-patch prose for a protocol that is still moving, the sections are removed and replaced with a RE-DO AFTER FSM REWRITE marker naming what went and why, so the gap is explicit instead of silently wrong. The subtype table is trimmed to the five subtypes actually on the wire. Gate: unit suite green, integration go test -count=1 green (202 s), mercury starts clean (the config struct shrank -- rebuild fully, stale objects segfault). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The third leg of the handshake existed to tell the answerer one thing -- "I heard your ACCEPT" -- and spent a full DATAC16 frame saying it. The pattern ACK machinery already carries exactly that signal for the data plane; only the connect path still took the coded branch. Measured on connect_bench, 24 seeds per point, bare connect (no payload queued, which is when this leg is sent at all): PER callee(ms) before -> after connects 0.00 13120 -> 10020 24/24 -> 24/24 0.10 22220 -> 19120 24/24 -> 24/24 0.20 28727 -> 25627 24/24 -> 24/24 0.30 36251 -> 33151 22/24 -> 22/24 0.50 54325 -> 51225 15/24 -> 15/24 -3.1 s at every point, connect success rate unchanged, caller-visible time unchanged. The sim erases patterns at the same rate as coded frames, so it does not model the pattern's robustness advantage -- on a real link the fringe should do better than this table, not worse, since a full-energy Welch-Costas correlation survives well below where a coded DATAC16 frame decodes. The cost is that the answerer must now run the pattern correlator during ACCEPTING, which the previous commit had just removed for good reason: it consumes a measured 3.5k samp/s against 8k arriving, and running it for the whole ~18 s ACCEPT window starves the decoders that must handle the caller's first data burst. So it is not simply switched back on. The confirm's arrival time is predictable -- one ISS post-ACK guard after our ACCEPT leaves the air -- so the FSM opens a bounded 4 s window at ACCEPT PTT-OFF and closes it on any state change, including the successful one into CONNECTED. Two things guard that window, because a failure here is silent: a missed confirm falls through to connecting on the caller's first DATA frame, which is correct, completes the transfer, and quietly hands back the 3.1 s. - test_connect_confirm_listen_window_is_bounded pins both halves (opens at PTT-OFF, closes on state change); each half was verified to fail alone. - TestMercuryARQTransfer now asserts the answerer's log says the handshake completed on the confirm. This is the one that matters: the sim delivers patterns without consulting expect_pattern_ack, so only a live-modem run proves the correlator is actually open when the burst lands. Verified to fail with the window forced shut -- and, as predicted, the transfer still passed everything else in that run. Also: tests/Makefile clean now removes connect_bench. It did not, so a clean+build cycle served a stale binary and the first A/B of this change read byte-identical before and after a 3.1 s move. Gate: unit suite green, integration go test -count=1 green (194 s). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Fourth attempt at shortening the ARQ control plane, and the first one that leaves behind a tool instead of a branch. The candidate: the existing 14-byte control frame plus CRC16, carried over our own 32-MFSK waveform with a short vendored LDPC code (H_128_256_5, K=128 N=256) -- 52 payload symbols plus the 8+8 preamble/postamble = 2.72 s on air against DATAC16's 3.74 s. It wins on AWGN and still loses, which is the whole point of writing it down. AWGN 52-sym short frame Eb/N0 5 dB for FER<0.02 Rayleigh 1.0 Hz 52-sym short frame 12 dB Rayleigh 1.0 Hz 410-sym long frame, same rate 1/2 8 dB Rayleigh 0.2 Hz 52-sym short frame 16-18 dB On AWGN that is SNR3k ~ -11.9 dB, 2.6 dB better than DATAC16's -9.3 dB floor and a second shorter -- which is exactly why this idea keeps coming back. Fading takes it away: shortening costs ~4 dB of time diversity at 1 Hz, more than the 2.6 dB it won, and it is worse again at 0.2 Hz, where a 2.08 s burst spans roughly one fade and has no time diversity at all. Slow fading is the measured OTA regime, so the candidate is rejected. Airtime is diversity. On a fading channel a control frame cannot be made both shorter and more robust by re-coding it, and no better code recovers the time-diversity term. DATAC13, DATAC14 and DATAC18 each hit this same wall. The sweep is calibrated, not merely plausible: run against the long-burst case it predicts the MFSK data mode's AWGN floor at -11.9 dB (measured -11.2) and its Watterson 2ms/1Hz floor at -8.9 dB (measured -4.5). So absolute numbers read ~4 dB optimistic -- it omits acquisition and the passband chain -- and the header says so. Relative comparisons, which is what the decision turns on, are sound. Also recorded: DATAC16 is acquisition-limited, not decode-limited. Its floor implies Eb/N0 ~ 9.9 dB where its class of code needs 3-4 dB, and the gap is the preamble detector. That is why DATAC18 died at acquisition, and it is the one lever on this frame that is not yet exhausted. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The previous commit claimed DATAC16 is acquisition-limited and backed it with an energy budget: the mode's floor implies Eb/N0 ~ 9.3 dB where a rate-0.2 code should manage 1-2 dB, so the ~7 dB remainder must be the preamble detector. That argument does not survive its own arithmetic. Subtract cyclic prefix (~1.3 dB), pilot carriers (~1.3 dB), channel-estimation and implementation loss (~1.5 dB) and short-code loss at N=640 (~2.5 dB) and about 1 dB is left, which supports no conclusion at all. The claim also leaned on two citations that say something narrower: docs/MFSK-PORT.md measured delivered==acquired for the MFSK mode, not for DATAC16, and docs/MODES.md attributes DATAC15/16 acquisition losses to the ~85% plateau at HIGH SNR on MPP, which is a different phenomenon from the fringe floor. So measure it. acquire_vs_decode drives the real freedv modems and records two independent outcomes per burst: did it reach FREEDV_RX_SYNC, and did a frame come out with the CRC intact. AWGN only -- under fading a fade during the preamble and a fade during the payload are indistinguishable in the totals. DATAC16, 100 trials: SNR3k acquired delivered decode|sync -8 dB 100/100 99/100 99% -9 dB 99/100 95/100 96% -10 dB 76/100 72/100 95% -11 dB 42/100 36/100 86% -12 dB 13/100 12/100 92% The conclusion holds -- below -9 dB acquisition collapses while frames that sync still decode 86-96% of the time -- but it is now a measurement, and the "~6 dB deficit" framing is gone. The harness calibrates against the independent 100-trial figures already in MODES.md: 95/100 delivered at -9 dB vs 96/100. Two harness bugs worth recording, because both produce confident nonsense: the raw-data API does not generate the CRC (the caller owns the last two bytes, as modem.c does), which read as a mode that syncs perfectly and never delivers; and counting FREEDV_RX_TRIAL_SYNC as acquisition credits false alarms on noise, which inflates the fringe rows in exactly the direction that flatters the acquisition-limited story. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…addresses nobody A Welch-Costas pattern plus session-derived suffix symbols is the one shape that could replace the 3.74 s coded ACCEPT without hitting the energy-per-bit wall, because it is a correlation against an expected sequence rather than a decode. mfsk_set_hail_target() already builds it. So measure what it costs. It does not work as written. The detector scores matched symbols across the whole 20-symbol sequence against a single threshold, and 16 of those 20 are the SHARED pattern -- so any threshold <= 16 is satisfiable without one suffix symbol matching. mfsk_set_hail_target() sets hail_detect_threshold = hail_match_threshold + MFSK_HAIL_SUFFIX_LEN = 12 of 20, squarely inside that regime. Transmitting session A's directed pattern and asking session B's detector: SNR3k detect@12 wrong-session@12 | detect@18 wrong-session@18 -12 dB 96/100 73/100 | 12/100 0/100 -10 dB 100/100 100/100 | 86/100 0/100 -8 dB 100/100 100/100 | 100/100 0/100 At its own threshold the directed hail is accepted by the wrong station essentially always. Latent, not shipping -- mfsk_set_hail_target() is not wired into any live path -- but it must be fixed before it is: the threshold has to exceed ack_pattern_nsymb so that (threshold - ack_pattern_nsymb) suffix symbols are forced to match. With that corrected the idea stands up. Threshold 18/20 forces >=2 of 4 suffix matches, gives 0/100 wrong-session acceptance, and detects to about -10.5 dB -- comparable to DATAC16's -9.5 dB floor, in 800 ms against 3740 ms. Roughly a dB better and ~2.9 s shorter, which is the "faster and more robust" the short coded frame could not deliver. Selectivity is what costs the dB: the plain 16-symbol pattern at threshold 8/16 reaches about -15 dB but addresses nobody, which is exactly why it is fine for the in-session connect confirm that shipped (session already established, correlator window bounded) and not fine for an ACCEPT. Not yet a green light. The residual false-accept rate against a random other session is a binomial estimate (~0.56% at thr=18, ~1 in 178), not a measurement, and it was checked against a single wrong key. Sweep many random session keys before wiring this into the FSM. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…properly Two things, both falling out of measuring the directed-ACCEPT idea. 1. The correlator was doing its FFTs twice. mfsk_pattern_detect() called mfsk_detect_pattern() once for the ack tones and once for the break tones over the same baseband buffer. The expensive part -- a GI removal, an FFT and a depad for every (candidate start, symbol) pair -- depends only on the samples, never on which tones were hoped for, so half of it was waste. mfsk_detect_patterns() does that work once and scores any number of tone lists against the same bins. This is not a micro-optimisation: the correlator was measured consuming 3.5k samp/s against 8k arriving, which is the sole reason arq.c only runs it inside bounded windows (expect_pattern_ack). Halving it widens what those windows can afford -- and a directed ACCEPT needs to score one list per bandwidth token, which at the old cost would have starved the RX outright. Guarded by test_detect_patterns_matches_per_list_calls: scores AND positions must be bit-identical to the per-list calls, over several independent buffers. A scoring shortcut that quietly changed a match count would move the ACK detection threshold with nothing else noticing. 2. Four suffix symbols is the wrong length for a directed pattern. Selectivity comes from how many suffix symbols the threshold FORCES to match, so with only 4 available a 1-in-10000 false-accept demands all 20 symbols match -- 100% of them, punishing at the fringe. Lengthening the suffix is much cheaper than tightening the threshold: suffix on air forced for <0.01% threshold % of symbols 4 800 ms >=4 of 4 20/20 100% 8 960 ms >=4 of 8 20/24 83% 12 1120 ms >=5 of 12 21/28 75% Eight at threshold 20/24 is the recommendation. Measured over a realistic 3-burst search window with the burst at a random offset, 150 trials at -10 dB: 150/150 detected, 0/150 accepted from a random other session, 0/150 false alarms on noise. Against the DATAC16 ACCEPT it would replace, same AWGN and axis: 98% vs 72% at -10 dB, 83% vs 36% at -11, 33% vs 12% at -12 -- about 2 dB more sensitive in 960 ms against 3740 ms. Still not wired into the FSM. Doing that needs MFSK_HAIL_SUFFIX_LEN raised from 4 and the FNV-1a derivation rehashed (one 32-bit hash only yields 6 symbols), plus the fading case, which is where the short coded frame died. Gate: unit suite green, integration go test -count=1 green (194 s). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…ays they lied Adds chanutil, a shared wrapper over the project's own Watterson model so different instruments measure different waveforms on the SAME channel instead of each rolling its own and producing numbers nobody can compare. Both acquire_vs_decode and hail_suffix_sweep now take a preset (awgn|mpg|mpp|mpd); under fading the sweep axis becomes noise density and the achieved SNR3k is measured and reported per row, because with fading the delivered SNR is not something a caller can dial. Getting there required fixing three separate errors, each of which produced a plausible table pointing the wrong way. They are worth recording because none announced itself: 1. CLIPPING. freedv emits DATAC16 at rms ~8000 / peak ~16400. At fringe SNRs the noise rms is a couple of times that, so signal+noise hits the int16 rail: 23% of samples clipped, delivering -7.07 dB when -9.00 was requested, and costing about a dB net once distortion is counted. Bursts are now normalised to a fixed peak before noise is added, and every row prints the clipped percentage so this cannot silently return. 2. TAP WARM-UP. watterson_init() zeroes the Doppler tap IIRs and the taps are generated by filtering internal noise, so they start at zero and ramp. A burst handed to a fresh channel therefore opens inside an artificial deep fade -- exactly where its preamble is. DATAC16 on MPP read 10% delivered at -7.3 dB against the 67% documented, until the taps were warmed first. 3. A 3.01 dB REPORTING OFFSET. watterson_measured_snr3k() takes Psig from the signal it is handed -- the analytic signal, power 2P -- while the pipeline hands the modem Re{}, power P. Taking the real part leaves the noise PSD alone, so only the signal term is doubled: exactly a factor of two, for any real-output path. Corrected at the chanutil boundary rather than in common/watterson.c, whose convention is right for the complex-domain use it was calibrated for; changing it would move every other measurement silently. The gate for all three is CHAN_AWGN_C, a calibration preset that runs AWGN through the complex pipeline with no fading. It must agree with the harness's own direct real-domain AWGN, which hits its requested SNR to 0.01 dB. Before these fixes the two read 3.3 dB apart on the same mode; they now agree within trial noise. Consequence: the acquire/decode table committed in 198902c was measured while clipping and its SNR labels were ~1 dB pessimistic. docs/MODES.md now carries the corrected 100-trial run, which reproduces the independent AWGN figures already in that document (98 vs 96 delivered at -9 dB, 78 vs 82 at -10, 48 vs 52 at -11). The conclusion is unchanged and now better supported -- acquisition falls 99 -> 52 -> 7 between -9 and -13 dB while frames that sync still decode 92-99% down to -11 dB -- but the decoder does join in below that, so the honest claim is that acquisition is the first-order limiter, not the only one. The directed-pattern ACCEPT still has no fading verdict; that is what this plumbing was built to answer and it is the next run. Gate: utils build, main build, unit suite green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Settles the gate this plumbing was built for. On AWGN a 960 ms directed pattern beats the 3.74 s DATAC16 ACCEPT by ~2 dB, which is why it looked like the answer. On the project's own Watterson MPP channel it loses by ~3 dB, both sides measured on the same instrument family with 0% clipping: SNR3k DATAC16 (3740 ms) directed pattern (960 ms, thr 20/24) -7 dB 80% 56% -9 dB 50% 28% -11 dB 20% 8% The DATAC16 column reproduces the independent MPP table in docs/MODES.md (50% at -9 dB against the documented 50), which is what licenses the comparison. This generalises the short-coded-frame result rather than repeating it. A correlation detector pays no energy-per-bit price whatsoever, and it hits the same wall just as hard: 960 ms spans about one fade at 1 Hz where 3.74 s spans four. Airtime is diversity, and that is a statement about time, not about coding. Threshold 17/24 does beat DATAC16 (88% vs 80% at -7 dB) but forces only one of eight suffix symbols to match, so ~1 in 5 random sessions is accepted. The pattern is only competitive when the addressing that makes it safe is given away. Which is also why the SHIPPED 0.64 s connect confirm is fine and an ACCEPT is not: the confirm needs no addressing (session already established, bounded correlator window), so it runs at 8/16 and keeps its ~10 dB margin. Verdict: the ACCEPT stays DATAC16. Connect optimisation stops at the CALL-retry anchor plus the pattern confirm, 12.8 s -> 9.7 s bilateral. Harness changes needed to get a trustworthy answer: - chanutil grew open/run/close. Re-initialising per burst re-ran the tap warm-up every time -- ~47 minutes of warm-up across a 50x7 sweep, which is why the first two attempts produced no output at all. One warmed channel per SNR point is also the more faithful model: a real link is one continuous fading process, not an independent one per transmission. - Fading level management is inverted. At fringe SNRs the noise alone overflows int16, so a fixed signal level clipped 60-80% of samples. The noise is now held where it fits and the SIGNAL is scaled to set the SNR; the channel's measured SNR is still what gets reported. Clipping is now 0.00% across the useful range and every row prints it. - hail_suffix_sweep takes a search-window width. Three bursts mirrors the RX loop and is what the false-alarm column needs, but costs ~9x the correlation work; a fading run may use one, and the header then says the false-alarm column is not comparable. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

rafael2k mentioned this pull request Aug 5, 2026

arq: MFSK weak-signal backend + delivery-driven stop-and-wait rethink #125

Closed

rafael2k and others added 25 commits August 7, 2026 21:29

rafael2k and others added 9 commits August 7, 2026 21:32

rafael2k force-pushed the mfsk-margin branch from b16931c to 44041c7 Compare August 7, 2026 20:45

rafael2k and others added 3 commits August 7, 2026 21:46

rafael2k force-pushed the mfsk-margin branch from b0e47f8 to f8effc6 Compare August 7, 2026 21:37

rafael2k force-pushed the mfsk-margin branch from f8effc6 to cfaa935 Compare August 7, 2026 21:38

rafael2k and others added 8 commits August 7, 2026 22:55

Navigation Menu

Uh oh!

Uh oh!

arq+mfsk: make the MFSK robust rung actually carry a transfer - #160

arq+mfsk: make the MFSK robust rung actually carry a transfer#160
rafael2k wants to merge 46 commits into
mercuryv2from
mfsk-margin

rafael2k commented Aug 5, 2026

Uh oh!

rafael2k commented Aug 6, 2026

Uh oh!

rafael2k commented Aug 6, 2026

Uh oh!

rafael2k commented Aug 6, 2026

Uh oh!

rafael2k commented Aug 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

rafael2k commented Aug 5, 2026

It works

Seven bugs, each with a test that fails without the fix

Test additions

Not done yet

Uh oh!

rafael2k commented Aug 6, 2026

The fringe floor was the sync accept threshold, not the code

The proof: the code doesn't matter

What the detector actually sees

Results

The hazard that had to be fixed with it

Tests

What I measured and deliberately did not ship

Where the next dB are

Uh oh!

rafael2k commented Aug 6, 2026

Second pass: the detector is not the bottleneck — measured, so I stopped there

The oracle settles it

The design curve — tone count beats code rate

…but fading inverts the ranking, so I am not proposing the change

Where the next real gain is

Uh oh!

rafael2k commented Aug 6, 2026

Time-interleaving implemented — 4579a69

Measured — 60 trials/point, M=32 rate 1/2, builds verified to differ only in the interleaver

What it did not do

Two defects found while wiring it up

Uh oh!

rafael2k commented Aug 7, 2026

Frequency search — ba1dce9. Plus two ideas measured and rejected.

1. Frequency search — landed, and it fixes a real defect

2. Goertzel detection — not worth doing

3. Smaller frames — rejected, it would make fading worse

Where this leaves the rung

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Time-interleaving implemented — `4579a69`

Frequency search — `ba1dce9`. Plus two ideas measured and rejected.