paper: final-submission pass — Table 4 corrected against the artifact, missing ablation landed, weak-baseline literature cited

.gitignore

@@ -32,6 +32,8 @@ results/p_repair/*
 !results/p_repair/linear_K16_e2e.json
 !results/p_repair/linear_K16_trained.json
 !results/p_repair/third_family_feasibility.json
+!results/p_repair/nonlinear_opmask_ablation.json
+!results/p_repair/linear_opmask_ablation.json
 checkpoints/
 dist/
 

paper/PROVENANCE.md

@@ -23,6 +23,7 @@ Commits are the short SHA at time of run. Re-record if a number is regenerated.
 | R11 | **A8.1 learned hybrid, 4 families** | **0.550 / 0.683 / 0.683 / 0.000** | same as R9 | same | d65e3db |
 | R11b | A8.1 significance (hybrid > langevin, 2-prop z) | p = 0.003 / <1e-4 / <1e-4 / 0.92 (sig on 3/4; CircleLine fails) | `python scripts/plot_hard_eval.py` (post-hoc from R9-R11 counts) | — | d65e3db |
 | R11c | **A8.1 random-restart CONTROL (4 families)** | **0.550 / 0.717 / 0.683 / 0.200** (ties/beats learned — learned has NO advantage over random multi-start here) | random_count in `scripts/run_hard_eval.py` (control added) | test seed0 100000 | 6d70762 |
+| R11c-note | **Drift corrected (this pass)** | R11c recorded the random-restart arm as 0.550/**0.717**/0.683/0.200 from the original run. The committed artifact `results/p_hard/hard_eval.json` reads 0.550/**0.683**/0.683/0.200 (41/60) and adds an `lm` arm at 60/60 on all four families. The paper's Appendix Table 4 and the main-text sentence now follow the committed artifact, so BilinearProduct is a **tie**, not a random win, and the hybrid battery is "no learned win on any of four" | `python3 scripts/verify_paper_numbers.py` now covers all four random-restart cells | — | this pass |
 | R12 | CoT baseline (Gemini flash-lite), N=25, k=1 | in-dist 1.000, held-out 1.000 | `GEMINI_API_KEY=… COT_N=25 python -m marc.eval.baselines.cot_baseline` | deterministic problems | 28f9b3b |
 | R13 | Cross-family (leave-one-out) learned(cross), 4 held-out | 0.683 / 0.683 / 0.000 / 0.000 (Prod/Quad/Sys/Circle); p<1e-4 on 2/4 | `python scripts/run_crossfamily_eval.py` (best-of-8, 60/family) | train seed0 0, test seed0 100000 | df60ebe |
 | R14 | MATH coverage (parser), MATH-500 sample | 0/48 = 0.000 | `python scripts/run_math_coverage.py` | fixed sample | 552fdcd |
@@ -38,7 +39,7 @@ Commits are the short SHA at time of run. Re-record if a number is regenerated.
 | R23 | v0.3 end-to-end repair + solver | nonlinear **0.933 = oracle = enumeration** vs 0.200/0.250 (N=60, 1 call vs 2.62); linear **0.300** vs 0.217/0.227, oracle=enum=1.000 (N=300, 1 call vs 2.54) | `scripts/run_repair_ranker.py --eval-only --solve-e2e ...`; exact solver linear, REFERENCE_SOLVER (scipy LM k=4) nonlinear; common restart seeds | exact commands/config in `linear_e2e.json`, `nonlinear_e2e_matched.json`; **Data Version 8** | after e81452c |
 | R24 | v0.3 cross-budget K scaling | K=4/8/16 full 0.300/0.187/0.113; random 0.227/0.120/0.107; enum calls 2.54/4.40/9.05; wall speedup 1.26x/2.44x/4.65x — accuracy advantage gone at K=16 (cost win only); direct K=16 training at chance | checkpoint-only `--solve-e2e --K {4,8,16}`; `linear_e2e.json`, `linear_K8_e2e.json`, `linear_K16_e2e.json`, `linear_K16_trained.json` | shared data seed protocol; N=300/150/150; **Data Version 8** | after e81452c |
 | R25 | **Geometry point-chain LEARNED arm** (the law's live prediction, tested) | learned **ties random at every n** and collapses with it: 0.625/0.175/0.025/0.000 (= random) at n=2/4/6/8, **0/4 significant wins (p=0.50)**; **3 seeds (N=120/k) confirm** (0.625/0.183/0.017/0.008 vs random 0.625/0.200/0.033/0.008) — reachability collapse (slope −0.77) is necessary but NOT sufficient; geometry is coupled, no per-var marginal to amortize | `PYTHONPATH=. python3 scripts/run_pointchain_learned.py --trials 40 --K 8 --epochs 200 --ntrain 200` | per-k inline x0 train (R5 methodology), test seed0 90000+k; `results/p_geometry/pointchain_learned.json` | sparsh/geometry-learned |
-| R26 | **External validity: named real systems** (synthetic-only critique) | classical **LM solves 8/8**; gradient-polish random restart 4/8; Langevin 1/8; deterministic 0/8. Systems where random fails (Rosenbrock/Himmelblau/trilateration/3R-IK) are polish-limited (LM fixes them), not a basin collapse — no learning-favorable regime (low-dim + coupled) | `PYTHONPATH=. python3 scripts/run_real_systems.py --K 8 --trials 200` | numeric acceptance max\|r\|<1e-6; 8 named systems (robotics/positioning/optimization/geometry/algebra) | sparsh/real-systems |
+| R26 | **External validity: named real systems** (synthetic-only critique) | classical **LM solves 8/8**; gradient-polish random restart 4/8; Langevin 1/8; deterministic 0/8. Systems where random fails (Rosenbrock/Himmelblau/trilateration/3R-IK) are polish-limited (LM fixes them), not a basin collapse — no learning-favorable regime (low-dim + coupled) | `PYTHONPATH=. python3 scripts/run_real_systems.py --K 8 --trials 200` | numeric acceptance max\|r\|<1e-6; 8 named systems (robotics/positioning/optimization/geometry/algebra); per-system single-start q now cited in the paper's Table 2 from `rows[].q_single_start.rate` (1.00 circle/conic, 0.98 2R-IK, 0.38 cyclic-4, 0.00 for the four polish-limited systems) | sparsh/real-systems |
 | R27 | **Crossover replicates + beats LM** (strengthens R5) | learned beats BOTH random and LM at high-n on **2/3** separable families (baseline: learned 1.000 vs random 0.000 / LM 0.000 at n=6; wide_roots: 0.225 vs 0/0 at n=4). LM also collapses ~p^n (baseline LM 0.825/0.575/0.200/0.100/0.000). double_well: learned failed to amortize (honest limit) | `PYTHONPATH=.:scripts python3 scripts/run_crossover_families.py --K 8 --test 40 --epochs 200 --ntrain 200` | per-n inline x0 train; test seed0 90000+n; `results/p_scaling/crossover_families.json` | sparsh/crossover-families |
 
 ## Checkpoint regeneration
@@ -54,7 +55,25 @@ regenerated first:
 | R28 | Geometry construction repair, v2 protocol scale (superseded by R28b — kept as the label-noise data point) | ranker 0.238 = restart_control (random 0.19–0.22), McNemar p=0.61; single-stream labels flip ~half on fresh streams | `python3 scripts/run_geo_repair.py --opt-seed {11,29,47} --train-ks 10,12 --transfer-ks 14 --n-train 250 --n-val 80 --n-test 120 --epochs 60` then `scripts/analyze_geo_repair.py` | 2-stream hard-failure population; GEO_REPAIR_VERSION 2; `geo_repair_s{11,29,47}.json`, `analysis.json` | merged e9d87a6 (was quang/geo-repair) |
 | R28b | Geometry construction repair, v3 (stream-stable labels, 5x data, recipe head) — the citable negative | ranker separates from random for the first time (0.246 ± 0.016 vs 0.185 ± 0.019; McNemar 35/8, Holm p=1.3e-4; transfer 24/8, p=0.021) but ties recipe_only 0.249/best_fixed 0.259 and loses to restart_control 0.270 (Holm p=1.0); enumeration 0.692@72.7 restarts, restart +16/+32 0.572/0.725; probe 0.698@19.5 (transfer 0.680@19.5 vs enum 0.616); N=367/331 | `python3 scripts/run_geo_repair.py --opt-seed {11,29,47} --train-ks 10,12 --transfer-ks 14 --n-train 1250 --n-val 400 --n-test 600 --epochs 60` then `scripts/analyze_geo_repair.py results/p_geo_repair/geo_repair_v3_s{11,29,47}.json --out results/p_geo_repair/analysis_v3.json` | data seed 20260722; GEO_REPAIR_VERSION 3; label streams 3; `geo_repair_v3_s{11,29,47}.json`, `analysis_v3.json` | merged e9d87a6 (was quang/geo-repair) |
 | R28c | Probe-concentration control + probe-outcome distillation — closes the probe-ranker gap | cross-fitted selection ceiling at K_REF: 0.199 [0.161,0.243] trained / 0.169 [0.133,0.213] transfer — BELOW restart_control 0.270; 11% of candidates ever pass a screen (probe = portfolio diversity, not selection); optimistic-vs-crossfit 0.762 vs 0.199 prices the selection-on-noise trap; distillation (3,874 probe-labeled train failures, epochs 15) lands 0.256 ± 0.004 trained / 0.182 ± 0.001 transfer — onto the prior (vs best_fixed W/L 6/6; vs restart_control 8/12, Holm p=1.0), as the ceiling predicts | `python3 scripts/probe_concentration.py --workers 6`; `python3 scripts/run_geo_repair.py --opt-seed {11,29,47} --train-ks 10,12 --transfer-ks 14 --n-train 6250 --n-val 400 --n-test 600 --train-label-restarts 1 --train-label-streams 1 --epochs 15` then `scripts/analyze_geo_repair.py ... --out analysis_p3.json` | screen salts +5/+6, grade salt +7 (protocol streams +0..+4 untouched); `probe_concentration.json`, `geo_repair_p3_s{11,29,47}.json`, `analysis_p3.json` | merged e9d87a6 (was quang/geo-repair) |
-| R30 | **Real-systems construction repair (external anchor, Issue #124)** | trilat_far fail 0.848 ± 0.020 (N=509 pooled) / conic_ghost 0.263 ± 0.015 (N=158); held-out selected construction **1.000 ± 0.000** both (Wilson [0.99,1.00] / [0.98,1.00]) vs enumeration-budget-matched restart **0.433 ± 0.049 / 0.114 ± 0.011** (per-seed McNemar at worst 5.7e-14) | `PYTHONPATH=. python3 scripts/pilot_real_repair.py --n 200 --out results/p_real_repair/real_repair.json`; `PYTHONPATH=. python3 scripts/run_real_repair_multiseed.py` (seed bases 20260722,21260722,22260722) | LM K=4, tol 1e-6; two-stream failure selection, construction selection held out by seed parity; ik3r_random, circles_far: no failure population (documented negatives); `real_repair.json`, `real_repair_multiseed.json` | merged e9d87a6/2e40649 |
+| R30 | **Real-systems construction repair (external anchor, Issue #124)** | trilat_far fail 0.848 ± 0.020 (N=509 pooled) / conic_ghost 0.263 ± 0.015 (N=158); held-out selected construction **1.000 ± 0.000** both (Wilson [0.99,1.00] / [0.98,1.00]) vs enumeration-budget-matched restart **0.433 ± 0.049 / 0.114 ± 0.011** (per-seed McNemar at worst 5.7e-14) | `PYTHONPATH=. python3 scripts/pilot_real_repair.py --n 200 --out results/p_real_repair/real_repair.json`; `PYTHONPATH=. python3 scripts/run_real_repair_multiseed.py` (seed bases 20260722,21260722,22260722) | LM K=4, tol 1e-6; two-stream failure selection, construction selection held out by seed parity; ik3r_random, circles_far: no failure population (documented negatives); own-budget restart control (`aggregate.*.restart4`) 0.079 ± 0.017 / 0.026 ± 0.010, cited in Appendix Table 8; `real_repair.json`, `real_repair_multiseed.json` | merged e9d87a6/2e40649 |
+| R23b | **Operator-feature ablation** (attribution check, cited in the paper's Method and repair section) | masking operator-identity features (factor degree/has\_cross/has\_square; edge diag-quadratic/max-exponent/cross) and retraining leaves the ranker intact: nonlinear **0.981** [0.960,0.991] vs 0.997, linear **0.379** [0.352,0.407] vs 0.339 — the v0.2→v0.3 gain is candidate conditioning, not operator flags | `python3 scripts/run_repair_ranker.py --mask-operator-features --train-data nonlinear --n-train 320 --n-val 120 --n-test 360 --epochs 120 --batch-size 16 --D 96 --L 3 --lr 0.0008 --out results/p_repair/nonlinear_opmask_ablation.json`; same with `--train-data aux_required --exclude-family shared ... --out results/p_repair/linear_opmask_ablation.json` | **Regenerated and committed this pass.** The nonlinear artifact now ships; the paper cites 0.978 [0.957,0.989]. **This arm is not bit-reproducible at fixed seed**: identical reruns gave 0.978 / 0.981 / 0.986 (torch CPU nondeterminism), all supporting the qualitative claim that masking operator features leaves the ranker intact. Cite the interval, not the third decimal, and do not "correct" it by rerunning. **The linear counterpart was also regenerated and it does not reproduce**: the masked linear arm now reads 0.229 [0.206,0.254] against a random arm of 0.249 (N=1200), where RESULTS.md recorded 0.379 [0.352,0.407] — non-overlapping intervals. This is consistent with the documented instability of the linear signal (R22: 0.317 +- 0.069, one seed at 0.227 below random's 0.248), not with a new fault. The paper therefore cites the nonlinear ablation only; do not restore the 0.379/0.339 pair | pending |
+
+## Figures (regeneration commands)
+Figures are rebuilt from the committed JSONs; none of these commands recomputes an
+experiment, so a figure refresh cannot move a cited number.
+- `paper/tex/figures/fig_crossover_theory.pdf` (main-text Figure 1) —
+  `PYTHONPATH=.:scripts python3 scripts/run_crossover_theory.py --replot`, which redraws from
+  `results/p_crossover/crossover_theory.json` and `results/p_scaling/scaling.json`. Omitting
+  `--replot` re-measures q(n) from scratch; do not do that for presentation-only changes.
+- `paper/tex/figures/fig_regime_map.pdf` (Figure 2) — `PYTHONPATH=.:scripts python3
+  scripts/plot_regime_map.py`. Reads crossover_theory / crossover_families / scaling /
+  coupled / pointchain_learned; asserts the plotted outcomes against those JSONs and fails
+  loudly if they move.
+- `paper/tex/figures/fig_repair.pdf` (Appendix Figure 3) — `python3 scripts/plot_repair.py`.
+- Both main-text figures are drawn at the width of the slot that holds them (Figure 1 at
+  `\textwidth`, Figure 2 at `\columnwidth`) so they print at ~1:1 and their point sizes are
+  the sizes the reader sees. Changing `figsize` in the scripts without changing the
+  `\includegraphics` width is what makes the annotations unreadable.
 
 ## Notes / caveats attached to specific numbers
 - **Dimension scaling is now cited from R15** (methodology unified-v2, `results/p_scaling/scaling.json`,

paper/RESULTS.md

@@ -293,7 +293,11 @@ offset 0.407, coupled 0.450, shared 0.380, each vs ~0.23–0.29 random (N=400 pe
 cell) — the linear pattern-transfer effect is not specific to one held-out choice.
 
 **Operator-feature ablation (attribution check; `nonlinear_opmask_ablation.json`,
-`linear_opmask_ablation.json`):** masking the operator-identity features (factor
+`linear_opmask_ablation.json`).**  *Regenerated 2026-07-29: nonlinear reproduces at 0.978
+[0.957,0.989] (not bit-stable at fixed seed; 0.978/0.981/0.986 observed).  Linear does NOT
+reproduce -- rerun gives 0.229 [0.206,0.254] vs random 0.249, against the 0.379 below, which
+matches the known linear seed instability.  The paper cites the nonlinear ablation only.*
+Original record: masking the operator-identity features (factor
 degree/has_cross/has_square; edge diag-quadratic/max-exponent/cross) and retraining
 leaves the ranker essentially intact — nonlinear 0.981 [0.960,0.991] vs 0.997, linear
 0.379 [0.352,0.407] vs 0.339.  The v0.2→v0.3 gain is therefore attributed to

paper/tex/STYLE_KIT_STATUS.md

@@ -33,3 +33,38 @@ Kit notes: `natbib` must be loaded with **no options** (line 23 is already fine)
 author block for submission stays anonymous (the `submission` option hides it).
 Reference template: `AuthorKit26/AnonymousSubmission/LaTeX/anonymous-submission-latex-2026.tex`
 (kit zip extracted at /tmp/aaai_kit/ if still around; otherwise re-download from the URL above).
+
+## AAAI-27 migration (2026-07-29)
+
+Source of the kit: `AuthorKit27.zip` supplied by the authors. `aaai2027.sty` and
+`aaai2027.bst` are copied into this directory unmodified.
+
+Changes forced by the 2027 kit, each of which the AAAI-26 draft violated:
+
+- **Font packages are forbidden.** The kit loads newtxtext/helvet/courier itself and says
+  "DO NOT add \usepackage{times}, \usepackage{helvet}, \usepackage{courier}, or any other
+  font package." The 2026 preamble loaded all three, plus a XeTeX `fontspec` shim. All
+  removed.
+- **`\clearpage` is on the disallowed-commands list** ("No page breaks of any kind"). The
+  appendix used it. Removed; the one-column switch alone now starts the supplementary block.
+- **Section order is mandated**: main content, content appendices, ethical statement,
+  acknowledgments, references, supplementary material. The "Use of AI Assistants" statement
+  had been placed after the bibliography and is now before it.
+- `\pdfinfo{/TemplateVersion (2027.1)}`, `\urlstyle{rm}` and `\def\UrlFont{\rm}` added per
+  the template.
+
+**This directory can no longer be built locally.** `aaai2027.sty` hard-requires pdfTeX
+("pdfTeX is required to compile this document") and aborts under XeTeX, which is the only
+engine `tectonic` provides. Build on Overleaf with pdfLaTeX, which is where the submission
+artifact should come from anyway.
+
+For local pagination estimates only, copy `paper/tex/` to a scratch directory and swap
+`aaai2027` for `aaai2026` plus the old font block; the two styles share the two-column
+letterpaper geometry and metrically equivalent Times clones, so the estimate is close but
+is **not** the submission artifact.
+
+**Open:** the 2027 kit does not state a numeric page limit ("Check the conference's
+instructions in their website"). Under the 2026-proxy build the content occupies 7 pages
+and the AI-use statement spills roughly four lines onto an eighth. Confirm the AAAI-27
+limit before submitting; if it is 7, cut four lines or move the statement after the
+references.