paper: final-submission pass — Table 4 corrected against the artifact, missing ablation landed, weak-baseline literature cited - #144
Merged
Conversation
… the missing ablation
Table 4 disagreed with the results file it is supposed to report. The paper had
BilinearProduct at 0.717 / "random wins"; results/p_hard/hard_eval.json has 41/60 =
0.683, identical to the learned arm, i.e. a tie. The CI in the paper ([0.59,0.81]) is
the Wilson interval for 43/60, so both the rate and its interval came from a run that
is not the one we ship. PROVENANCE R11c recorded the same stale value, so the drift
predates this pass and verify_paper_numbers.py never covered that table.
* Appendix Table 4 is now generated from hard_eval.json, and the main text reads
"no win on any of the four families: three ties and one complete failure on
CircleLine" instead of "ties on two and loses on two"
* added the LM arm the table omitted (60/60 on all four families) with a caption
line saying why it is there: the strong classical solver saturates these families,
which is exactly why learned-vs-random is the informative contrast
* +7 verifier rows over the hybrid battery so this cannot drift again (52 checks)
* R11c kept as history; a dated R11c-note records the correction
Operator-mask ablation: RESULTS.md recorded it but the two JSONs were never committed
(missing from the .gitignore whitelist), so nothing could check the cited numbers.
Both regenerated and committed.
* nonlinear reproduces at 0.978 [0.957,0.989] vs 0.997 unmasked; it is NOT
bit-reproducible at fixed seed (0.978/0.981/0.986 across identical reruns), so the
paper now cites the interval and says so
* linear does NOT reproduce: 0.229 [0.206,0.254] against a random arm of 0.249, where
RESULTS.md had 0.379 [0.352,0.407] — non-overlapping. This matches the documented
linear seed instability (R22: one seed at 0.227 below random's 0.248), so the paper
cites the nonlinear ablation only and PROVENANCE says not to restore 0.379
Related work: cite the weak-baseline critique literature this result sits in — Nath &
Kuhnle's max-cut benchmark and learned-local-search study, and Rodrigues et al.'s
budget-matched LLM HPO study. We claim neither the observation nor the remedy as new;
what is new is that one polish and one checker make best-of-K the exact bound a learned
proposal must clear.
Citations: three entries had no author field and rendered as "arXiv:2502.00277. 2025.".
All three are real papers; two had paraphrased titles. Corrected against arXiv.
Also: title and abstract tracked to the OpenReview record (92% verbatim, three forced
changes — false range, operator-aware vs candidate-conditioned, and an abstract claim
of reach that the body disclaims); figures redrawn at the width of the slot that holds
them so their labels are legible; prose de-theatricalized and em-dash-free; "Use of AI
Assistants" statement added after the bibliography.
7 content pages, 52/52 number checks, 431 tests.
The submission is AAAI-27 but the source built against aaai2026.sty. The 2027 kit
forbids three things this draft did.
* font packages are forbidden -- the kit loads newtxtext/helvet/courier itself and
says not to add times/helvet/courier "or any other font package". We loaded all
three plus a XeTeX fontspec shim; all removed
* \clearpage is on the disallowed-commands list ("No page breaks of any kind"). The
appendix used it; the one-column switch alone now starts the supplementary block
* section order is mandated (main content, content appendices, ethical statement,
acknowledgments, references, supplementary). The "Use of AI Assistants" statement
sat after the bibliography and now precedes it
Also adds \pdfinfo{/TemplateVersion (2027.1)}, \urlstyle{rm}, \def\UrlFont{\rm}, and
copies aaai2027.sty / aaai2027.bst in unmodified.
paper/tex can no longer be built locally: aaai2027.sty hard-requires pdfTeX and aborts
under XeTeX, which is the only engine tectonic provides. Build on Overleaf with
pdfLaTeX. STYLE_KIT_STATUS.md records a scratch-directory recipe for local pagination
estimates and flags the open question -- the kit states no numeric page limit, and under
the 2026-proxy build the content is 7 pages with the AI-use statement spilling about
four lines onto an eighth.
52/52 number checks, 431 tests.
…im ~70 words - AI-use statement replaced with the authors' shorter text (double space fixed) - Appendix Table 4 was 32pt overfull after the LM column landed: headers shortened, tabcolsep tightened, column meanings moved into the caption - Table 3 was 0.7pt overfull: the two stationary-point rows now use a bare nabla, explained in the caption - three body paragraphs compressed Local build remains a 2026-style proxy (aaai2027.sty needs pdfTeX). Under that proxy the content runs about half a page past 7; see the note below for the levers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The blocking fix
Appendix Table 4 disagreed with the results file it reports. The paper had BilinearProduct at 0.717 / "random wins";
results/p_hard/hard_eval.jsonhas 41/60 = 0.683, identical to the learned arm — a tie. The CI in the paper ([0.59, 0.81]) is the Wilson interval for 43/60, so both the rate and its interval came from a run we do not ship.PROVENANCER11c carried the same stale value, so the drift predates this pass, andverify_paper_numbers.pynever covered that table.The missing artifact
The operator-mask ablation was cited in the paper and recorded in
RESULTS.md, but both JSONs had never been committed — they were missing from the.gitignorewhitelist, so nothing could check them. Regenerated and committed:RESULTS.mdrecorded 0.379 [0.352, 0.407] — non-overlapping intervals, masked arm below chance. Consistent with the documented linear seed instability (R22: one seed at 0.227 below random's 0.248), not a new fault. The paper now cites the nonlinear ablation only; PROVENANCE says not to restore 0.379.Shipping the original sentence would have meant a reviewer running the artifact finding a cited number that inverts.
Positioning
Added the weak-baseline critique literature this result sits inside: Nath & Kuhnle's max-cut benchmark and learned-local-search study, and Rodrigues et al.'s budget-matched LLM HPO study. We claim neither the observation nor the remedy as new. What is new is that one polish and one checker make best-of-$K$ the exact bound a learned proposal must clear, and the measured$\log q(n)$ slope predicts which families leave room above it.
Citations
Three entries had no author field and rendered as "arXiv:2502.00277. 2025." All three are real papers; two had paraphrased titles. Corrected against arXiv (Feng & Yang; Ding et al.; Wang et al.).
Presentation
operator-aware→candidate-conditioned, which the ablation contradicts; and an abstract claim of reach that the Conclusion disclaims)--replotadded torun_crossover_theory.pyso the figure can be redrawn without re-measuringState
7 content pages, 52/52 number checks, 431 tests, no overfull boxes, no unresolved references.
Still open for the team
aaai2026.styvs the AAAI-2027 submission record — page limits move between years