Skip to content

paper: final-submission pass — Table 4 corrected against the artifact, missing ablation landed, weak-baseline literature cited - #144

Merged
ImSpxrsh merged 3 commits into
mainfrom
sparsh/final-submission-pass
Jul 29, 2026
Merged

paper: final-submission pass — Table 4 corrected against the artifact, missing ablation landed, weak-baseline literature cited#144
ImSpxrsh merged 3 commits into
mainfrom
sparsh/final-submission-pass

Conversation

@ImSpxrsh

Copy link
Copy Markdown
Member

The blocking fix

Appendix Table 4 disagreed with the results file it reports. The paper had BilinearProduct at 0.717 / "random wins"; results/p_hard/hard_eval.json has 41/60 = 0.683, identical to the learned arm — a tie. The CI in the paper ([0.59, 0.81]) is the Wilson interval for 43/60, so both the rate and its interval came from a run we do not ship. PROVENANCE R11c carried the same stale value, so the drift predates this pass, and verify_paper_numbers.py never covered that table.

  • Table 4 is now generated directly from the JSON
  • main text: "no win on any of the four families: three ties and one complete failure on CircleLine" (was "ties on two and loses on two")
  • added the LM arm the table omitted — 60/60 on all four families — with a caption line explaining why it belongs: the strong classical solver saturates these families, which is precisely why learned-vs-random is the informative contrast. Omitting it was a cherry-picking exposure.
  • +7 verifier rows over the hybrid battery (52 checks total)
  • R11c preserved as history, annotated rather than rewritten

The missing artifact

The operator-mask ablation was cited in the paper and recorded in RESULTS.md, but both JSONs had never been committed — they were missing from the .gitignore whitelist, so nothing could check them. Regenerated and committed:

  • nonlinear reproduces: 0.978 [0.957, 0.989] vs 0.997 unmasked. It is not bit-reproducible at fixed seed (0.978 / 0.981 / 0.986 across identical reruns), so the paper cites the interval and says so.
  • linear does not reproduce: 0.229 [0.206, 0.254] against a random arm of 0.249, where RESULTS.md recorded 0.379 [0.352, 0.407] — non-overlapping intervals, masked arm below chance. Consistent with the documented linear seed instability (R22: one seed at 0.227 below random's 0.248), not a new fault. The paper now cites the nonlinear ablation only; PROVENANCE says not to restore 0.379.

Shipping the original sentence would have meant a reviewer running the artifact finding a cited number that inverts.

Positioning

Added the weak-baseline critique literature this result sits inside: Nath & Kuhnle's max-cut benchmark and learned-local-search study, and Rodrigues et al.'s budget-matched LLM HPO study. We claim neither the observation nor the remedy as new. What is new is that one polish and one checker make best-of-$K$ the exact bound a learned proposal must clear, and the measured $\log q(n)$ slope predicts which families leave room above it.

Citations

Three entries had no author field and rendered as "arXiv:2502.00277. 2025." All three are real papers; two had paraphrased titles. Corrected against arXiv (Feng & Yang; Ding et al.; Wang et al.).

Presentation

  • title/abstract tracked to the OpenReview record — 92% verbatim, three forced changes (false range; operator-awarecandidate-conditioned, which the ablation contradicts; and an abstract claim of reach that the Conclusion disclaims)
  • both main-text figures redrawn at the width of the slot that holds them; they were being shrunk to ~41% and their labels were unreadable
  • --replot added to run_crossover_theory.py so the figure can be redrawn without re-measuring
  • prose pass: em dashes 58 → 0, colon sentences 33% → 21%, "The" openers 30% → 21%
  • "Use of AI Assistants" statement added after the bibliography

State

7 content pages, 52/52 number checks, 431 tests, no overfull boxes, no unresolved references.

Still open for the team

  • aaai2026.sty vs the AAAI-2027 submission record — page limits move between years
  • the title still foregrounds the diffusion proposer, the half the paper explicitly does not claim

ImSpxrsh added 3 commits July 29, 2026 01:16
… the missing ablation

Table 4 disagreed with the results file it is supposed to report. The paper had
BilinearProduct at 0.717 / "random wins"; results/p_hard/hard_eval.json has 41/60 =
0.683, identical to the learned arm, i.e. a tie. The CI in the paper ([0.59,0.81]) is
the Wilson interval for 43/60, so both the rate and its interval came from a run that
is not the one we ship. PROVENANCE R11c recorded the same stale value, so the drift
predates this pass and verify_paper_numbers.py never covered that table.

  * Appendix Table 4 is now generated from hard_eval.json, and the main text reads
    "no win on any of the four families: three ties and one complete failure on
    CircleLine" instead of "ties on two and loses on two"
  * added the LM arm the table omitted (60/60 on all four families) with a caption
    line saying why it is there: the strong classical solver saturates these families,
    which is exactly why learned-vs-random is the informative contrast
  * +7 verifier rows over the hybrid battery so this cannot drift again (52 checks)
  * R11c kept as history; a dated R11c-note records the correction

Operator-mask ablation: RESULTS.md recorded it but the two JSONs were never committed
(missing from the .gitignore whitelist), so nothing could check the cited numbers.
Both regenerated and committed.

  * nonlinear reproduces at 0.978 [0.957,0.989] vs 0.997 unmasked; it is NOT
    bit-reproducible at fixed seed (0.978/0.981/0.986 across identical reruns), so the
    paper now cites the interval and says so
  * linear does NOT reproduce: 0.229 [0.206,0.254] against a random arm of 0.249, where
    RESULTS.md had 0.379 [0.352,0.407] — non-overlapping. This matches the documented
    linear seed instability (R22: one seed at 0.227 below random's 0.248), so the paper
    cites the nonlinear ablation only and PROVENANCE says not to restore 0.379

Related work: cite the weak-baseline critique literature this result sits in — Nath &
Kuhnle's max-cut benchmark and learned-local-search study, and Rodrigues et al.'s
budget-matched LLM HPO study. We claim neither the observation nor the remedy as new;
what is new is that one polish and one checker make best-of-K the exact bound a learned
proposal must clear.

Citations: three entries had no author field and rendered as "arXiv:2502.00277. 2025.".
All three are real papers; two had paraphrased titles. Corrected against arXiv.

Also: title and abstract tracked to the OpenReview record (92% verbatim, three forced
changes — false range, operator-aware vs candidate-conditioned, and an abstract claim
of reach that the body disclaims); figures redrawn at the width of the slot that holds
them so their labels are legible; prose de-theatricalized and em-dash-free; "Use of AI
Assistants" statement added after the bibliography.

7 content pages, 52/52 number checks, 431 tests.
The submission is AAAI-27 but the source built against aaai2026.sty. The 2027 kit
forbids three things this draft did.

  * font packages are forbidden -- the kit loads newtxtext/helvet/courier itself and
    says not to add times/helvet/courier "or any other font package". We loaded all
    three plus a XeTeX fontspec shim; all removed
  * \clearpage is on the disallowed-commands list ("No page breaks of any kind"). The
    appendix used it; the one-column switch alone now starts the supplementary block
  * section order is mandated (main content, content appendices, ethical statement,
    acknowledgments, references, supplementary). The "Use of AI Assistants" statement
    sat after the bibliography and now precedes it

Also adds \pdfinfo{/TemplateVersion (2027.1)}, \urlstyle{rm}, \def\UrlFont{\rm}, and
copies aaai2027.sty / aaai2027.bst in unmodified.

paper/tex can no longer be built locally: aaai2027.sty hard-requires pdfTeX and aborts
under XeTeX, which is the only engine tectonic provides. Build on Overleaf with
pdfLaTeX. STYLE_KIT_STATUS.md records a scratch-directory recipe for local pagination
estimates and flags the open question -- the kit states no numeric page limit, and under
the 2026-proxy build the content is 7 pages with the AI-use statement spilling about
four lines onto an eighth.

52/52 number checks, 431 tests.
…im ~70 words

- AI-use statement replaced with the authors' shorter text (double space fixed)
- Appendix Table 4 was 32pt overfull after the LM column landed: headers shortened,
  tabcolsep tightened, column meanings moved into the caption
- Table 3 was 0.7pt overfull: the two stationary-point rows now use a bare nabla,
  explained in the caption
- three body paragraphs compressed

Local build remains a 2026-style proxy (aaai2027.sty needs pdfTeX). Under that proxy the
content runs about half a page past 7; see the note below for the levers.
@ImSpxrsh
ImSpxrsh merged commit 97473cc into main Jul 29, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant