docs: fix remaining stale TTFT/reuse-ceiling claims repo-wide - #224
Merged
Conversation
Follow-up to #223 (README fix). Swept the rest of the repo for the same disproven "167ms vs 192ms, Ollama wins cold TTFT" claim and the unqualified 9.8x reuse-ceiling framing, both already corrected in the blog post and paper.md: - docs/index.md (squish.run homepage): same stale TTFT table row and prose claim as the old README, now replaced with the corrected cold/unique floor (800ms vs 812ms, Squish wins). Qualified the 9.8x bullet as the repeated-prompt ceiling. - BENCHMARKS.md (included verbatim into docs/RESULTS.md, so this also fixes the live "Benchmark Results" page): section 0 was an entirely stale v9.32.0 headline snapshot with different, older numbers (69.63s/12.78s/5.4x, TTFT 131ms/279ms) that no longer matched README's current numbers despite claiming to source from it - updated to the current v9.34.14 figures. Section 1b's "TTFT (loaded, 75 tok)" row and "only loss" claim used the same retracted 167/192 methodology - removed, with a pointer to section 1c. Section 1c (the already-correct cold/unique sweep) gained the missing 75-token data point from CTX75_SUPPLEMENT_RESULTS.md and an updated summary range (1.15-1.32x, not the earlier 1.2-1.3x approximation). - overrides/home.html: a fourth, previously-missed copy of the same claim in the marketing landing page's comparison table footnote ("TTFT is comparable, Ollama 167ms / Squish 192ms"). Replaced with the corrected cold/unique numbers and a repeated-prompt qualifier on the 9.8x row. Left untouched: a dated CHANGELOG.md entry under a historical version heading (that's an accurate record of what a past release's notes said, not a live claim), and the benchmarks/ollama_vs_squish/*.md research artifacts that already correctly document the investigation and retraction themselves.
wesleyscholl
marked this pull request as ready for review
July 7, 2026 11:54
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #223, which fixed the stale "167ms vs 192ms, Ollama wins cold TTFT" claim in the README. This PR does a full repo-wide sweep (all file types, not just
.md) for the same claim and the related unqualified 9.8× reuse-ceiling framing, and finds/fixes three more places it was hiding:docs/index.md— the squish.run homepage. Had the identical stale TTFT table row and prose as the old README. Replaced with the corrected cold/unique floor (Squish 800ms vs Ollama 812ms, a win not a loss). Qualified the "9.8× long-context wins" bullet as the repeated-prompt ceiling.BENCHMARKS.md(included verbatim intodocs/RESULTS.mdvia--8<--, so this also fixes the live "Benchmark Results" page):CTX75_SUPPLEMENT_RESULTS.md, and tightened the summary range from an approximate "1.2-1.3×" to the precise "1.15-1.32×".overrides/home.html— a fourth, previously-missed copy of the same claim in the marketing landing page's comparison table footnote ("TTFT is comparable, Ollama 167ms / Squish 192ms"). This one a plain-markdown search wouldn't have caught since it's HTML, not.md. Replaced with the corrected numbers, and added a repeated-prompt qualifier to the 9.8× table row + footnote.Deliberately left untouched:
CHANGELOG.mdentry under a historical version heading (v5.1.1) — that's an accurate record of what that release's notes said at the time, not a live claim; changelogs aren't rewritten after the fact.benchmarks/ollama_vs_squish/*.mdresearch artifacts (CTX75_SUPPLEMENT_RESULTS.md,COLD_UNIQUE_H2H_RESULTS.md, etc.) — these already correctly document the investigation and retraction themselves; they're the source of truth the other fixes cite.Type of change
Checklist
pytest tests/passes locally — N/A, docs-only changeruff check squish/ tests/reports no errors — N/A, no Python changedsquish benchrun — N/AVerified locally with
mkdocs build+ a repo-wide grep across every file type (not just.md) for the stale phrases — confirmed zero remaining matches outside the two deliberately-excluded historical/source-of-truth files. Also took a Playwright screenshot of the rendered homepage comparison table and footnote to confirm the corrected copy displays cleanly.Related issues
Follow-up to #215–#223.
Generated by Claude Code