Skip to content

Compare whole-build fixtures against chant (#62) - #65

Merged
lex00 merged 1 commit into
mainfrom
conformance-project-cross-check
Sep 12, 2026
Merged

lex00 merged 1 commit into
mainfrom
conformance-project-cross-check

Conversation

@lex00

@lex00 lex00 commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Part of #62, which stays open until the chant pin moves.

chant grew a public whole-build entry — chant#2408, merged as chant PR #2410 — so the taint fixpoint is answerable by two implementations rather than one.

What landed

  • foldProject on the adapter may be async. chant resolves modules from a filesystem, so the chant adapter writes a fixture's sources to a temp directory and hands over the paths; runFixtures, runProjectFixture and compareAdapters follow.
  • The comparison asserts more than the verdict. A seed and a taint casualty are both run, so where both implementations report the tentative verdict and the file whose taint reached each tainted one, those are compared too. Comparing only the final verdict would pass an implementation that got both edges wrong in compensating ways.
  • A skipped project fixture must have a reason. A fixture naming a host is skipped for chant, since its entity classes come from a package chant cannot resolve and its entry takes an intrinsic registry rather than a whole host. Anything else skipped fails the test.

The result, run against the local chant checkout

Eight whole-build fixtures, nineteen file verdicts, zero disagreements — including both tentative verdicts, and the taint source and edge kind for all four taint casualties:

Fixture File ref / chant tentative tainted by
F-Succ/forward-into-imports config.ts run / run fold / fold app.ts, importer
F-Succ/backward-along-captures shared.ts run / run fold / fold breaker.ts, importer
F-Succ/backward-along-captures sibling.ts run / run fold / fold shared.ts, capture
F-Val-Live/plain-data-is-not-live source.ts run / run fold / fold breaker.ts, importer

chant independently classifies the capturing sibling as reached by a capture edge rather than an import, which is the distinction chant#2406 existed for. And F-Val-Live/plain-data-is-not-live agrees: chant also declines to taint a file whose only tie to another is a call returning computed plain data, which is F-Identity's entity test (#59) holding in an implementation that has never read it.

Why #62 stays open

The pinned chant is 0.69.1 and predates the entry, so CI skips every whole-build fixture. The test asserts that explicitly rather than tolerating it, and the numbers above came from a local checkout, not from CI. paper/measurements.md is deliberately unchanged: it will claim two implementations agree on J3 when CI checks it, and not before.

🤖 Generated with Claude Code

https://claude.ai/code/session_01RfnRhfdYHLFAZJKHwZhBYc

chant grew a public whole-build entry (chant#2408, merged as chant PR
#2410), so the taint fixpoint is answerable by two implementations instead
of one.

`foldProject` on the adapter may now be async: chant resolves modules from
a filesystem, so the chant adapter writes a fixture's sources to a temp
directory and hands over the paths. `runFixtures`, `runProjectFixture` and
`compareAdapters` follow.

The comparison asserts more than the verdict. A seed and a taint casualty
are both "run", so where both implementations report the tentative verdict
and the file whose taint reached each tainted one, those are compared too.
That is where J3 actually lives; comparing only the final verdict would pass
an implementation that got both edges wrong in compensating ways.

Run against the local chant checkout ahead of a release: eight whole-build
fixtures, nineteen file verdicts, zero disagreements, including both
tentative verdicts and the taint source and edge kind for all four taint
casualties. chant independently classifies the capturing sibling as reached
by a capture edge rather than an import, which is the distinction chant#2406
existed for.

A host-dependent fixture is still skipped for chant: its entity classes come
from a package chant cannot resolve, and its entry takes an intrinsic
registry rather than a whole host. The test asserts that a skip has a host to
explain it, so a silently skipped fixture fails instead of passing.

The pinned chant is 0.69.1 and predates the entry, so CI still skips every
whole-build fixture and the test asserts that too, explicitly, rather than
tolerating it. #62 stays open until the pin moves.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RfnRhfdYHLFAZJKHwZhBYc
@lex00
lex00 merged commit 0cade94 into main Sep 12, 2026
2 checks passed
@lex00
lex00 deleted the conformance-project-cross-check branch September 12, 2026 01:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant