conformance(chant): answer the whole-build fixtures in isolated mode (chant#2442, chant#2438) - #115
Merged
Merged
Conversation
…(chant#2442) `isolated` returned "unavailable" because chant had no honest way to answer it. Its nearest setting, `--sandbox` (chant#1093), is not the same rule: isolated refuses to INVOKE, so F-Call step 5 fires and step 6 never imports, while chant's sandbox refuses to import project code at all. Wiring them together traded one disagreement for another — the four negatives agreed and `good.ts` and `named.ts` broke, reference `fold` against chant `run` — so the adapter skipped visibly instead. That was chant#2442 rather than a difference between the two rules. A host's `Composite` was not recognised as the registration form, so nothing was interpreted and every fixture fell to the invocation arm the sandbox refuses. chant-v0.72.1 recognises it, and returns the host's members as the factory built them rather than wrapping them in chant's own `Composite`, whose member validation asks for chant's `Declarable` and refused an entity carrying a different marker. So `foldOnDisk` now passes `sandbox: true` under isolated, behind a probe. The probe is a behaviour test rather than a version comparison, matching `acceptsHostPackages` next to it: it folds a host-registered composite under sandbox, which answers only if the form is recognised AND the host's own entity survives validation. A chant without the fix reports `run` and the mode stays unavailable, so an older pin skips visibly instead of reporting wrong verdicts. Three fixtures move from skipped to answered, and all three agree with the reference on every verdict and every export: F-IsolatedRefusal/a-project-factory-is-refused-not-invoked S-FactoryBody/consts-then-a-final-return S-FactoryParams/one-plainly-bound-parameter With nothing left unanswered, the skip allowance in the agreement test is retired. It let any fixture naming a host skip, first for chant#2408 and then for chant#2442; both are fixed, so the test now asserts zero skips across all 56 project fixtures. A silent return to "unavailable" fails there rather than reading as agreement, which is the failure that cost a day when `compareAdapters` reported nothing for an unavailable side.
The release carrying chant#2442. Both pins move together — the devDependency and corpus.yml's CHANT_REF — because a corpus number is only reproducible if the engine under test and the corpus come from the same tag, and moving one without the other measures a mixture of two. The numbers do not move, which is the point of recording them: files 441, comparable 372, agreed 372, both fold 258 no host 1, host call in value position 68 data-host: 441 files, 441 agreed, 142 both fold `isolated` becoming answerable adds three fixtures to the agreement suite and takes nothing away from the corpus, so a change in these would have been a regression rather than the fix landing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the last gap in chant's adapter.
isolatedis answerable, the three fixtures it was blocking now agree with the reference, and the pin moves to the release that made that possible.Also satisfies chant#2438's done-when — "no skipped fixture except the round-trip family, and
compareAdapterscovers findings and hosted builds" — so that issue closes with this too, not just #2442.Why
isolatedwas unavailablechant has no mode that means isolated. Its nearest setting,
--sandbox(chant#1093), is a different rule: isolated refuses to invoke, so F-Call step 5 fires and step 6 never imports, while chant's sandbox refuses to import project code at all. Wiring them together traded one disagreement for another — the four negatives agreed andgood.tsandnamed.tsbroke, referencefoldagainst chantrun— so the adapter skipped visibly instead.That turned out to be chant#2442 rather than a real difference between the two rules. A host's
Compositewas not recognised as the registration form, so nothing was interpreted and every fixture fell through to the invocation arm the sandbox refuses. chant-v0.72.1 recognises it, and returns the host's members as the factory built them instead of wrapping them in chant's ownComposite, whose member validation asks for chant'sDeclarableand refused an entity carrying a different marker.What changed
foldOnDiskpassessandbox: trueunder isolated, behind a probe. The probe is a behaviour test rather than a version comparison, matchingacceptsHostPackagesbeside it: it folds a host-registered composite under sandbox, which answers only if the form is recognised and the host's own entity survives validation. A chant without the fix reportsrun, so an older pin skips visibly rather than reporting wrong verdicts. Verified by running the suite against 0.72.0, where it correctly goes unavailable.Three fixtures move from skipped to answered, all agreeing on every verdict and every export:
Measured 3 → 0 skips across 56 project fixtures, taken both ways — the before figure came from checking out the previous adapter and re-running, not from assuming.
Two things beyond the plain restore
The skip allowance is retired. It let any fixture naming a host skip, first under chant#2408 and then chant#2442. Both are fixed, so it had nothing left to excuse, and the test now asserts zero skips. That closes a real hole:
compareAdaptersreports nothing when one side is"unavailable", so a silent return to unavailable read as agreement — the false signal that cost a day earlier in this work.The hold-out list is split by reason.
F-Call/a-host-factory-is-invokedwas sitting insidefoldsAtDepth, a list named for chant#2441's depth bug, so four entries sat under two unrelated reasons with a guard test that said three. NowfoldsAtDepth(chant#2441) andopenSpecQuestion(#110) are separate,heldOutis their union, and the guard runs per list. Each empties on its own event — a chant fix retires one, a J1 ruling the other — and neither can go on excusing the other's fixtures.Pins and the corpus
Both pins move together, the devDependency and
corpus.yml'sCHANT_REF, because a corpus number is only reproducible if the engine under test and the corpus come from the same tag.Rerun at
chant-v0.72.1(75c05827) in a worktree seeded per the usual procedure. The numbers do not move, which is the point of recording them:data-host: 441 files, 441 agreed, 142 both fold.
isolatedbecoming answerable adds three fixtures to the agreement suite and takes nothing from the corpus, so movement here would have been a regression rather than the fix landing.Gates
tsc --noEmitclean; 37 passed / 10 skipped.