Repository navigation
Forward compat - #90
Merged
Merged
Conversation
BaboonComparator now classifies, per evolution step and per type, whether an older version's codec can decode newer-version data: IDENTICAL (byte-identical), PREFIX_ANY_MODE / PREFIX_COMPACT (appended-only fields; top-level framed UEBA reads), JSON_ADDITIVE (tolerant JSON readers), or nothing. Step tiers are composed into contiguous suffix runs (BaboonEvolution.typesForwardReadable) via an order-sensitive own-structure check plus a fixpoint over codec-relevant dependencies; emitted in baboon-meta.json as "forwardReadable". Scala + TypeScript backends emit the metadata as baboonForwardReadable (per-type constant + BaboonGenerated member + BaboonMetadata lookup), with end-to-end stub specs proving each tier against real cross-version blobs, including negative controls. Design: docs/drafts/20260911-0937-forward-compat-metadata.md. Note: ForwardCompatComparatorTest documents a pre-existing sameIn overclaim (deepSchemaRepr sorts flattened dep reprs, erasing member order inside deps; a reordered-enum host stays 'unmodified' although its UEBA bytes change). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
C#, Python, Rust, Kotlin (+KMP), Java, Dart and Swift now emit the baboonForwardReadable version->tier map as a per-type constant, a BaboonGenerated(-equivalent) member, and a forwardReadableVersions(typeId) registry lookup, mirroring the existing baboonSameInVersions surfaces. Hand-written runtime-test stubs implementing the extended interfaces are updated (C#/Kotlin/KMP/Java/Dart); Swift and Python use source-compatible defaults overridden by generated code; Rust emits the real table where it previously had only degenerate own-version sameIn data. docs/forward-compat.md documents tiers, the prefix-* client contract, and metadata locations. docs/ueba-format.md: corrected the enum wire description (single positional u8, not i32 of the discriminant; const values never hit the wire) — verified against the C#, Scala, TypeScript and Rust generators. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
sameIn/unmodified overclaimed byte-identity: deepSchemaRepr sorted the flattened dependency repr lines per field (and sorted ADT branch reprs), erasing member/field/branch order inside dependencies and type-constructor argument order — while UEBA is positional (enum discriminants, ADT branch indices, field bytes, map K/V order). A host of a reordered dep enum, a reordered dep ADT, or a dep with swapped map arguments classified as 'unmodified', so sameIn ranges promised byte-identity the wire does not have. Fix: dependency reprs stay contiguous and internally ordered (determinism now comes from sorting the dependency IDs instead of the flattened lines), each field line carries its full type-ref rendering, and ADT branch order — the UEBA discriminant order — is preserved. Reproduced fail-first in ForwardCompatComparatorTest (EnumReorderHost / SumReorder / SumReorderHost / MapSwapHost fixtures), which now pins the corrected classification. Consequences: - Lockfiles persist deepId-derived signatures: Locks now carries a 'scheme' marker (2 = this hashing; absent/1 = legacy). A stale-scheme lockfile is incomparable — enforcement is skipped for that one run and the file is re-signed in place even under create-only (T156 matrix extended). - The M20 manual→sugared ADT rewrite is honestly classified: the sugared expansion reorders branches, shifting positional UEBA discriminants, so the step is deepModified with a derivable CopyAdtBranchByName conversion. PR-D's branch sort had masked this wire break; M20AdtEvolutionTest updated to assert the conversion-derivability guarantee instead of byte-identity. Also adds .jvmopts (-Xmx8g -Xss8m): sbt's 1024 MB launcher default OOMs the Scala.js parallel optimizer on many-core machines during 'sbt +test' (reproduced on a clean clone of main, pre-existing). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…dReadPolicy — Scala/TS pilot Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… 9 backends Every runtime now carries BaboonTypeMeta.domainVersionReadableMin (five-field construction keeps readable-min = minCompat), writes/reads the JSON envelope key $rv (elided when equal to the effective $uv), exposes the generated per-type baboonMinReaderVersions (tier -> oldest reader) that feeds it, and resolves JSON codecs under a facade-level ForwardReadPolicy (Tolerant default: honour $rv for payloads newer than any registered version; Lossless: $uv only). UEBA resolution is unchanged (v1 envelope has no slot for the bound). Codegen: baboonMinReaderVersions emitted by all 9 DomainTreeTools / the Rust facade generator. Dart test fakes gain the new provider getter. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…acade meta tables
All nine binary envelope readers treated any hasMinCompat value other than 1 as
"elided", so an illegal flag byte (e.g. 0x02) was silently misparsed — the
min-compat string was consumed as the type identifier. codec-envelope.md §2.1
mandates rejection; readers now return None/null/undefined/Ok(None) for any
value other than 0x00/0x01. Reproduced fail-first in the Scala and TypeScript
stub suites (BaboonTypeMetaCodecSpec, TypeMetaFlagByte.test.ts), verified in
Python directly.
The TypeScript and Rust generated facade metadata registries emitted degenerate
own-version tables (sameInVersions = [own], forward = {own: identical}); they
now carry the real per-type sameIn / forward-readable tables like the other
backends (unknown type ids resolve to empty rather than fabricated data).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…TextTree The TypeScript and Rust domain-facade generators and the GraphQL SDL translator assembled generated code with StringBuilder appends and shipped it through TextTree.verbatim / OutputFile(String). Every other emitter in the compiler builds a TextTree; these three now do too (q-interpolated templates, joinN/joinNN, shift/trim), with per-version, registration and type-definition pieces as composable subtrees instead of imperative append sequences. GraphQL doc comments and the BaboonAny scalar description are interpolated as verbatim nodes: plain String interpolation is escape-processed at render time and stripMargin would strip `|` margins from user text. Generated output is unchanged: schema.graphql files are byte-identical for the whole test model directory, and the 36 TS + 36 Rust facades differ only in the trailing newline, which now matches every other TextTree-rendered file. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…aders decode newer payloads with their newest codec The v1 binary envelope has one bound slot, domainVersionMinCompat, and its layout is frozen (a new field, a flag bit or a trailer would all be a format change). Forward reads for UEBA therefore ride on the VALUE of that slot, chosen by the writer through ForwardWritePolicy on BaboonCodecContext, next to the index mode the bound depends on: - Strict (default): the byte-identical bound, exactly as before. - Tolerant: the prefix-read bound from baboonMinReaderVersions for the payload's index mode (prefix-compact for compact, prefix-any-mode for indexed). Equal to the Strict bound when no prefix relationship exists, so such envelopes stay byte-identical. Readers in all nine runtimes now trust the bound: a payload from a version newer than every registered one is decoded with the reader's newest codec as soon as the bound reaches a registered version, instead of with the bound version's codec via the sameIn scan. Readability is monotone along the chain, so the newest codec is correct under both bound semantics and loses the fewest fields. Reproduced fail-first over a new three-version shared fixture (fwd-e2e-chain-ok): a reader registering 1.0.0 and 2.0.0 decoded a 3.0.0 envelope bound at 1.0.0 as ChainAppend(1) and dropped the 2.0.0 field. The cost is documented in codec-envelope.md §2.1.2 and forward-compat.md: a binary reader cannot tell a Tolerant envelope from a byte-identical one, so ForwardReadPolicy.Lossless has no effect on binary reads and re-encoding intermediaries must run at the writer's version or newer. The Rust runtime crossed the JVM 64KB embedded-constant limit with this change; BaboonTypeMeta and its wire codec moved to baboon_type_meta.rs and are re-exported from baboon_codecs_facade.rs so every existing path still resolves (CLAUDE.md records the symptom and the fix). E2E coverage: ForwardCompatBinEnvelopeSpec (Scala stub) and ForwardCompatBinEnvelope.test.ts (TS stub) — Strict unchanged/refused, Tolerant compact decoded by an old reader, json-additive/identical/grown-enum envelopes identical to Strict, indexed context not lowering a var-len append, and the three-version chain decoded with the mid reader's newest codec. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…mbination Adds a "Worked examples: what changed on the wire" section: the three knobs (writer ForwardWritePolicy, index mode, reader ForwardReadPolicy) and the reader-rule change; per-type writer bounds for the fwd-e2e fixtures; the real JSON envelopes with both reader policies' outcomes; annotated UEBA bytes for Strict vs Tolerant; the full value x context matrix as seen by a 1.0.0-only reader; the three-version chain before/after the newest-codec rule; and a summary matrix. All envelopes and outcomes were captured from the generated Scala stub. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…very mode Reworks the worked-examples section around the fixture types themselves: each of FwdAppendVar, FwdMidInsert, FwdStable, FwdEnumHost and the three-version ChainAppend is shown as its .baboon source across versions, its per-tier bounds, the JSON envelope with both reader policies' outcomes, and the UEBA envelope under Strict/Tolerant x compact/indexed with annotated bytes and the older reader's result. All envelopes and outcomes were captured from the generated Scala stub. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…rs accept v1 and v2, writers default to v1
The v1 binary envelope has one bound slot, so a v1 reader cannot tell a
byte-identical read from a prefix read. metaVersion 2 is the JSON envelope's
field set in binary form:
02 | domainId | domainVersion | flags | [minCompat] | [readableMin] | typeId | payload
flags bit 0: minCompat follows (elided when == domainVersion); bit 1:
readableMin follows (elided when == effective minCompat); any other bit is
rejected. minCompat is always the byte-identical bound and readableMin the
prefix bound for the payload's index mode, so ForwardWritePolicy is
irrelevant under v2 and the reader's ForwardReadPolicy applies to binary
exactly as it does to JSON: Tolerant decodes with the newest codec once
readableMin reaches a registered version, Lossless requires minCompat to.
The layout is selected per encode through BaboonCodecContext.envelopeVersion
(V1 default, V2 opt-in) in all ten runtime directories; the fully specified
context constructor gained the parameter. writeBin dispatches on
meta.metaVersion and fails fast on anything else; readMeta accepts 1 and 2 and
rejects other versions and unknown v2 flag bits. Binary getCodec now passes
the reader's policy through; v1 metas carry readableMin == minCompat, so their
behaviour is unchanged.
Reproduced fail-first in the Scala and TypeScript stubs: a hand-assembled v2
envelope was rejected by readMeta ("v2 envelope must be readable"). New specs
BinEnvelopeV2Spec / BinEnvelopeV2.test.ts cover the reader, the byte-identical
default, per-type flags and bounds in both index modes, round trips through
the writer and old readers under each policy, the three-version chain with
Lossless now enforceable for binary, and rejection of unknown flag bits and
metaVersions.
baboon_runtime.swift crossed the JVM 64KB embedded-constant limit;
BaboonVersion/BaboonDomainVersion/BaboonTypeMeta/BaboonTypeMetaCodec moved to
baboon_type_meta.swift in the same module (translator emits it; CLAUDE.md
records the pattern).
Spec: codec-envelope.md §1, §2.1.3 (new), §3 (byte 2 active), §5, §6.
forward-compat.md: v2 section, knobs, per-type v2 examples, summary matrix.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Adds the captured envelope-v2 bytes for FwdMidInsert and FwdEnumHost (flags 0: the v1 bytes with 02 in front) so the per-type walk-through covers v2 for all five fixture types, and explains how the v2 blocks relate to the shared v1 header. All v2 examples were re-captured from the generated Scala stub and match the bytes asserted by BinEnvelopeV2Spec. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… sameIn dyn and Python multi-version facade A byte-for-byte check of the binary envelope across backends (same values, same contexts) found two asymmetries the per-language suites had not: - Rust: the generated `baboon_same_in_versions_dyn` returned `[own version]` for every type while the Meta struct carried the real sameIn table. The envelope's byte-identical bound is the head of that run, so Rust-written envelopes of unchanged types (v1 and v2) elided the bound every other backend publishes — an older reader in any language refused them. The dyn impl now emits the real run (RsBaboonTranslator). - Python: `_register_version` sorted by `v.version.version`, an attribute that does not exist, so no domain facade with more than one version could be constructed (no runtime test built one). Sort by `v.version`. Every backend that had no envelope test — C#, Kotlin, KMP, Java, Python, Dart, Swift, Rust — gains a golden-bytes suite asserting the exact sequences from docs/forward-compat.md (v1 Strict, v1 Tolerant, v2 compact and indexed for FwdAppendVar; FwdStable v1 and v2; FwdEnumHost v2; the three-version chain v2), the v1/Strict default of the built-in contexts, and a v2 round trip through the writer's own facade. Scala and TypeScript already assert the same sequences structurally. Swift: test/sw-stub/Package.swift is hand-written and had no targets for the fwde2e.fwd / fwde2e.chain fixtures, so the Swift lane never compiled or tested them (SwiftPM only warns about unowned Sources/ dirs). Targets, per-module test targets and the RuntimeTests dependencies are added. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Adds §2.1.4 to codec-envelope.md, the v2 counterpart of the v1 conformance block: annotated canonical bytes for FwdAppendVar (flags bit 1, readableMin present) and FwdStable (flags bit 0, minCompat present), byte counts against the v1 form, and pointers to the golden-bytes suites that pin them in every backend. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
forward-compat.md gains a "Known gaps" section: no runtime e2e for prefix-any-mode (and why), non-uniform missing-tier handling across runtimes (and why it is unreachable for generated types), and the timestamp kind-byte round-trip flake tracked as #91. CLAUDE.md records that test/sw-stub/Package.swift is hand-written and silently skips fixture domains until their targets are added. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ral fallback in the converter round-trip test Missing-tier handling: Python, Swift and Rust fell back to the byte-identical bound when baboonMinReaderVersions lacked the json-additive or prefix tier, because their base types gave the member a default; the other seven runtimes failed fast. The defaults are removed (abstract property / no protocol-extension default / required trait method), the writers raise an encoder failure, the Rust facade's type-meta construction returns Result instead of a bare value, and the hand-written test fakes in the Swift and Rust stubs now provide the four tiers. Fail-first tests: test_min_reader_tiers_fail_fast.py, MinReaderTiersFailFastTests.swift, min_reader_tiers_fail_fast_tests.rs. Timestamp kind byte (#91): the C# RpDateTime intentionally carries its DateTimeKind on the wire and the Scala/JVM side is left unchanged, so a tso/tsu is not byte-stable across writers by design. RTCodecTest therefore falls back to a structural, equal-length comparison when the re-encoded bytes differ, which is the property the round trip actually promises. Python wrote 2 (Local) for every non-zero offset; it now writes 0 like the other non-.NET runtimes (test_timestamp_kind_byte.py). docs/ueba-format.md defines the byte. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Address the backend/runtime review across all targets with shared semantic plans, backend-local scalar and service emitters, and runtime-owned envelope and MCP helpers. Fix renamed ADT branch evolution and source/target conversion validation, preserving custom validator contracts. Add behavioral regressions, compatibility coverage, and the implementation ledger. Verified the full 205-action CI matrix, 200 serialization checks, and 162 RPC checks; repeated clean JVM/JS compilation, compiler tests, and native build before commit.
…clared renames are UEBA-readable UEBA identifies a field, an enum member or an ADT branch by position and never writes its name; JSON identifies them by name and is blind to position. The two formats therefore forgive disjoint sets of changes and neither contains the other, so the single linear tier chain could not describe a step: a declared rename is byte-identical in UEBA and broken in JSON, which is the exact mirror of a mid-position insert. The chain collapsed every rename to "unreadable", discarding a legal UEBA forward read. A step (and a chain, by meet) now carries a ForwardGuarantee: an independent UEBA axis (Full | PrefixAnyMode | PrefixCompact | absent) and a JSON flag. Dependencies propagate per axis too, since a nested value must be UEBA byte-identical to keep the outer sequential read in sync but only JSON-readable to keep the outer JSON read alive. `baboonMinReaderVersions` keeps its four capability keys and each is now resolved from its own axis, so a rename lowers `prefix-compact` and `prefix-any-mode` and leaves `json-additive` alone, while a mid-position insert does the reverse. `identical` still means byte-identical in both formats and still equals the type's sameIn head. No wire change was needed. A rename sits at the strongest UEBA tier, and both the v1 and v2 binary envelopes already carry a bound the writer computes per format. Only `baboonForwardReadable` gains values: `ueba-identical`, `ueba-prefix-any-mode`, `ueba-prefix-compact`, where the prefix means the guarantee is UEBA-only. That map is informational; no runtime reads it. Renames are taken from the declared `was` annotation, never inferred, so the claim is semantics-preserving by construction. `prevName` survives into later versions, so it is honoured only while it still names a field of the version being compared; without that filter a type carried its rename forever and every later step was misclassified. A new field that takes a name over from a different field via `was` also disqualifies the JSON axis, which closes a pre-existing hole where a declared name swap was reported JSON-readable while handing the old reader the wrong value. ADT branch renames and type renames remain unclassified: they change the TypeId, so the type drops out of the version intersection and out of its dependents' dependency sets. Recorded under known gaps. Coverage: ForwardCompatComparatorTest gains two cases over new isolated fixtures (rename, rename-then-append, enum member rename, and hosts of both) plus the per-format minReaders bounds; ForwardCompatRenameSpec proves it end to end over a new shared model, including that the UEBA payload matches the 1.0.0 layout byte for byte and that JSON refuses under either read policy. Swift Package.swift gains the targets for the new fixture domain. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A `was` annotation survives into every later version of a type, while the comparator only ever diffs adjacent version pairs. Two defects followed. 1. A carried-forward annotation was rejected as a typo. After `r: str was b` in 1.1.0, touching the type again in 1.2.0 failed with `InvalidFieldRename` because 1.1.0 no longer has a `b`. The per-pair check is gone; `evolve` now validates ancestry once per package against every earlier version of the type's lineage (following `Domain.renames`), and each pair honours a `prevName` only while it still names a member of the version it compares against. 2. A rename whose target name the previous version also used was mishandled. `keptMembers = names1.intersect(names2)` did not exclude renamed names, so a declared name swap was simultaneously "kept" and "renamed" and produced two conflicting ops for one target field; `BaboonValidator`'s `RemovedDtoFields` check had the same defect from the other side. Worse, a swap that preserves field order leaves both `shallowId` (sorted) and `deepId` (positional) intact, so the type was classified unchanged and `BaboonRules` took the transfer-everything fast path — the declared move was ignored outright. `hasEffectiveRename` now forces such a type into `shallowModified`. That fast path also built its ops from the SOURCE fields. `Field` equality includes `prevName`, so dropping a stale annotation left every target field unmapped and crashed all nine conversion translators with a `NoSuchElementException` at `ops(f)`. It builds from the target defn now. `RenameSoundnessTest` covers all of it: four cases fail with the comparator and validator reverted, and the annotation-dropped case fails with only `BaboonRules` reverted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…d types Follow-up to 2b016c2, which fixed the field and enum-member arms and left `diffAdts` alone on the mistaken reasoning that a branch-level rename could not reach the defective code. Both shapes are reachable (#92). 1. A branch rename onto a name the previous version also used hid the removal of the branch that used to carry that name: `removed = members1 \ members2 \ renamedOld` cancels out, `AdtBranchRemoved` never fired, and an old value of the dropped branch was silently converted into the renamed one. Both `diffAdts` arms now subtract the rename's two ends before intersecting, as `diffDtos`/`diffEnums` already do. The type-level kept/added/removed sets had the same defect one level up, which additionally derived a conversion from the old type onto the id the rename took over; they get the same treatment. 2. A declared swap of two names cannot be honoured at all -- both declared sources keep their own TypeIds in the new version, so there is nothing to map one onto the other and the classification would compare each id against its own namesake. It was previously discarded without a diagnostic; it now fails with `EvolutionIssue.RenameSourceStillPresent`, reported per direction. This applies to any declared type rename whose source is still defined. 3. A `was` clause naming something that never existed is now an error at every level, not only for fields and enum members: a type-level `was[Other]` naming a type no earlier version declares, and a field clause on a type introduced by the version carrying it, both fail rather than being discarded in silence. Two exemptions, each forced by observed behaviour rather than convenience: - The oldest version of a package. `BaboonSchemeRenderer` emits one version at a time and the rendered source must load back (`SchemeRoundtripTest`), so a lone version containing a rename has to stay legal -- and with no earlier version the clause makes no checkable claim. `RenameSoundnessTest` now pins this directly instead of relying on the renderer to catch a regression. - An ancestor that was only ever an excluded declaration. A type no root reaches is dropped from `defs` and survives only as an id in `excludedIds`, so its members are not observable; `m20-was-propagation` renames a branch of an `adt` that `root adt Outer` absorbs structurally. Existence is therefore judged against declared ids (retained plus excluded) while member names are judged only against shapes some version actually retained. `RenameSoundnessTest` covers all of it (15 cases); the two ADT cases fail with the comparator reverted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The documented gap said a type rename collapsed both forward-readability axes
for the renamed type and its dependents. Measuring it turned up something
larger: it also broke CONVERSION DERIVATION. With `Leaf` renamed to `Renamed`
and `Host { l: Leaf }` keeping its own name, the field read as an incompatible
type change, so `Host` needed a hand-written conversion -- while the compiler
emitted the `Leaf -> Renamed` conversion it needed right next to it. No fixture
caught this because `conv-test` renames the holder too, degrading the step to
remove+add.
One root cause: `TypeRef` equality was not rename-aware across a step. Neither
wire format puts a nested value's type name on the wire (UEBA is positional;
JSON tags a DTO field by the FIELD name), verified in the generators rather than
assumed, so a reference to a renamed type denotes the same bytes as the
reference that replaced it.
Forward-compat:
- `stepForwardGuarantees` pairs a renamed type with its counterpart and keys the
step by the OLD id, so hosts resolve it as a dependency. The renamed type still
gets no bound of its own -- `computeForwardReadable` finds no successor under
that id -- which is the right answer while the envelope identifies a payload by
typeId and no runtime resolves a renamed one.
- `ownForwardGuarantee` compares field types modulo renames. For an ADT the axes
part company: renames resolve on the UEBA side (branch INDEX) and deliberately
not on the JSON side (branch NAME), so a branch rename is `ueba-identical`.
Conversions:
- `DtoOp.KeepField` and `FieldOp.Transfer` carry `sourceTpe`, the field's type as
the old version spells it; it differs from the target only for rename-affected
fields, so every existing path is unchanged. All nine translators already had a
source-type parameter -- C# carried a comment noting the hole ("no `oldTpe` for
a plain `Transfer` op") -- and each needed one line.
Adding the field does not break translator consumers at compile time, so a missed
backend would have emitted a bad old-type reference silently. `rename-nested-ok`
therefore goes in the shared model-dir (with the hand-written Swift targets) so
all nine codegen lanes compile it.
Also fixes an unrelated Python defect the fixture exposed: conversion locals were
annotated with the TARGET field's type resolved in the SOURCE domain, naming the
old version's class for a new-version value in every nested user-typed field, and
crashing outright once that type was a rename target.
Full gate `mdl --seq :build :test` green, 200/200 lanes; JVM suite 877/877.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… tests Three follow-ups to 65c4513. (A) `BaboonSchemeRenderer` dropped type-level renames. It emits field- and enum-member-level `was`, but a TYPE rename lives in `Domain.renames` rather than on the member, and nothing read it -- so rendering a version turned a rename into a remove+add for anyone who fed the scheme back in, changing that model's conversions and forward-compat. `SchemeRoundtripTest` could not catch it: it compares two renders of the same model, so a field both of them omit stays invisible. `renderClauses` now emits `was[...]` ahead of the derivations, and the new test asserts the clause survives AND that the reloaded model carries the rename. Fail-first verified by reverting the renderer alone. The same function turned up a second hole: `foreignEnclosed` parses the same `: ...` clause list as every other type header, but `renderForeign` never emitted one, so a `foreign` lost its `derived[...]` too. Same fix. (B) `FieldOp.Transfer.sourceTpe` can be ignored by a consumer without any compile error. Pinned at the model level: the transfer op must address the source by its OLD spelling and build the target with the NEW one. The translator half -- each backend actually using it -- stays guarded by the nine codegen lanes compiling `rename-nested-ok`, which is a complete guard for this class: a renamed type's new name never exists in the old version, so ignoring `sourceTpe` always yields a dangling reference that fails to compile. A nine-backend emission test would only cut detection latency, at the cost of duplicating the per-target child-injector wiring that lives in `Baboon.processTarget`. (C) A rename mid-chain was reasoned about but never run. With the rename at step 2 of 3, a 1.0.0 reader must still read 1.1.0's `Leaf` -- the typeId is unchanged there -- and must not claim 1.2.0, while the host spans the whole chain. Now pinned rather than inferred from the keying. JVM suite 880/880; full gate `mdl --seq :build :test` green, 200/200 lanes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…liberate non-goal Links the decision record (#94) and corrects the runtime count (nine, not ten -- my own inconsistency, every other reference in the file says nine). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…, not by threading
The per-domain subcontexts were already in place and every submodule-bound
component injected `Domain`/`BaboonEvolution` correctly. The threading was one
layer down: those components handed the CURRENT domain to a stateless outer
helper on every call, because `*TypeTranslator` is bound outside the subcontext
and takes `(domain, evo)` per method. 152 call sites in subcontext-scoped code
passed a domain the caller had already been injected with.
Two facts, both checked rather than assumed, make the current domain safe to
resolve from the subcontext:
- The outer per-family users (`*BaboonTranslator`, `*McpServerGenerator`) call
only `to*Pkg`-style methods, which take no `Domain`. Nothing outside a domain
scope needs the domain-taking API.
- The genuine exceptions are the source-side renders in conversions -- `srcDom`
(16 sites) and `higherDom` (1) -- already distinguished by `@Id("source")` vs
`@Id("current")`. Those keep the explicit-domain API unchanged.
So each backend gains a `*DomainTypes` facade bound INSIDE its
`makeSubcontext(...).withSubmodule`, closing over the injected current `Domain`
and `BaboonEvolution` and forwarding to the untouched translator. Domain-free
helpers (`escapeSwiftKeyword`, `toSnakeCase`, ...) stay where they are, so the
facade carries only what the subcontext actually makes ambient.
Across nine backends: ~413 call sites shed their two trailing arguments, and the
compiler then identified 12 now-unused `evo` constructor parameters and 5
`makeFixture(domain, evolution)` parameter pairs, all removed.
The nine `*ConversionTranslator.Factory` bindings deliberately stay factories:
conversions are built per source-version pair from outside any domain scope, so
a subcontext would turn five named arguments into five `.provide(...)` calls at
the same site -- no threading removed, and the `@Id` disambiguation lost.
Pure refactor, verified as such: generated output for all nine backends over the
shared model-dir is byte-identical to the pre-refactor baseline (4512 files),
checked both before and after the parameter cleanup. Full gate
`mdl --seq :build :test` green, 200/200 lanes; JVM suite 877 passing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ntext too Second pass over the same question, measuring CROSS-COMPONENT ambient passing -- a value handed from one DI component to another -- rather than every mention of `domain`. Threading between private methods of a single class is ordinary parameter passing, not a missing scope; a subcontext only helps when separate components need the same ambient value. That measurement found 121 remaining passes in four groups, and widening the ambient set to `domain.version` / `domain.id` is what surfaced the most interesting one. - `DomainEnquiries` (new, language-agnostic, bound in all nine submodules): `BaboonEnquiries` is shared by the typer, the validator and every backend, so it keeps its explicit-domain API; the facade fixes the current domain for the 57 generator-side calls to `isRecursiveTypedef`, `hasForeignType`, `isEnum`, `unfold` and `collectParents`. - `DomainEvolution` (new, likewise shared): `BaboonEvolution` is keyed by version because it describes a whole lineage, and inside a per-domain subcontext exactly one of those keys is ever meaningful. 20 sites. - `currentPkg` on each `*DomainTypes`, absorbing `toXxPkg(domain.id, domain.version, evo)`. 26 sites. - `toPyModule(tid, domain.version, evolution, pkgBase)`. 10 sites. - `translateServiceRt(domain)` loses its parameter in the eight backends that had one: both caller and callee are subcontext-scoped and both already inject the current domain, so the argument was passing a value to its own owner. TypeScript was already parameterless -- the proof it was removable. The payoff is larger than the call sites: a clean compile then reported 67 injected dependencies that had become entirely unused, pruned to fixpoint over two waves (59, then 8 more that only became dead once the first were gone). `trans`, `typeTranslator`, `enquiries`, `evo` and `evolution` disappear from constructors across all nine backends -- dependencies that existed only to be forwarded. Left alone deliberately: `*FileTools.basename(domain, evo)` (15 sites) would need nine facades for two calls each, or file-path naming folded into the type-rendering facade; `csTypeInfo` (5 sites, C# only) would couple `CSDomainTypes` to it for five calls; and a per-lineage subcontext would mean extracting nine new DI classes to remove ~45 private-method parameters. Pure refactor: generated output for all nine backends is byte-identical to the pre-refactor baseline (4512 files), re-verified after each batch and after the prune. Full gate green, 200/200 lanes; `RTCodecTest` 9/9 with the gate's artifacts present. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`build-macos-amd64-13` failed while every other job in the run passed -- and it
failed in POST-JOB CLEANUP, after build, tests, `:smoke` and the 35MB artifact
upload had all succeeded:
##[error]The template is not valid. sbt/setup-sbt/v1/action.yml (Line: 81, Col: 14):
hashFiles('**/*.sbt, **/*.properties') couldn't finish within 120 seconds
duration_ms=120219 -> conclusion=failure
The step is `sbt/setup-sbt`'s own "sbt 2.x disk cache", which caches
$HOME/Library/Caches/sbt -- a directory sbt 1.x never writes. We pin
sbt.version=1.11.7 (now 1.13.0), so that cache is empty by construction, and we
were paying a whole-workspace `hashFiles('**/*.sbt', '**/*.properties')` to
compute its key. By teardown the workspace holds every backend's generated code
plus node_modules, cargo target, .dart_tool and Swift .build, where `*.properties`
is abundant; the walk exceeds the 120s expression limit.
Why only this job: it is genuine Intel hardware (`os: macos-15-intel`,
RUNNER_ARCH=X64 -- not x64-under-Rosetta) and the slowest in the matrix at 2h42m
against 1h30m for macos-aarch64 and 1h19m for linux-amd64. The others walk the
same oversized tree inside the limit. So the fix is not Intel-specific in
principle, but this is the only job that trips it -- and the only job that uses
`sbt/setup-sbt` at all, since Linux gets sbt through nix.
Dependency caching is untouched: graalvm/setup-graalvm already does it with
`cache: 'sbt'` two steps earlier.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Latest stable 1.x (1.12.15 is the previous line). sbt 2.x is blocked on io.7mind.izumi.sbt:sbt-izumi, which publishes only for sbt 1 (_2.12_1.0) while the other three plugins already ship _sbt2_3 builds -- and it is load-bearing, supplying the whole scalacOptions set (including -Wconf:cat=other-match-analysis:error), the git-derived version macros and sbt-release, so there is nothing to work around locally. Verified: nix-pinned launcher fetches 1.13.0, `sbt +compile` passes, scalacOptions unchanged, full gate green 200/200. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
All 13 are same-major moves, so no API migration; verified by `sbt +compile` (cross-build JVM + Scala.js, which is where the stricter -Wconf lives), the JVM suite (877 passing) and the full gate (200/200 lanes, native-image included). scala 2.13.16 -> 2.13.18 izumi 1.2.24 -> 1.2.25 classgraph 4.8.181 -> 4.8.195 scala-parser-combinators 2.4.0 -> 2.5.0 scalatest 3.2.19 -> 3.2.20 circe 0.14.1 -> 0.14.16 magnolia1_2 1.1.10 -> 1.1.14 kind-projector 0.13.3 -> 0.13.4 case-app 2.1.0-M30 -> 2.1.0 (off a prerelease) sbt-native-packager 1.10.4 -> 1.12.0 sbt-izumi 0.0.101 -> 0.0.121 sbt-scalajs 1.19.0 -> 1.22.0 sbt-scalajs-crossproject 1.3.2 -> 1.4.0 Notes on the two riskiest: - kind-projector is `cross CrossVersion.full`, so it resolves per exact Scala version; kind-projector_2.13.18 0.13.4 was confirmed present before moving Scala. - sbt-native-packager backs `GraalVMNativeImagePlugin`, so the native-image build is the real check for it; the gate's `:build` lane passes. Generated output for all nine backends is byte-identical to the pre-bump baseline (4512 files) -- the check that matters for circe and magnolia, since those drive metadata JSON emission. Held back, as genuine majors to decide separately: - jline 3.26.3 -> 4.4.5: the explore shell's interactive path, which the suite exercises only thinly (ExplorerTest drives commands directly, not the JLine terminal), so a break could pass CI and surface only in `baboon explore`. - json-schema-validator 1.5.9 -> 3.0.7: test-only, so contained, but 1 -> 3. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both were held back in the previous commit as "genuine majors"; that was excess
caution rather than a finding. Tried properly, both land.
jline 3.26.3 -> 4.4.5: compiles with no source changes. Everything this build
touches is unchanged across the major -- TerminalBuilder, LineReaderBuilder,
LineReader.Option, Completer/Candidate/ParsedLine, EndOfFileException and
UserInterruptException. Since compiling says nothing about runtime linkage for a
major bump, the shell was exercised for real: `:explore` builds its terminal and
line reader, loads a domain, accepts a piped command and exits 0. That matters
because the suite drives explore COMMANDS directly (ExplorerTest) and never the
JLine terminal, so a linkage break here would otherwise have passed CI and
surfaced only for someone running `baboon explore`.
json-schema-validator 1.5.9 -> 3.0.7: this one did break, in one test file.
The API was renamed at 2.0, so there is no gentler intermediate (2.0.7 has the
new shape too):
JsonSchemaFactory.getInstance(SpecVersion.VersionFlag.V202012)
-> SchemaRegistry.withDefaultDialect(SpecificationVersion.DRAFT_2020_12)
SchemaValidatorsConfig.builder().build()
-> dropped; getSchema(String, InputFormat) uses the registry default
com.networknt.schema.JsonSchema -> com.networknt.schema.Schema
validate(str, InputFormat.JSON) -> unchanged, now returns List[Error]
`validate(String, InputFormat)` surviving is what keeps this cheap: 3.x moved to
Jackson 3 (`tools.jackson`), so had the string overload gone the test would have
needed Jackson 3 types of its own. As it is, Jackson 3 only arrives
transitively and no test code touches it.
The migration is not vacuous: the file's `assertRejects` case (an instance
missing a required field must produce validation messages) still passes, so the
validator remains live and discriminating rather than silently accepting.
JVM suite 880/880 with zero cancellations; full gate green 200/200.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
My dependency-bump commits were incomplete. `deps.lock.json` is a tracked
lockfile of resolved sbt dependencies, generated by `squish-lockfile` and
consumed by flake.nix (`mkCoursierCache { lockfilePath = ./deps.lock.json; }`).
It pinned every version those commits moved -- izumi 1.2.24 x64, circe 0.14.1
x24, scala 2.13.16 x14, plus classgraph, jline, json-schema-validator and
case-app -- so the nix build path could no longer resolve the dependency set.
That is what failed `check-flake-runs-linux-aarch64` and fail-fast-cancelled the
rest of the run for b0d7b0b. `mdl :build :test` does not exercise the nix path,
so no number of green local gates would have caught it; the build even has
`refreshFlakeTask` wired up for exactly this, which I should have reached for
when changing versions.
Regenerated with `squish-lockfile ./lockfile-config.json` (657 artifacts;
+1026/-1178 lines) and verified with the CI job's own command:
nix run . -- --model-dir ./test/conv-test :cs --output ... :scala --output ...
which exits 0 and emits 198 files, i.e. baboon builds from the lockfile and runs.
One residual `2.13.16` is correct and stays: scala-reflect-2.13.16.pom, a
transitive POM read during resolution, not a stale pin.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…e signatures
With `deduplicate` on (the CLI default), a type whose schema is unchanged
across versions is emitted once in the latest namespace and references from
older namespaces are rewritten to the surviving twin. That rewrite happens in
`CSBaboonTranslator.renderType`, i.e. in the final `mapRender` over the
`CSValue` nodes of the emitted tree.
Services are never deduplicated, so an unchanged service is re-emitted per
version — and its method signature mixed the two worlds. The parameter stayed
a tree and was rewritten correctly, but the return type was flattened to a
fully-qualified string (`csFqName` in `CSDefnTranslator`, `renderFq` in
`CSServiceWiringTranslator`) before the renderer ran, so it kept the obsolete
version's namespace and named a type no file declares:
public Task<Either<Repro.Svc.v1_0_0.Svc.M.Err,Repro.Svc.v1_0_0.Svc.M.Out>>
M(Repro.Svc.Svc.M.In arg); // CS0234
The obsolete wiring's result declarations had the same defect. The trigger is
the method I/O types being unchanged, not the service itself: a service that
gains a method while an existing method's inline types stay identical breaks
the same way.
The flattening is there because the result container is a user-configurable
`$error`/`$success` pattern string. `ResolvedServiceResult.renderReturnTypeTree`
and `expandPattern` now splice that pattern around the argument trees instead
of around rendered strings, so the type nodes survive to the renderer. Call
sites pass `.fullyQualified` trees, which keeps the rendered shape identical
for every type that is not deduplicated away — verified byte-identical output
for the whole shared model-dir matrix in the default and the errors/async
configurations.
C#-only: `deduplicate` has no consumer outside `CSTypeInfo`, and the other
seven backends keep the string-based `renderReturnType`.
Regression test: `CSharpDedupServiceSignatureTest` over the new isolated
fixture `baboon-compiler/src/test/resources/dedup-service-ok/` (outside the
shared model-dir, per the D9 note in CLAUDE.md).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`BaboonCodecsFacade.IsDeprecated` is the predicate behind the `Deprecated` arm of `ConvertClassified`: it answers whether a chain of registered conversions leads from a stored value's type to the latest version. It did not walk the chain. For every newer registered version it looked for a conversion whose `TypeFrom` is the ORIGINAL value's type and never moved to the type each step produces, so in a three-version lineage a 1.0.0 value matched at the 2.0.0 step and missed at the 3.0.0 step (whose registry holds `v2_0_0.X -> X`). The method returned true while `Convert` succeeded, and callers saw `Deprecated` for a perfectly convertible value. Advancing the type is necessary but not sufficient. Deduplicated codegen emits conversions that reach the latest CLASS several versions early — `Convert__T1_D2__From__1_0_0` in `testpkg.pkg0` is registered under 2.0.0 yet targets the 3.0.0 class, and 3.0.0 registers nothing for that type — and a conversion's `VersionTo()` is the registry's version, not the produced class's, so a strict per-version lockstep still reports a live type as deprecated. The walk is now reachability over the registered conversion graph following each conversion's `TypeTo`, with a visited set. A type that implements `IBaboonGeneratedLatest`, or one that a conversion of the newest registered version accepts, completes the path; the latter preserves the previous answer for a partially registered facade. Converters are still never executed. `AbstractBaboonConversions.FindConversions(Type)` is the type-keyed overload the walk needs, since it holds no instance of the intermediate types. C#-only: no other backend runtime has `IsDeprecated`. Regression test: `test/cs-stub/BaboonTests/FacadeDeprecationTests.cs` — real generated `fwde2e.chain` codegen for the conversion-at-every-step shape, plus synthetic registries for the reach-latest-early, ADT-member and dead-end shapes. Their converters throw from `DoConvert`, which pins the "without executing converters" half of the contract. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ode and cross-format helpers
`BaboonCodecContext` carries `useIndices`, `forwardWritePolicy`, `envelopeVersion` and
`facade`. The first three are UEBA-only for encoding — `BaboonTypeMeta.from` computes
`$rv` unconditionally, so JSON needs no policy input. `facade` is not: the `any` JSON
encoder needs it whenever a field holds an `AnyOpaqueUeba`, because that payload has to
be transcoded through `facade.uebaToJson`.
`encodeToJson` hardcoded a facade-less context, so encoding such a value failed with
Cannot encode AnyOpaqueUeba into JSON without a facade reference.
Pass BaboonCodecContext.WithFacade(useIndices, facade) into Encode(),
or supply AnyOpaqueJson directly.
— advice the signature made impossible to follow, while the sibling `encodeToBin(ctx, …)`
in the same class was configurable. Kotlin and Kotlin-KMP alone already took a context
here; the other eight runtimes were inconsistent with Kotlin and with their own binary
path. The existing `AnyRoundTrip*` cross-format suites call the generated codec directly
with `withFacade`, never the facade, which is why this was never caught.
`jsonToUebaBytes` / `uebaToJson` had the same hole in all ten runtimes — hardcoded
`Compact` on both legs — so a payload whose content carries a nested `any` in the
opposite wire form failed one level down, and `jsonToUebaBytes` could never emit
indexed UEBA. Those are the very methods `ctx.facade` dispatches into, so this is the
recursion step of the same defect.
All three entry points now take a codec context and use it, and the `any` runtime codecs
pass their own `ctx` into the facade call so nested transcoding inherits the facade.
BREAKING CHANGE: the context is a required leading parameter on `encodeToJson`
(plus Scala's `encodeToJsonString` and Rust's `encode_to_json_with_override` /
`_with_declared_trait`), `jsonToUebaBytes` and `uebaToJson`. No ctx-less overload is
kept: `encodeToBin` already forces the parameter, and an implicit default is exactly
the footgun that hid this. Call sites pass `Compact` unless they need the facade.
Also corrects the Python facade's `ctx: str` annotations on `encode_to_bin` and
`_encode_to_bin_stream` — the value is a `BaboonCodecContext`, as the private
`_bin_type_meta` already declared and as `ctx.use_indices` requires.
Decode entry points still pin `Compact` deliberately: the UEBA reader takes indexing
from the wire header, `IndexElementsCount` is a generated constant, and the `any`
decoders take no context, so no facade is needed to read. Recorded as a note in the
ledger rather than changed.
Rust's JSON side still refuses `AnyOpaque::Ueba` outright — a serde-side limitation
predating this change, left alone.
Regression tests: `EncodeToJson_WithFacadeContext_TranscodesUebaAnyPayloads` and
`EncodeToJson_WithoutFacadeContext_StillReportsTheMissingFacade` in
`test/cs-stub/BaboonTests/AnyRoundTripTests.cs`. In the other backends the signature
change is itself the guard — a facade-less JSON encode no longer compiles.
Verified with `mdl --seq :build :test` (200/200).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…s can transcode
Follow-up to the facade context threading. Rust is the only backend whose JSON codec is
not generated: `RsJsonCodecGenerator` emitted serde helpers only, and the dyn adapter read
fn encode_json_dyn(&self, _ctx: &BaboonCodecContext, value: &dyn BaboonGeneratedDyn)
-> ... { serde_json::to_value(v) }
so the context arrived and died there. `serde::Serialize::serialize(&self, S)` has nowhere
to put one, which means `Serialize for AnyOpaque` can reach neither the facade nor the
per-field static fallbacks and errors out on the `Ueba` branch. Reproduced against the
`any-ok` fixture: `facade.encode_to_json` failed identically with and without a facade in
the context.
A scoped thread-local carrying the facade would have been ~40 lines and no codegen, and it
was rejected: `decode_any_field` yields the WIRE meta, which for variants B/C/D is
deliberately incomplete, so an ambient facade covers variant A only and fails the realistic
decode-UEBA-then-encode-JSON path for the other five — with no kind validation either. The
statics live at the field site, so the fix has to live there too.
Every generated type now gets a context-carrying inherent encoder, mirroring the UEBA
side's `encode_ueba(ctx, writer)`:
pub fn encode_json(&self, ctx: &BaboonCodecContext)
-> Result<serde_json::Value, BaboonCodecError>
Any-bearing types first rewrite their `any` slots to the JSON branch through the new
`any_field_codec::resolve_any_field_for_json` — which checks the declared kind byte and
passes the same static-fallback table `RsUEBACodecGenerator.anyStaticFallbacks` computes —
and then hand the value to serde unchanged. Renames, hex bytes, decimal-as-number,
timestamps, user map-key adapters and ADT wrapping are therefore still produced by the
derive and cannot drift. Types with no `any` below them emit a one-line
`serde_json::to_value` passthrough. Containment is a least fixpoint over the typespace,
iterative rather than recursive because recursive DTOs make the reference graph cyclic.
`to_json` / `to_json_value` stay context-less and keep refusing UEBA-form payloads: they
have no facade to reach, and silently emitting bytes would break cross-language readers.
Rust's JSON side still cannot be driven from `serde_json::to_value` for such values — that
remains the documented raw-serde behaviour, pinned by the existing fail-fast test.
Tests: `test/rs-stub/tests/facade_json_any_ctx_tests.rs` covers all six DSL variants plus
the opt/lst/map-value nested positions transcoding through the facade, the facade-less
error, the kind-mismatch rejection, and a drift guard asserting `encode_json` is
byte-identical to `serde_json::to_value` wherever no transcoding happens.
Verified with `mdl --seq :ci` — 206/206, cross-language acceptance matrix included.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…de derive
First step of replacing Rust's derive-based JSON codec with generated ones. The encoder
is converted here; the decoder, the removal of the derives and the `serde` dependency
drop follow. The derives stay in place for now, which is what makes the step verifiable:
while both exist, the derive is the specification and the new encoder must agree with it
exactly.
`RsJsonCodecGenerator` now emits a real field-by-field `encode_json(ctx)` for any-bearing
types — DTO fields keyed by wire name, ADT branches as `{"Branch": ...}`, recursion
through `opt`/`lst`/`set`/`map`. Plain leaves keep calling `serde_json::to_value`: that is
what the derive did, so it is exact by construction, and it names no serde trait, so it
survives dropping the direct dependency. Shapes serde expressed through field attributes
(hex bytes, decimal-as-number, `tsu`/`tso`) are written by matching `json_tools` helpers.
Types with no `any` below them keep a one-line passthrough.
Supporting runtime:
- `baboon_runtime::json_tools` — leaf conversions with typed errors: object/array/
string/bool accessors, the lenient i64/u64 readers reproducing the `lenient_numeric`
attribute, upper-case hex bytes, decimal via `normalize()` + `Number::from_str`, and
timestamps through the existing `time_formats` free functions. No serde.
- `any_field_codec::any_to_json` replaces the interim `resolve_any_field_for_json`:
it writes the `any` envelope directly (kind check, static fallbacks, facade transcode
for a UEBA-form payload) instead of rewriting the slot and deferring to serde.
The oracle is the point of this commit. `RsCodecTestsTranslator` emits
`test_<type>_json_encode_matches_derive` for every type with a JSON codec — 145 of them —
asserting `encode_json(Compact) == serde_json::to_value(value)` over the existing random
fixtures. rs-stub goes 604 -> 749 tests, all green.
Verified the oracle is not vacuous: renaming one emitted field key to `fAnyBROKEN` failed
exactly `my::ok::holder_tests::test_holder_json_encode_matches_derive` and left the other
144 passing.
Verified with `mdl --seq :build :test-gen-regular-adt :test-rust-regular
:test-gen-wrapped-adt :test-rust-wrapped`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The explicit-encoder commit hand-rolled a second least-fixpoint over the typespace to decide which types carry an `any`. `RsFieldRepresentation.containsAny` already computes exactly that — same iterative fixpoint, cycle-safe, ADT members followed through `dataMembers` — and already drives boxing and the Ord/Eq derive decisions. Dropped the duplicate and took the ctor deps needed to construct the shared helper. No behaviour change: rs-stub stays at 749 passing, the 145 encoder-vs-derive differentials included. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Reported during a consumer migration: every generated TypeScript MCP server fails to
compile under `noImplicitOverride`, which Deno enables by default.
TS4114: This member must have an 'override' modifier because it overrides a member
in the base class 'AbstractAsyncBaboonMcpServer<Ctx>'.
Both runtime base classes define `findTool` concretely (`BaboonMcpRuntime.ts:241` for the
sync server, `:382` for the async one), while the generator emitted the derived member
without the modifier. The emitter is shared by both variants, so both were affected.
Reproduced by generating the `mcp-stub-ok` server and running `tsc` with
`noImplicitOverride`: exactly one TS4114 at the generated `findTool`, in each variant.
Not a regression from the recent facade work — the concrete base implementations predate
it.
`test/ts-stub/tsconfig.json` now sets `noImplicitOverride: true`. Its absence is the
reason CI never saw this: the stub compiled with `strict` but not the flag the consumer's
toolchain applies, so the generated code type-checked here and failed there.
Also records the second report from that migration in the defect ledger as
MIGRATION-2026-09-24-D02, status open: i64/u64 have no consistent JSON representation
across backends, TypeScript's decoder silently rounds a numeric token beyond
Number.MAX_SAFE_INTEGER, and the generated cross-language JSON test cannot detect either
because it asserts TypeScript-side self-consistency rather than fidelity to the producer.
Fixing the reader alone makes the existing C#-to-TypeScript interop fail loudly rather
than quietly, so it waits on a wire-contract decision; the prototype and its test are
held outside the tree and referenced from the ledger entry.
Verified with `mdl --seq :build :test-gen-ts-mcp :test-ts-mcp :test-gen-regular-adt
:test-typescript-regular`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…de derive
Second half of step 1. `decode_json(ctx, wire)` now exists alongside `encode_json(ctx)`
for every generated type, and both are held to the derive across the whole fixture
corpus. The derives are still in place — removing them is the next step, and these
oracles are what makes that safe.
Any-bearing types get an explicit walk:
- DTO: `expect_object`, then per-field binds. `opt` is read straight off the map,
because serde treats an ABSENT `Option` field as `None` rather than an error, so
absent and explicit-null must behave alike. Recursive fields are rebuilt with
`Box::new`.
- ADT: single-key envelope, arity-checked, dispatched by branch name with an explicit
unknown-branch error.
- Leaves: attribute-driven shapes (`bytes`, `f128`, `tsu`/`tso`) go through
`json_tools`; `i64`/`u64` use the lenient readers that reproduce the
`lenient_numeric` attribute; everything else calls `serde_json::from_value`, which
is what the derive did and is therefore exact. Foreign leaves keep the host type's
own impl, since Baboon cannot know their shape.
- Collections: `Vec` / `BTreeSet` / `BTreeMap`, map keys parsed back via `FromStr`.
`any_field_codec::any_from_json` reads an `any` envelope into the JSON branch and checks
the declared kind byte, which only the field site knows.
Oracles. The per-type `test_*_json_decode_matches_derive` compares the explicit decoder
against serde ON THE SAME WIRE rather than merely round-tripping the value: a shared
misreading cancels itself out under a round-trip, which is precisely how the TypeScript
i64 corruption stayed invisible (see MIGRATION-2026-09-24-D02). rs-stub goes 749 -> 894.
Seven behaviours escape any corpus oracle, because it can only feed back documents the
encoder produced, so `test/rs-stub/tests/explicit_json_codec_tests.rs` covers them
directly: absent optional -> None, explicit null -> None, unknown keys ignored, absent
required -> error naming the field, non-object document -> error, `any` kind mismatch
rejected, and decoded `any` slots arriving as the JSON branch. Each one also asserts the
explicit and derived decoders agree, so the expectations record the derive's actual
behaviour rather than an assumption about it.
Verified with `mdl --seq :build :test-gen-regular-adt :test-rust-regular
:test-gen-wrapped-adt :test-rust-wrapped`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…licit JSON codecs
Defect introduced by the explicit decoder: map keys were read with `(k).parse::<K>()`,
which compiles for exactly one of the three eligible user key kinds.
id type Display yes, FromStr no -- its reader is the free `parse_repr`
enum Display yes, FromStr yes
wrapper Display no, FromStr no -- the adapter peels it to the inner scalar
A DTO that is both any-bearing and carries an id- or wrapper-keyed map therefore produced
Rust that does not compile. No fixture combines those two, so all 894 corpus tests stayed
green; the gap is recorded as RSJSON-2026-09-24-N01.
Such fields now delegate to the `<key>_as_map_key` adapter module the definition
translator already emits, which is the only thing that knows how each kind converts. The
eligibility lookup moved out of `RsDefnTranslator` into a shared `RsMapKeyAdapter` so both
emitters read the same answer — the same duplication trap as the earlier `containsAny`
fixpoint. Enum and builtin keys keep the native path, where serde handles them directly.
Found by building the combination as a purpose-made model and compiling the generated
crate rather than reading the output. That also caught a second mistake in the first
attempt: the value was wrapped in `IntoDeserializer`, but `serde_json::Value` is itself a
`Deserializer`.
Verified with `mdl --seq :build :test-gen-regular-adt :test-rust-regular
:test-gen-wrapped-adt :test-rust-wrapped`, plus the probe crate compiling clean with
id-keyed and wrapper-keyed maps on adapters and enum-keyed maps on the native path.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…key adapters a
serializer-independent entry point
Two steps towards removing the derives, both verifiable while they are still in place.
The convenience helpers and the dyn decode adapter now run the explicit codecs:
`to_json`, `to_json_pretty`, `from_json`, `to_json_value`, `from_json_value` and
`decode_json_dyn` all go through `encode_json` / `decode_json` under a facade-less
context, which is the behaviour they always had — no facade means an `any` field holding
a UEBA payload is refused rather than silently mis-encoded. Their error type becomes
`BaboonCodecError`, since `serde_json::Error` cannot survive the conversion.
The map-key adapters were the real blocker. They exist as whole-map serde functions only
because `#[serde(with = ...)]` attaches to a field, and their `deserialize` is bounded on
`V: Deserialize` — which stops compiling the moment the derives go. The explicit codecs
build the map themselves and only ever needed the key, so the adapter now also exposes
pub fn key_to_string(k: &K) -> String
pub fn key_from_string(s: &str) -> Result<K, String>
reusing the per-key expressions that were already being computed inside the serde
functions: `Display` / `parse_repr` for an id type, peel-and-rewrap for a single-primitive
wrapper, and the host-registered key codec for a Custom foreign. The codecs call these
per key instead of delegating the whole field, so nothing in the JSON path is bounded on
serde traits any more. The whole-map serde functions are now dead and go out with the
derives.
`RsMapKeyAdapter` gained `keyAdapterPathFor`, the same lookup against a bare key type.
Verified with the corpus at 901 passing and a purpose-built probe model (an any-bearing
DTO carrying id-keyed, enum-keyed and wrapper-keyed maps) compiling as a standalone crate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… strings everywhere
Two changes that both come from the same place — a JSON wire that no single
mechanism owned.
Rust: replace the serde derives with explicit codecs
----------------------------------------------------
`serde::Serialize::serialize(&self, S)` has no parameter a `BaboonCodecContext`
could travel in, so no derive-produced encoder could reach the facade that an
`any` field holding a UEBA payload needs in order to transcode itself to JSON.
That was the Rust half of the facade-context defect, and unfixable while the
derive owned the wire.
Every generated DTO, enum and ADT now carries `encode_json(ctx)` /
`decode_json(ctx, wire)`, mirroring the UEBA side. `serde_json::Value` stays as
the JSON DOM, so builtin scalars and foreign leaves still go through
`to_value`/`from_value` — Baboon cannot know a foreign type's shape, and a std
scalar's serde form is exact by construction.
Before the derives were deleted, the generated per-type oracles compared the
explicit codecs against the derive across the whole fixture corpus: 290
differential tests, all passing. That comparison cannot survive the removal of
its own subject, so it is recorded here rather than kept.
Consequences worth knowing:
* `encode_json`/`decode_json` are emitted for every generated type, not only
the ones JSON codecs are active for, and the generated `Cargo.toml` enables
`json-helpers` unconditionally. They are the structural round-trip the
conversions copy unrelated-but-identical types through; that availability
used to come from the unconditional derive.
* `preserve_order` on `serde_json` is now load-bearing: the encoders build a
`serde_json::Map` per object, and without the feature that Map is a BTreeMap
and every document comes out alphabetised. Two harness crates were missing
it (the derive never needed it).
* `Serialize`/`Deserialize for AnyOpaque` and the dead serde adapter modules
go with the derives. A second, context-less encoding path beside the
explicit codecs is exactly the drift hazard this removes.
* Timestamp map keys go through `time_formats::format_ts{u,o}`, not chrono's
`Display`, which renders `2026-05-02 12:00:00.123 +05:30` — a form no other
backend writes or reads.
* The map-key decode diagnostic keeps its `malformed key: ` prefix, which is
the cross-language contract the per-backend tests assert.
64-bit integers: decimal strings on the wire
---------------------------------------------
A JSON number beyond 2^53 has already been rounded by `JSON.parse` before any
codec runs, so `Int64.MaxValue` written by C# as a bare number reached a
TypeScript reader as 9223372036854775808 — a different, wrong, perfectly
valid-looking integer. The backends also disagreed about the representation in
both directions, and the spec contradicted itself.
Every backend now writes `i64` and `u64` as decimal strings. Backward
compatibility is carried by the readers, all of which accept both forms, so
documents written by an older compiler still decode. The one case that cannot
be made compatible is a numeric token above 2^53 read by TypeScript, where the
value is already gone: `BaboonInt64.read` refuses it with a diagnostic naming
the producer requirement instead of returning a rounded integer.
Found along the way: Kotlin wrote `u64` as `JsonPrimitive(value.toLong())`, so
values above `Long.MAX_VALUE` went on the wire as negative numbers. It now
writes the unsigned form, and its reader still accepts the old signed one.
Scala's reader needed no change — measured, not assumed: circe's
`decodeBigInt` already accepts a JSON string.
`docs/json-codecs.md` gains a "64-bit integers" section, and the OpenAPI/MCP
schema projections describe `i64`/`u64` as strings, the treatment `f128`
already had.
Verified with `mdl --seq :ci` (206 actions, green), which includes the
cross-language serialization acceptance matrix.
`defects.md` is removed: it is an artifact of a superseded process.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…y rests on
`i64`/`u64` now go on the wire as decimal strings, and documents written before
that change stay readable only because every reader still accepts a JSON
number. Nothing writes numbers any more, so the number arm of all ten decoders
was dead as far as the suite was concerned — one reader-side simplification
away from silently dropping backward compatibility.
One test per backend, decoding the `identifier.ok` fixture (`id LongId { x: i64 }`,
`id UInts { …, d: u64 }`) from both wire forms. Kotlin carries a fifth case: an
older Kotlin writer emitted `u64` as `JsonPrimitive(value.toLong())`, so values
above `Long.MAX_VALUE` went out as negative numbers, and that form has to keep
reading back as the same value.
Confirmed the tests actually run rather than passing by absence — 42 cases
across C# (4), Scala (4), Python (4), Rust (4), TypeScript (4), Kotlin (5),
Kotlin-KMP (5), Java (4), Dart (4), Swift (4) — and that they fail for the right
reason: deleting the `Value::Number` arm from Rust's `read_i64`/`read_u64`, and
the `typeof wire === "number"` branch from TypeScript's `BaboonInt64.read`,
fails exactly the two legacy-numeric cases in each while the string cases stay
green.
Verified with `mdl --seq :ci` (green).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.