Repository navigation
table: use the table for messages with a oneof - #476
Merged
Merged
Conversation
This was referenced Sep 23, 2026
|
All contributors have signed the CLA ✍️ ✅ |
iainmcgin
added this pull request to stack #497
October 1, 2026 18:26
iainmcgin
marked this pull request as ready for review
October 1, 2026 18:46
zayedu
pushed a commit
to zayedu/buffa
that referenced
this pull request
Oct 2, 2026
Adds `CodecStrategy::Table`, the generator for the table codec proposed in anthropics#463. `Unrolled` stays the default. Stacked on anthropics#468 (the `buffa::table` runtime), which is stacked on anthropics#467. ```rust buffa_build::Config::new() .codec_strategy(CodecStrategy::Table) .codec_strategy_in(CodecStrategy::Unrolled, &[".wa.Message"]) ``` The plugin takes `codec_strategy=table` and repeatable `codec_strategy_in=<path>=<strategy>`. Rules match like `preserve_unknown_fields_in`: prefix, last match wins, rules over the global setting. A message with a `oneof`, `map` or group field, a custom string, bytes or collection type, `MessageSet`, or extension ranges with JSON stays unrolled, and so does every message that holds one. `table_plan.rs` computes that closure and reports it in one `TableCodecFallbackSummary` warning, silent when the user's own `Unrolled` rule is the cause. A rule that names such a message by its exact path is an error. `compile()` errors on rustc older than 1.77, and the MSRV job now also tests the table code on 1.77, so its timeout goes from 10 to 20 minutes. The 1,644 KB to 817 KB figure in anthropics#463 was measured with oneofs and maps flattened, so a real schema saves less until the follow-up that lets table messages hold unrolled, extern and well-known-type children. The conformance suite does not run under `Table`: `TestAllTypesProto3` has oneofs and maps, so its messages fall back. Parity rests on `buffa-test`, which compiles each schema twice under renamed packages and compares bytes, sizes, decoded values and errors, including on truncated, bit-flipped and noise input. Under an experiment that set all 66 `buffa-test` protos to `Table`, 602 tests pass with 119 table messages. A table message differs from an unrolled one in three ways, documented in the guide: a length past the end of its enclosing message fails at once with `UnexpectedEof`; `merge_field` cannot gather a non-contiguous buffer, so a type that another crate or run uses as a group or `DELIMITED` field must stay `Unrolled`; and `clear()` releases capacity. The generated code is tied to `buffa::table`, so regenerate it whenever `buffa` updates. About 1,300 lines are outside test files, well over the 250-line guideline. Followed by anthropics#475 (message fields that hold messages without a table), anthropics#476 (`oneof`) and anthropics#477 (`map`), stacked in that order.
azdagron
previously approved these changes
Oct 7, 2026
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Oct 7, 2026
A oneof is an Option of an enum whose layout is unspecified, so a table entry cannot address a member by offset. Add the OneofMember kind, a OneofEnum trait that generated code implements for the enum, and the descriptors the interpreters use to read the member that is set and to replace it. Members decode by their payload kind through the arms that ordinary fields use, and the member with the lowest number writes and sizes the whole oneof, so the bytes match unrolled code.
The planner no longer rejects a oneof. Each member gets a table entry of its own, all at the offset of the field that holds the oneof, and the oneof's enum implements the accessor trait the interpreters read it through. The accessors are safe code, so generated crates that forbid unsafe code still compile, and they name the type of each member's value so a mismatch is a compile error.
Members of every type, two oneofs interleaved with ordinary fields, recursion through a oneof, sparse and extreme field numbers, nested and keyword-named members, closed enums that reject a number, merging into a oneof that is set, and a build with a type prefix, no unknown fields and inline message members.
The member with the lowest number sizes and writes the whole oneof, and the others used to enter the same function to find that out. A separate kind for the leader lets the followers fall through the dispatch, which takes the size pass of a message with two oneofs from 981 to 603 instructions.
A message member that is not the one that is set is decoded into a new default member, which becomes the one that is set only if the decoding succeeds, so a failure leaves the oneof as it was. Before, the default member was installed first and a failure left it in place. The oneof accessors also check that the default member they build is the one that was asked for, and a payload lookup checks that the entry is a member of the calling oneof. A message member's descriptor must be one that reaches the message directly.
The generated payload accessors turned a reference into a raw pointer with an `as` cast, which `trivial_casts` flags in user crates. They now coerce it, which does not, and the same reference type still rejects a variant of another type. The planner no longer sets a placeholder kind on oneof members, the error for too many descriptors is shared, and the codegen test fixtures are named for what they hold.
… closed enums The differential tests now compare the message left after a failed merge between the table and unrolled codecs, for a member of every kind, for message members, and for the recursion and size limits. New schemas cover a custom pointer for message members, oneofs named like Rust keywords with members out of number order and idiomatic field names, a closed enum in a oneof with no unknown fields, and a oneof whose members are messages of another package in each layout.
Decoding a oneof now follows unrolled code exactly, so the documented difference goes. The size claims quote the measured schema, the oneof mechanism is described in one place, and the changelog fragment for the oneof work is folded into the table codec's.
place_with swaps the new member into the oneof on success and drops whichever member is left in the local afterwards, so the drop glue of the oneof enum is instantiated once in it, not twice.
Cover the round trip into every kind of sink, merging into the member that is set, and a failed decode, for a member whose message has no table here. The Table::new safety docs and the check_member message name MsgVt::direct_via_message next to MsgVt::direct.
A member whose message is set to unrolled, and one whose message another crate generates, are reached through the message's impl.
A oneof member that holds a message set to unrolled, a well-known type, a message from another crate, and a message with a bytes field of a non-default type now goes through the table in a differential test. The holder of the last stays unrolled, and decodes its Bytes fields without a copy.
3 of 752 messages stay unrolled, for their maps, and the text section at opt-level z is 42% below unrolled code.
The generated MsgVt::direct names the payload type, so a table of another message is a type error, with a compile_fail doctest. Table::new rejects a direct descriptor on an ordinary message field.
The test read the slot between calls through one pointer, which Tree Borrows rejects. The table tests now pass under both Stacked and Tree Borrows, and the CI comment names the oneof cases Miri covers.
Also correct the comment on the swap in place_with_impl: the member that is dropped there is the one the new member replaced, and a new member that failed to decode is dropped where fresh goes out of scope.
…ract The module documentation of the private oneof module was not rendered, so the mechanism moves to the trait, and it now says that decoding uses arms of its own. EnumShape::accepts must agree with set, the __table_entry! docs give the leader argument, and a SAFETY comment names the field it means.
… and wide oneofs A failed merge into a message member stored inline agrees with unrolled code. Members that share a message or enum type share their descriptor, checked in the generated table and by a differential test. A oneof in a message of 258 entries, which has no dense lookup, agrees with unrolled code.
The buffa-test one covers a oneof as well as a message with a map, and the codegen one is about a holder that is a table.
…a message member OneofVt::new no longer instantiates the in-place decoding for its enum, and OneofVt::with_messages does. Generated code picks the constructor from the oneof's members, and Table::new rejects a message member of a oneof built with new.
Collaborator
Author
|
[claude code] The stack stopped compiling when it was rebased onto Pushed one commit to The push dismissed the earlier approvals, so the three PRs need a re-approval. |
azdagron
approved these changes
Oct 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Messages with a
oneofuse the table underCodecStrategy::Table. On the WhatsApp schema (752 messages) with the table requested, 43 messages fall back on the bridge PR (#475) and 3 here, all withmapfields. Text section, fat LTO, bridge PR → this one (fully unrolled in brackets):zs3A oneof is an
Option<Enum>of unspecified layout, so a member has no offset. Every member gets a table entry at the offset of theOption, and generated code implements the newbuffa::table::OneofEnum: the number of the member that is set, a pointer to its value, and a way to set a member by number. The impl is safe code that names each value's type, so a mismatch does not compile and generated crates keepforbid(unsafe_code). The lowest-numbered member sizes and writes whichever member is set, at its own position, so the bytes equal unrolled code's; the others only decode. Decoding follows unrolled code, including the message a failed merge leaves behind. A message member whose message has no table is reached through itsMessageimpl, and #475's bytes rule covers oneof members. Only a oneof with a message member gets in-place decoding (1.2 KB atz).EnumShape, anunsafetrait, gains a required methodacceptsthat must agree withset; if it does not, a rejected closed-enum value leaves a default member where another member was set. The newunsafe(table/oneof.rs,table/shape.rs, the interpreters) runs under Miri through hand-written enums (CI: Stacked Borrows; Tree Borrows run locally). GeneratedOneofEnumcode runs natively only.Table ÷ unrolled for a message of two oneofs and two fields, by instruction count: size 4.9×, encode 2.9×, decode 1.4×.
Stacked on #475.