Repository navigation
table: let a table message hold messages that have no table - #475
Merged
Merged
Conversation
|
All contributors have signed the CLA ✍️ ✅ |
This was referenced Sep 23, 2026
iainmcgin
added this pull request to stack #497
October 1, 2026 18:26
iainmcgin
force-pushed
the
table-bridge
branch
from
October 1, 2026 18:40
d953e07 to
c8b5493
Compare
iainmcgin
marked this pull request as ready for review
October 1, 2026 18:46
… wording A table message that holds a message of another crate with a bytes field, such as google.protobuf.Any, copies the field on decode. Say so in the CodecStrategy::Table docs, the guide, DESIGN.md and the runtime docs, and point at the guide for the measurements. Correct the bridge module docs (dangling list, renamed constructors, Rope copy), qualify the size claim, drop the fallback warning's claim that a fallback is larger, and shorten the changelog fragment to a user-level summary.
…ested bytes Add codegen tests for a holder declared before the message it holds, cycles with and without a bytes field, repeated bytes, map values, the bytes-key carve-out and a nested message. Add buffa-test checks that the messages set to unrolled really have no table, and a test that pins the copy of an Any payload.
iainmcgin
force-pushed
the
table-bridge
branch
from
October 2, 2026 00:30
c8b5493 to
41f9bc9
Compare
zayedu
pushed a commit
to zayedu/buffa
that referenced
this pull request
Oct 2, 2026
Adds `CodecStrategy::Table`, the generator for the table codec proposed in anthropics#463. `Unrolled` stays the default. Stacked on anthropics#468 (the `buffa::table` runtime), which is stacked on anthropics#467. ```rust buffa_build::Config::new() .codec_strategy(CodecStrategy::Table) .codec_strategy_in(CodecStrategy::Unrolled, &[".wa.Message"]) ``` The plugin takes `codec_strategy=table` and repeatable `codec_strategy_in=<path>=<strategy>`. Rules match like `preserve_unknown_fields_in`: prefix, last match wins, rules over the global setting. A message with a `oneof`, `map` or group field, a custom string, bytes or collection type, `MessageSet`, or extension ranges with JSON stays unrolled, and so does every message that holds one. `table_plan.rs` computes that closure and reports it in one `TableCodecFallbackSummary` warning, silent when the user's own `Unrolled` rule is the cause. A rule that names such a message by its exact path is an error. `compile()` errors on rustc older than 1.77, and the MSRV job now also tests the table code on 1.77, so its timeout goes from 10 to 20 minutes. The 1,644 KB to 817 KB figure in anthropics#463 was measured with oneofs and maps flattened, so a real schema saves less until the follow-up that lets table messages hold unrolled, extern and well-known-type children. The conformance suite does not run under `Table`: `TestAllTypesProto3` has oneofs and maps, so its messages fall back. Parity rests on `buffa-test`, which compiles each schema twice under renamed packages and compares bytes, sizes, decoded values and errors, including on truncated, bit-flipped and noise input. Under an experiment that set all 66 `buffa-test` protos to `Table`, 602 tests pass with 119 table messages. A table message differs from an unrolled one in three ways, documented in the guide: a length past the end of its enclosing message fails at once with `UnexpectedEof`; `merge_field` cannot gather a non-contiguous buffer, so a type that another crate or run uses as a group or `DELIMITED` field must stay `Unrolled`; and `clear()` releases capacity. The generated code is tied to `buffa::table`, so regenerate it whenever `buffa` updates. About 1,300 lines are outside test files, well over the 250-line guideline. Followed by anthropics#475 (message fields that hold messages without a table), anthropics#476 (`oneof`) and anthropics#477 (`map`), stacked in that order.
This was referenced Oct 5, 2026
azdagron
approved these changes
Oct 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Message fields of a table message can now hold messages that have no table: unrolled ones,
extern_pathtypes, and well-known types. Onwhatsapp.proto(752 messages,Boxfields, fat LTO), the text section atopt-level = "z"is 1,669,507 bytes unrolled, 1,236,115 withCodecStrategy::Tableon #469, and 1,009,611 here; table messages go from 636 to 709 of 752, because 73 of the 116 that fell back on #469 only held a message with aoneoformap, and 43 have one. The guide has thesand3figures.A field's descriptor holds the child's table or a
DynVt, function pointers that call the child'sMessageimpl.MsgVt::new_via_messageandRepVt::new_via_messageare newconst fns inbuffa::table; a child reached this way decodes from the slice of the nearest enclosing table message, so an overrun fails at that message's end.Differences from #469:
&[u8]and would copy theBytesfields. Another crate's child is not inspected, so a holder ofgoogle.protobuf.Any(valueisBytes) keeps its table and copies the payload.DynVtwrite function takes thePreSizedcursor thatMessage::encodeuses for everyBufMut. Any other sink, including aBufMutpassed toMessage::write_to, gets each child from a scratchVec; aRopecopies again.Instructions per call against all unrolled: a table parent with nine unrolled children is +6.5% to size, +7.3% to encode, +6.5% to decode; a table message of two
Timestampfields is +54%, +41%, +31%. Table-to-table fields cost 0.3% to 1.2% more than on #469.The
unsafeintable/bridge.rsis three thunks that cast*const u8back to the child type, and theChildmethods forwarding to them.