Skip to content

table: let a table message hold messages that have no table - #475

Merged
iainmcgin merged 11 commits into
mainfrom
table-bridge
Oct 7, 2026
Merged

iainmcgin merged 11 commits into
mainfrom
table-bridge

Conversation

@iainmcgin

Copy link
Copy Markdown
Collaborator

Message fields of a table message can now hold messages that have no table: unrolled ones, extern_path types, and well-known types. On whatsapp.proto (752 messages, Box fields, fat LTO), the text section at opt-level = "z" is 1,669,507 bytes unrolled, 1,236,115 with CodecStrategy::Table on #469, and 1,009,611 here; table messages go from 636 to 709 of 752, because 73 of the 116 that fell back on #469 only held a message with a oneof or map, and 43 have one. The guide has the s and 3 figures.

A field's descriptor holds the child's table or a DynVt, function pointers that call the child's Message impl. MsgVt::new_via_message and RepVt::new_via_message are new const fns in buffa::table; a child reached this way decodes from the slice of the nearest enclosing table message, so an overrun fails at that message's end.

Differences from #469:

  • A message holding, transitively and through any field kind, a message of the run with a non-default bytes type stays unrolled, because the table decodes a contiguous &[u8] and would copy the Bytes fields. Another crate's child is not inspected, so a holder of google.protobuf.Any (value is Bytes) keeps its table and copies the payload.
  • The DynVt write function takes the PreSized cursor that Message::encode uses for every BufMut. Any other sink, including a BufMut passed to Message::write_to, gets each child from a scratch Vec; a Rope copies again.

Instructions per call against all unrolled: a table parent with nine unrolled children is +6.5% to size, +7.3% to encode, +6.5% to decode; a table message of two Timestamp fields is +54%, +41%, +31%. Table-to-table fields cost 0.3% to 1.2% more than on #469.

The unsafe in table/bridge.rs is three thunks that cast *const u8 back to the child type, and the Child methods forwarding to them.

@github-actions

Copy link
Copy Markdown

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@iainmcgin
iainmcgin added this pull request to stack #497 October 1, 2026 18:26
@iainmcgin
iainmcgin marked this pull request as ready for review October 1, 2026 18:46
Base automatically changed from table-codegen to main October 2, 2026 00:30
… wording

A table message that holds a message of another crate with a bytes field,
such as google.protobuf.Any, copies the field on decode. Say so in the
CodecStrategy::Table docs, the guide, DESIGN.md and the runtime docs, and
point at the guide for the measurements. Correct the bridge module docs
(dangling list, renamed constructors, Rope copy), qualify the size claim,
drop the fallback warning's claim that a fallback is larger, and shorten
the changelog fragment to a user-level summary.
…ested bytes

Add codegen tests for a holder declared before the message it holds, cycles
with and without a bytes field, repeated bytes, map values, the bytes-key
carve-out and a nested message. Add buffa-test checks that the messages set
to unrolled really have no table, and a test that pins the copy of an Any
payload.
zayedu pushed a commit to zayedu/buffa that referenced this pull request Oct 2, 2026
Adds `CodecStrategy::Table`, the generator for the table codec proposed
in anthropics#463. `Unrolled` stays the default. Stacked on anthropics#468 (the
`buffa::table` runtime), which is stacked on anthropics#467.

```rust
buffa_build::Config::new()
    .codec_strategy(CodecStrategy::Table)
    .codec_strategy_in(CodecStrategy::Unrolled, &[".wa.Message"])
```

The plugin takes `codec_strategy=table` and repeatable
`codec_strategy_in=<path>=<strategy>`. Rules match like
`preserve_unknown_fields_in`: prefix, last match wins, rules over the
global setting. A message with a `oneof`, `map` or group field, a custom
string, bytes or collection type, `MessageSet`, or extension ranges with
JSON stays unrolled, and so does every message that holds one.
`table_plan.rs` computes that closure and reports it in one
`TableCodecFallbackSummary` warning, silent when the user's own
`Unrolled` rule is the cause. A rule that names such a message by its
exact path is an error. `compile()` errors on rustc older than 1.77, and
the MSRV job now also tests the table code on 1.77, so its timeout goes
from 10 to 20 minutes.

The 1,644 KB to 817 KB figure in anthropics#463 was measured with oneofs and maps
flattened, so a real schema saves less until the follow-up that lets
table messages hold unrolled, extern and well-known-type children.

The conformance suite does not run under `Table`: `TestAllTypesProto3`
has oneofs and maps, so its messages fall back. Parity rests on
`buffa-test`, which compiles each schema twice under renamed packages
and compares bytes, sizes, decoded values and errors, including on
truncated, bit-flipped and noise input. Under an experiment that set all
66 `buffa-test` protos to `Table`, 602 tests pass with 119 table
messages.

A table message differs from an unrolled one in three ways, documented
in the guide: a length past the end of its enclosing message fails at
once with `UnexpectedEof`; `merge_field` cannot gather a non-contiguous
buffer, so a type that another crate or run uses as a group or
`DELIMITED` field must stay `Unrolled`; and `clear()` releases capacity.
The generated code is tied to `buffa::table`, so regenerate it whenever
`buffa` updates.

About 1,300 lines are outside test files, well over the 250-line
guideline.


Followed by anthropics#475 (message fields that hold messages without a table),
anthropics#476 (`oneof`) and anthropics#477 (`map`), stacked in that order.
@iainmcgin
iainmcgin added this pull request to the merge queue Oct 7, 2026
Merged via the queue into main with commit 6c528d3 Oct 7, 2026
11 checks passed
@iainmcgin
iainmcgin deleted the table-bridge branch October 7, 2026 15:36
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 7, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants