Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
3112169
table: reach the members of a oneof through its enum
iainmcgin Sep 23, 2026
05606d8
codegen: give messages with oneofs a table
iainmcgin Sep 23, 2026
25ef40e
test: compare the table with the unrolled codec on oneofs
iainmcgin Sep 23, 2026
dda257e
table: skip a oneof's followers in the size and write loops
iainmcgin Sep 23, 2026
0a88d38
docs: describe oneof support in the table codec
iainmcgin Sep 23, 2026
2294c71
codegen: name the placeholder kind of a oneof member correctly
iainmcgin Sep 23, 2026
73832bc
table: decode a oneof message member the way unrolled code does
iainmcgin Sep 23, 2026
5b17b63
codegen: coerce the oneof accessor pointers instead of casting them
iainmcgin Sep 23, 2026
06a1d5b
tests: compare oneof decoding of failures, custom pointers, names and…
iainmcgin Sep 23, 2026
c4e6892
docs: describe oneof decoding and the size measured, drop the deviation
iainmcgin Sep 23, 2026
60f85cc
table: drop the discarded oneof member at a single site
iainmcgin Sep 23, 2026
d46ba9b
table: test a oneof member reached through its Message impl
iainmcgin Sep 23, 2026
26e9aa5
codegen: test oneof members that have no table
iainmcgin Sep 23, 2026
db25c0f
buffa-test: compare oneof members whose messages have no table
iainmcgin Sep 23, 2026
e416c0e
docs: quote the size of the WhatsApp schema with oneofs in the table
iainmcgin Sep 23, 2026
c2f3178
table: expect the new text of the direct-descriptor panic
iainmcgin Sep 23, 2026
82fbb76
table: tie a oneof member's message descriptor to its payload type
iainmcgin Sep 23, 2026
cd1d901
table: derive the raw slot pointer again after each read in a Miri test
iainmcgin Sep 23, 2026
4f5b4f0
table: assert in debug builds that a oneof payload pointer is not null
iainmcgin Sep 23, 2026
c68534c
table: document the oneof mechanism on OneofEnum and the accepts cont…
iainmcgin Sep 23, 2026
818761a
buffa-test: cover failed merges of inline members, shared descriptors…
iainmcgin Sep 23, 2026
90aad03
tests: name two tests for what they cover and fix an article
iainmcgin Sep 23, 2026
9a34308
docs: reflow comments split by earlier edits and name the OneofEnum d…
iainmcgin Sep 23, 2026
6850514
table: decode into a oneof member in place only for oneofs that have …
iainmcgin Sep 23, 2026
65927aa
docs: update the table codec size figure for oneofs without in-place …
iainmcgin Sep 23, 2026
1202c52
changelog: say which oneofs use the table
iainmcgin Sep 23, 2026
0623470
table: propagate a oneof enum name conflict from the table codec
iainmcgin Oct 7, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
kind: Added
body: |-
**Table-driven message codec** (#469, refs #463). `buffa_build::Config::codec_strategy(CodecStrategy::Table)` (plugin option `codec_strategy=table`, `CodeGenConfig::codec_strategy`) generates each message's binary `Message` implementation from a static table and interpreters that every message shares, instead of code specialised to the message's fields. The compiled code is substantially smaller, and messages made of many small fields are slower. `CodecStrategy::Unrolled` stays the default. `codec_strategy_in(strategy, &[paths])` (plugin option `codec_strategy_in=<path>=<strategy>`, repeatable) chooses the strategy for matching messages and the messages nested in them, and the last matching rule wins. The wire format does not change. Messages the table cannot handle stay unrolled, and `CodeGenWarning::TableCodecFallbackSummary` counts them. A rule that selects the table, by exact path, for a message that cannot use it is an error. The generated code needs Rust 1.77 or later, which `buffa-build` checks, and compiles in a crate with `#![forbid(unsafe_code)]`. The guide's "Smaller generated code" section has the measurements, the messages that stay unrolled and the behavioural differences.
**Table-driven message codec** (#469, refs #463). `buffa_build::Config::codec_strategy(CodecStrategy::Table)` (plugin option `codec_strategy=table`, `CodeGenConfig::codec_strategy`) generates each message's binary `Message` implementation from a static table and interpreters that every message shares, instead of code specialised to the message's fields. The compiled code is substantially smaller, and messages made of many small fields are slower. `CodecStrategy::Unrolled` stays the default. `codec_strategy_in(strategy, &[paths])` (plugin option `codec_strategy_in=<path>=<strategy>`, repeatable) chooses the strategy for matching messages and the messages nested in them, and the last matching rule wins. The wire format does not change, and a message with a `oneof` uses the table, including when a member is a message that has no table. Messages the table cannot handle stay unrolled, and `CodeGenWarning::TableCodecFallbackSummary` counts them. A rule that selects the table, by exact path, for a message that cannot use it is an error. The generated code needs Rust 1.77 or later, which `buffa-build` checks, and compiles in a crate with `#![forbid(unsafe_code)]`. The guide's "Smaller generated code" section has the measurements, the messages that stay unrolled and the behavioural differences.
time: 2026-09-23T03:10:00+00:00
5 changes: 4 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -305,7 +305,10 @@ jobs:
# provenance, alignment and initialisation on the kinds and field
# shapes the module's tests build, including message fields stored
# inline, boxed and in a `Vec`, whose child pointers point into the
# parent or the heap.
# parent or the heap, and oneofs, through hand-written enums with
# inline and boxed message members. The `OneofEnum` code that codegen
# emits runs natively only, in the differential tests of `buffa-test`,
# not under Miri.
- name: Miri (table interpreter soundness)
run: cargo +${{ env.MIRI_TOOLCHAIN }} miri test -p buffa --lib -- 'table::'

Expand Down
5 changes: 3 additions & 2 deletions DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -560,12 +560,13 @@ Within that bound, the shortcuts do not pay off. `merge` does not help: it consu

The size, write, and merge code of an unrolled message is specialised to its fields. `CodecStrategy::Table` (a per-message option, `Unrolled` by default) replaces it with a static `buffa::table::Table<M>` and a `Message` impl that forwards to interpreters in `buffa::table`. The decision to keep `Unrolled` as the default is in [#463](https://github.com/anthropics/buffa/issues/463), and the guide's "Smaller generated code" section has the measurements and the list of messages that stay unrolled.

A table holds a sorted array of 12-byte entries `{tag, offset, kind, tag_len, aux}`, a dense array that maps field numbers below 64 to entries, and the offset of the unknown-fields slot. `kind` is the field type crossed with its cardinality, so the interpreter dispatches once per field. Message, repeated-message, and enum fields carry a small descriptor (`Aux`) with the accessors that their storage needs, because a `MessageField`, a `Vec`, and an `EnumValue` cannot be read through an offset alone. Offsets come from `core::mem::offset_of!`, so the table needs Rust 1.77 and the generated code refers to it through `buffa::__table!`, which is a compile error on an older compiler.
A table holds a sorted array of 12-byte entries `{tag, offset, kind, tag_len, aux}`, a dense array that maps field numbers below 64 to entries, and the offset of the unknown-fields slot. `kind` is the field type crossed with its cardinality, so the interpreter dispatches once per field. Message, repeated-message, and enum fields carry a small descriptor (`Aux`) with the accessors that their storage needs, because a `MessageField`, a `Vec`, and an `EnumValue` cannot be read through an offset alone; a oneof has a `Group` descriptor and each of its members a `Member` one. Offsets come from `core::mem::offset_of!`, so the table needs Rust 1.77 and the generated code refers to it through `buffa::__table!`, which is a compile error on an older compiler.

Three decisions shape the runtime:
Four decisions shape the runtime:

- **The `unsafe` lives in `buffa`.** `__table!` and `__table_entry!` contain the `unsafe` blocks and witness each field's type against its kind, so a table that names the wrong kind for a field does not compile, and generated code compiles under `#![forbid(unsafe_code)]`. `Table::new` also checks the layout constants at compile time. The interpreters run under Miri in CI.
- **The interpreters are not generic over the sink or the input where that is avoidable.** `Message::encode` and its siblings write any `BufMut` through one shared, non-generic cursor (`buffa/src/encode_sink.rs`), and decoding runs over a contiguous `&[u8]`, so the interpreters are compiled once in `buffa`, at its `opt-level`, and not once per caller.
- **A oneof is one table entry per member,** all at the offset of the `Option<Enum>`, because that enum has no specified layout. Generated code implements the safe trait `OneofEnum` for it, so the interpreters can find the member that is set, and the `unsafe` stays in `buffa`. The bytes and the decoding are the same as unrolled code's; the documentation of `OneofEnum` in `buffa/src/table/oneof.rs` describes the mechanism and what an implementation must guarantee.
- **A child reaches the interpreters through its table or its `Message` impl.** A message field's `Aux` descriptor holds either the child's `MessageTable` or a `DynVt` of function pointers instantiated for the child type (`table/bridge.rs`), so a table message can hold any message it is not required to leave unrolled. The write pointer takes the `PreSized` cursor because a function pointer cannot be generic over the sink. A pointer per child type and sink type would avoid the copy that other sinks cost, at the price of code in every crate that uses the table. The planner in `buffa-codegen` (`table_plan.rs`) selects the messages that asked for the table and drops those the interpreters cannot handle. Among them it drops, transitively, the holders of a message of the run with a non-default bytes type, because the table decodes from one contiguous slice and would copy the `bytes::Bytes` fields that unrolled code shares with a `Bytes` input. A child from another crate is not inspected, so a holder of `google.protobuf.Any`, whose `value` is `bytes::Bytes`, keeps its table and copies the payload.

### Owned decode: intentional throughput trade-offs
Expand Down
11 changes: 6 additions & 5 deletions buffa-codegen/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1094,7 +1094,7 @@ pub enum CodecStrategy {
///
/// Not every message can use it. These stay [`Unrolled`](Self::Unrolled):
///
/// - a message with a `oneof`, a `map` field, or a group field;
/// - a message with a `map` field or a group field;
/// - the message type of a group field;
/// - a message that uses the `MessageSet` wire format;
/// - a message with extension ranges, when JSON code is generated and
Expand All @@ -1108,9 +1108,10 @@ pub enum CodecStrategy {
/// contiguous slice, so it would copy `bytes::Bytes` fields that
/// unrolled code decoding from a `Bytes` shares with the input.
///
/// A table message may hold any other message. It reaches a child that is a
/// table message through the child's table, and any other child, whether
/// it is [`Unrolled`](Self::Unrolled), generated by another crate, or a
/// A table message may hold any other message, including as a member of a
/// `oneof`. It reaches a child that is a table message through the
/// child's table, and any other child, whether it is
/// [`Unrolled`](Self::Unrolled), generated by another crate, or a
/// well-known type, through the child's `Message` impl, which costs a
/// function call per child. Selecting a message does not select the
/// messages it holds, and keeping a child [`Unrolled`](Self::Unrolled) does
Expand Down Expand Up @@ -1156,7 +1157,7 @@ pub enum CodecStrategy {
#[derive(Debug, Clone, PartialEq, Eq)]
#[non_exhaustive]
pub struct TableCodecFallbackReason {
/// The reason as a predicate, such as `has a oneof`. The wording is for
/// The reason as a predicate, such as `has a map field`. The wording is for
/// people and may change between releases.
pub reason: String,
/// The proto paths of all of them, with a leading dot, in declaration
Expand Down
Loading
Loading