table: let a table message hold messages that have no table

.changes/unreleased/added-20260923-codec-strategy-table.yaml

@@ -1,4 +1,4 @@
 kind: Added
 body: |-
-    **Table-driven message codec** (#469, refs #463). `buffa_build::Config::codec_strategy(CodecStrategy::Table)` (plugin option `codec_strategy=table`, `CodeGenConfig::codec_strategy`) generates each message's binary `Message` implementation from a static table and interpreters that every message shares, instead of code specialised to the message's fields. It roughly halves the compiled size of a large schema at `opt-level = "z"`, and messages made of many small fields are slower. `CodecStrategy::Unrolled` stays the default. `codec_strategy_in(strategy, &[paths])` (plugin option `codec_strategy_in=<path>=<strategy>`, repeatable) chooses the strategy for matching messages and the messages nested in them, and the last matching rule wins. The wire format does not change. A message that has a `oneof`, `map`, or group field, or holds a message that is not a table, stays unrolled, and `CodeGenWarning::TableCodecFallbackSummary` counts those; a rule that names such a message by its exact path is an error. The generated code needs Rust 1.77 or later, which `buffa-build` checks, and compiles in a crate with `#![forbid(unsafe_code)]`; see the guide's "Smaller generated code" section for the rest.
+    **Table-driven message codec** (#469, refs #463). `buffa_build::Config::codec_strategy(CodecStrategy::Table)` (plugin option `codec_strategy=table`, `CodeGenConfig::codec_strategy`) generates each message's binary `Message` implementation from a static table and interpreters that every message shares, instead of code specialised to the message's fields. The compiled code is substantially smaller, and messages made of many small fields are slower. `CodecStrategy::Unrolled` stays the default. `codec_strategy_in(strategy, &[paths])` (plugin option `codec_strategy_in=<path>=<strategy>`, repeatable) chooses the strategy for matching messages and the messages nested in them, and the last matching rule wins. The wire format does not change. Messages the table cannot handle stay unrolled, and `CodeGenWarning::TableCodecFallbackSummary` counts them. A rule that selects the table, by exact path, for a message that cannot use it is an error. The generated code needs Rust 1.77 or later, which `buffa-build` checks, and compiles in a crate with `#![forbid(unsafe_code)]`. The guide's "Smaller generated code" section has the measurements, the messages that stay unrolled and the behavioural differences.
 time: 2026-09-23T03:10:00+00:00

DESIGN.md

@@ -558,15 +558,15 @@ Within that bound, the shortcuts do not pay off. `merge` does not help: it consu
 
 ### 13. Table-Driven Codec (`CodecStrategy::Table`)
 
-The size, write, and merge code of an unrolled message is specialised to its fields. `CodecStrategy::Table` (a per-message option, `Unrolled` by default) replaces it with a static `buffa::table::Table<M>` and a `Message` impl that forwards to interpreters in `buffa::table`. The measurements and the decision to keep `Unrolled` as the default are in [#463](https://github.com/anthropics/buffa/issues/463).
+The size, write, and merge code of an unrolled message is specialised to its fields. `CodecStrategy::Table` (a per-message option, `Unrolled` by default) replaces it with a static `buffa::table::Table<M>` and a `Message` impl that forwards to interpreters in `buffa::table`. The decision to keep `Unrolled` as the default is in [#463](https://github.com/anthropics/buffa/issues/463), and the guide's "Smaller generated code" section has the measurements and the list of messages that stay unrolled.
 
 A table holds a sorted array of 12-byte entries `{tag, offset, kind, tag_len, aux}`, a dense array that maps field numbers below 64 to entries, and the offset of the unknown-fields slot. `kind` is the field type crossed with its cardinality, so the interpreter dispatches once per field. Message, repeated-message, and enum fields carry a small descriptor (`Aux`) with the accessors that their storage needs, because a `MessageField`, a `Vec`, and an `EnumValue` cannot be read through an offset alone. Offsets come from `core::mem::offset_of!`, so the table needs Rust 1.77 and the generated code refers to it through `buffa::__table!`, which is a compile error on an older compiler.
 
 Three decisions shape the runtime:
 
 - **The `unsafe` lives in `buffa`.** `__table!` and `__table_entry!` contain the `unsafe` blocks and witness each field's type against its kind, so a table that names the wrong kind for a field does not compile, and generated code compiles under `#![forbid(unsafe_code)]`. `Table::new` also checks the layout constants at compile time. The interpreters run under Miri in CI.
 - **The interpreters are not generic over the sink or the input where that is avoidable.** `Message::encode` and its siblings write any `BufMut` through one shared, non-generic cursor (`buffa/src/encode_sink.rs`), and decoding runs over a contiguous `&[u8]`, so the interpreters are compiled once in `buffa`, at its `opt-level`, and not once per caller.
-- **A table refers to the tables of its children,** so a message can use the table only if every message it holds does. The planner in `buffa-codegen` (`table_plan.rs`) starts from the messages that asked for the table, removes those the interpreters cannot handle (oneofs, maps, groups and the types of group fields, `MessageSet`, extension ranges with JSON, custom string, bytes, or collection types), and then removes every message that holds a removed one, until none is left. A message that holds a type from another crate, such as a well-known type, is removed the same way, because the static table of that type is not visible.
+- **A child reaches the interpreters through its table or its `Message` impl.** A message field's `Aux` descriptor holds either the child's `MessageTable` or a `DynVt` of function pointers instantiated for the child type (`table/bridge.rs`), so a table message can hold any message it is not required to leave unrolled. The write pointer takes the `PreSized` cursor because a function pointer cannot be generic over the sink. A pointer per child type and sink type would avoid the copy that other sinks cost, at the price of code in every crate that uses the table. The planner in `buffa-codegen` (`table_plan.rs`) selects the messages that asked for the table and drops those the interpreters cannot handle. Among them it drops, transitively, the holders of a message of the run with a non-default bytes type, because the table decodes from one contiguous slice and would copy the `bytes::Bytes` fields that unrolled code shares with a `Bytes` input. A child from another crate is not inspected, so a holder of `google.protobuf.Any`, whose `value` is `bytes::Bytes`, keeps its table and copies the payload.
 
 ### Owned decode: intentional throughput trade-offs
 

buffa-build/src/lib.rs

@@ -1554,14 +1554,13 @@ impl Config {
     /// Choose how the binary `Message` implementation of every message is
     /// generated (default: [`CodecStrategy::Unrolled`]).
     ///
-    /// On a schema it fully covers, [`CodecStrategy::Table`] makes the
-    /// compiled size about half as big at `opt-level = "z"`, and it slows
-    /// messages made of many small fields; a message that cannot use it keeps
-    /// its size. [`CodecStrategy::Table`] has the measurements, says which
-    /// messages stay unrolled, and lists how a table message behaves
-    /// differently. This build reports
-    /// those in one `cargo:warning`. The option never changes the wire
-    /// format. The generated code needs Rust 1.77 or later, and `compile`
+    /// On a large schema, [`CodecStrategy::Table`] makes the compiled code
+    /// about 40% smaller at `opt-level = "z"`, and it slows messages made of
+    /// many small fields; a message that cannot use it keeps its size.
+    /// [`CodecStrategy::Table`] says which messages stay unrolled and how a
+    /// table message behaves differently, and this build reports the messages
+    /// that stay unrolled in one `cargo:warning`. The option never changes the
+    /// wire format. The generated code needs Rust 1.77 or later, and `compile`
     /// returns an error on an older compiler when a build script runs it (the
     /// compiler is read from `RUSTC`).
     ///
@@ -1601,13 +1600,8 @@ impl Config {
     /// names the message by its exact path. A rule that matches no message
     /// produces a warning.
     ///
-    /// A table message holds only table messages, and a rule does not extend
-    /// to the messages a message holds. Selecting a message with a rule
-    /// therefore also needs rules for everything it holds, unless the global
-    /// setting is [`CodecStrategy::Table`]. Choosing [`CodecStrategy::Unrolled`]
-    /// for a message keeps every message that holds it unrolled, with no
-    /// warning, and that usually includes the root message an application
-    /// encodes.
+    /// A rule does not extend to the messages a message holds; see
+    /// [`CodecStrategy::Table`].
     #[must_use]
     pub fn codec_strategy_in(mut self, strategy: CodecStrategy, paths: &[impl AsRef<str>]) -> Self {
         for raw in paths.iter().map(AsRef::as_ref) {

buffa-codegen/src/lib.rs

@@ -1086,11 +1086,11 @@ pub enum CodecStrategy {
     #[default]
     Unrolled,
     /// Emit one static table per message and forward `Message` to interpreters
-    /// shared by every message, in the `buffa` crate. On a schema the table
-    /// fully covers, it roughly halves the compiled size at
-    /// `opt-level = "z"`; a message that cannot use it keeps its size. It costs
-    /// run time on messages made of many small fields. The measurements are in
-    /// anthropics/buffa#463.
+    /// shared by every message, in the `buffa` crate. On a large schema it
+    /// makes the compiled code about 40% smaller at `opt-level = "z"`; a
+    /// message that cannot use it keeps its size. It costs run time on messages
+    /// made of many small fields. The guide's "Smaller generated code" section
+    /// has the measurements.
     ///
     /// Not every message can use it. These stay [`Unrolled`](Self::Unrolled):
     ///
@@ -1102,19 +1102,40 @@ pub enum CodecStrategy {
     /// - a message with a field of a non-default string, bytes, or collection
     ///   type, such as `use_bytes_type`, `string_type`, `bytes_type`, and
     ///   `repeated_type` select;
-    /// - a message with a field whose message type stays `Unrolled` for any
-    ///   reason, is not selected for the table, or is generated by another
-    ///   crate, such as a well-known type.
-    ///
-    /// A table message therefore holds only table messages, and selecting a
-    /// message does not select the messages it holds.
-    ///
-    /// A table message differs from an unrolled one in three ways. A field that
+    /// - a message that holds, in a singular, repeated, `oneof`, or map value
+    ///   field, a message of the same run that has a `bytes` field of a
+    ///   non-default type, or that holds one. The table decodes from one
+    ///   contiguous slice, so it would copy `bytes::Bytes` fields that
+    ///   unrolled code decoding from a `Bytes` shares with the input.
+    ///
+    /// A table message may hold any other message. It reaches a child that is a
+    /// table message through the child's table, and any other child, whether
+    /// it is [`Unrolled`](Self::Unrolled), generated by another crate, or a
+    /// well-known type, through the child's `Message` impl, which costs a
+    /// function call per child. Selecting a message does not select the
+    /// messages it holds, and keeping a child [`Unrolled`](Self::Unrolled) does
+    /// not keep the messages that hold it unrolled: to keep a whole path
+    /// specialised, set its holders to `Unrolled` too.
+    ///
+    /// A table message differs from an unrolled one in these ways. A field that
     /// declares a length past the end of its enclosing message fails at once
     /// with `DecodeError::UnexpectedEof`, where unrolled code can report a
-    /// different error for the same rejected input. `clear()` resets the message
-    /// to its default, which releases the capacity of its strings and vectors.
+    /// different error for the same rejected input; a child reached through its
+    /// `Message` impl is read from the slice of the nearest enclosing table
+    /// message, so it is bounded there. `clear()` resets the message to its
+    /// default, which releases the capacity of its strings and vectors.
     /// Decoding gathers a `Buf` that is not contiguous into one buffer first.
+    /// Encoding into any sink other than the cursor that `Message::encode` and
+    /// its siblings write a `BufMut` through, meaning a `Rope`, a sink defined
+    /// outside `buffa`, or a `BufMut` passed straight to `Message::write_to`,
+    /// stages each child reached through its `Message` impl in a scratch
+    /// buffer, and a `Rope` copies it again and cannot share the `bytes`
+    /// fields inside it by reference count. Codegen cannot see the fields of a
+    /// message from another crate, so a table message that holds one with a
+    /// `bytes` field of a non-default type copies it on decode. The well-known
+    /// type `google.protobuf.Any` is one, because its `value` is
+    /// `bytes::Bytes`. To keep the payload shared with a `Bytes` input, set the
+    /// holder to [`Unrolled`](Self::Unrolled) with `codec_strategy_in`.
     ///
     /// A message that another crate or codegen run uses as the type of a group
     /// or editions `DELIMITED` field must not be selected, because
@@ -1929,12 +1950,8 @@ pub struct CodeGenConfig {
     /// asked for something impossible. A rule that matches no generated message
     /// produces a [`CodeGenWarning::CodecStrategyRuleMatchedNothing`].
     ///
-    /// A table message holds only table messages, and a rule does not extend to
-    /// the messages a message holds. A [`CodecStrategy::Table`] rule for a
-    /// message therefore needs rules for the messages it holds, or a global
-    /// [`CodecStrategy::Table`]; an exact-path rule fails without them.
-    /// Setting a message to [`CodecStrategy::Unrolled`] keeps every message that
-    /// holds it unrolled too, without a warning.
+    /// A rule does not extend to the messages a message holds; see
+    /// [`CodecStrategy::Table`].
     pub codec_strategy_in: Vec<(String, CodecStrategy)>,
 }
 
@@ -2390,15 +2407,12 @@ pub enum CodeGenWarning {
     /// [`codec_strategy`](CodeGenConfig::codec_strategy) or by a
     /// [`codec_strategy_in`](CodeGenConfig::codec_strategy_in) rule, cannot use
     /// the table and are generated [`CodecStrategy::Unrolled`]. One warning
-    /// covers the whole run. A message that falls back only because a message
-    /// it holds is set to `Unrolled` is not counted, because the setting is
-    /// the reason.
+    /// covers the whole run.
     #[non_exhaustive]
     TableCodecFallbackSummary {
         /// The number of messages that fell back.
         fallbacks: usize,
-        /// The number of messages selected for the table, not counting those
-        /// kept unrolled only by a message the user set to `Unrolled`.
+        /// The number of messages selected for the table.
         selected: usize,
         /// Why they fell back, most common reason first.
         reasons: Vec<TableCodecFallbackReason>,
@@ -2562,7 +2576,7 @@ impl core::fmt::Display for CodeGenWarning {
                 write!(
                     f,
                     "{fallbacks} of {selected} messages selected for the table codec use the \
-                     unrolled codec instead, which works but is larger"
+                     unrolled codec instead"
                 )?;
                 for (i, r) in reasons.iter().enumerate() {
                     let sep = if i == 0 { ": " } else { "; " };

buffa-codegen/src/table_codec.rs

@@ -196,22 +196,22 @@ fn field_entry(
             ))
         })
     };
-    let type_path = |what: &str| -> Result<String, CodeGenError> {
-        let type_name = field
+    let type_name = || {
+        field
             .type_name
             .as_deref()
-            .ok_or(CodeGenError::MissingField("field.type_name"))?;
+            .ok_or(CodeGenError::MissingField("field.type_name"))
+    };
+    let type_path = |what: &str| -> Result<String, CodeGenError> {
+        let type_name = type_name()?;
         ctx.rust_type_relative(type_name, current_package, nesting)
             .ok_or_else(|| CodeGenError::Other(format!("{what} type '{type_name}' not found")))
     };
 
     // The table is not among the imports that `idiomatic_imports` shortens
     // paths with, so its path is built from the unshortened one.
     let unshortened_path = || -> Result<String, CodeGenError> {
-        let type_name = field
-            .type_name
-            .as_deref()
-            .ok_or(CodeGenError::MissingField("field.type_name"))?;
+        let type_name = type_name()?;
         let split = ctx
             .rust_type_relative_split(type_name, current_package, nesting)
             .ok_or_else(|| CodeGenError::Other(format!("message type '{type_name}' not found")))?;
@@ -225,23 +225,30 @@ fn field_entry(
     match f.ty {
         Type::TYPE_MESSAGE => {
             let child = type_path("message")?;
-            let child_table = table_path(&unshortened_path()?)?;
             let child_ty = rust_path_to_tokens(&child);
+            // A child without a table here is reached through its `Message`
+            // impl.
+            let child_table = if ctx.uses_table_codec(type_name()?) {
+                Some(table_path(&unshortened_path()?)?)
+            } else {
+                None
+            };
             let (slot, aux_item) = if f.card == Card::Repeated {
+                let vt = match &child_table {
+                    Some(table) => quote! { ::buffa::table::RepVt::new::<#child_ty>(&#table) },
+                    None => quote! { ::buffa::table::RepVt::new_via_message::<#child_ty>() },
+                };
                 (
                     quote! { ::buffa::alloc::vec::Vec<#child_ty> },
-                    quote! {
-                        ::buffa::table::Aux::Rep(&::buffa::table::RepVt::new::<#child_ty>(&#child_table))
-                    },
+                    quote! { ::buffa::table::Aux::Rep(&#vt) },
                 )
             } else {
                 let slot = classify_field(scope, msg, field, resolver)?.rust_type;
-                (
-                    slot.clone(),
-                    quote! {
-                        ::buffa::table::Aux::Msg(&::buffa::table::MsgVt::new::<#slot>(&#child_table))
-                    },
-                )
+                let vt = match &child_table {
+                    Some(table) => quote! { ::buffa::table::MsgVt::new::<#slot>(&#table) },
+                    None => quote! { ::buffa::table::MsgVt::new_via_message::<#slot>() },
+                };
+                (slot, quote! { ::buffa::table::Aux::Msg(&#vt) })
             };
             let aux = aux_u16()?;
             Ok((