table: use the table for messages with map fields

.changes/unreleased/added-20260923-codec-strategy-table.yaml

@@ -1,4 +1,4 @@
 kind: Added
 body: |-
-    **Table-driven message codec** (#469, refs #463). `buffa_build::Config::codec_strategy(CodecStrategy::Table)` (plugin option `codec_strategy=table`, `CodeGenConfig::codec_strategy`) generates each message's binary `Message` implementation from a static table and interpreters that every message shares, instead of code specialised to the message's fields. The compiled code is substantially smaller, and messages made of many small fields are slower. `CodecStrategy::Unrolled` stays the default. `codec_strategy_in(strategy, &[paths])` (plugin option `codec_strategy_in=<path>=<strategy>`, repeatable) chooses the strategy for matching messages and the messages nested in them, and the last matching rule wins. The wire format does not change, and a message with a `oneof` uses the table, including when a member is a message that has no table. Messages the table cannot handle stay unrolled, and `CodeGenWarning::TableCodecFallbackSummary` counts them. A rule that selects the table, by exact path, for a message that cannot use it is an error. The generated code needs Rust 1.77 or later, which `buffa-build` checks, and compiles in a crate with `#![forbid(unsafe_code)]`. The guide's "Smaller generated code" section has the measurements, the messages that stay unrolled and the behavioural differences.
+    **Table-driven message codec** (#469, refs #463). `buffa_build::Config::codec_strategy(CodecStrategy::Table)` (plugin option `codec_strategy=table`, `CodeGenConfig::codec_strategy`) generates each message's binary `Message` implementation from a static table and interpreters that every message shares, instead of code specialised to the message's fields. The compiled code is substantially smaller, and messages made of many small fields are slower. `CodecStrategy::Unrolled` stays the default. `codec_strategy_in(strategy, &[paths])` (plugin option `codec_strategy_in=<path>=<strategy>`, repeatable) chooses the strategy for matching messages and the messages nested in them, and the last matching rule wins. The wire format does not change, and a message with a `oneof` or a `map` field uses the table, including when a member or value is a message that has no table. Messages the table cannot handle stay unrolled, and `CodeGenWarning::TableCodecFallbackSummary` counts them. A rule that selects the table, by exact path, for a message that cannot use it is an error. The generated code needs Rust 1.77 or later, which `buffa-build` checks, and compiles in a crate with `#![forbid(unsafe_code)]`. The guide's "Smaller generated code" section has the measurements, the messages that stay unrolled and the behavioural differences.
 time: 2026-09-23T03:10:00+00:00

DESIGN.md

@@ -560,7 +560,7 @@ Within that bound, the shortcuts do not pay off. `merge` does not help: it consu
 
 The size, write, and merge code of an unrolled message is specialised to its fields. `CodecStrategy::Table` (a per-message option, `Unrolled` by default) replaces it with a static `buffa::table::Table<M>` and a `Message` impl that forwards to interpreters in `buffa::table`. The decision to keep `Unrolled` as the default is in [#463](https://github.com/anthropics/buffa/issues/463), and the guide's "Smaller generated code" section has the measurements and the list of messages that stay unrolled.
 
-A table holds a sorted array of 12-byte entries `{tag, offset, kind, tag_len, aux}`, a dense array that maps field numbers below 64 to entries, and the offset of the unknown-fields slot. `kind` is the field type crossed with its cardinality, so the interpreter dispatches once per field. Message, repeated-message, and enum fields carry a small descriptor (`Aux`) with the accessors that their storage needs, because a `MessageField`, a `Vec`, and an `EnumValue` cannot be read through an offset alone; a oneof has a `Group` descriptor and each of its members a `Member` one. Offsets come from `core::mem::offset_of!`, so the table needs Rust 1.77 and the generated code refers to it through `buffa::__table!`, which is a compile error on an older compiler.
+A table holds a sorted array of 12-byte entries `{tag, offset, kind, tag_len, aux}`, a dense array that maps field numbers below 64 to entries, and the offset of the unknown-fields slot. `kind` is the field type crossed with its cardinality, so the interpreter dispatches once per field. Message, repeated-message, and enum fields carry a small descriptor (`Aux`) with the accessors that their storage needs, because a `MessageField`, a `Vec`, and an `EnumValue` cannot be read through an offset alone; a oneof has a `Group` descriptor and each of its members a `Member` one. A map field is one entry of kind `Map`, whose descriptor (`MapVt`) instantiates only the iteration of the collection and the insertion of a decoded entry per collection type; `buffa/src/table/map.rs` describes the mechanism. Offsets come from `core::mem::offset_of!`, so the table needs Rust 1.77 and the generated code refers to it through `buffa::__table!`, which is a compile error on an older compiler.
 
 Four decisions shape the runtime:
 

buffa-codegen/src/lib.rs

@@ -1094,23 +1094,25 @@ pub enum CodecStrategy {
     ///
     /// Not every message can use it. These stay [`Unrolled`](Self::Unrolled):
     ///
-    /// - a message with a `map` field or a group field;
+    /// - a message with a group field;
     /// - the message type of a group field;
     /// - a message that uses the `MessageSet` wire format;
     /// - a message with extension ranges, when JSON code is generated and
     ///   unknown fields are preserved;
     /// - a message with a field of a non-default string, bytes, or collection
-    ///   type, such as `use_bytes_type`, `string_type`, `bytes_type`, and
-    ///   `repeated_type` select;
+    ///   type, such as `use_bytes_type`, `string_type`, `bytes_type`,
+    ///   `repeated_type`, and a custom `map_type` select. `HashMap` and
+    ///   `BTreeMap` maps are supported; a `map` whose keys or values are of a
+    ///   non-default string or bytes type is not;
     /// - a message that holds, in a singular, repeated, `oneof`, or map value
     ///   field, a message of the same run that has a `bytes` field of a
     ///   non-default type, or that holds one. The table decodes from one
     ///   contiguous slice, so it would copy `bytes::Bytes` fields that
     ///   unrolled code decoding from a `Bytes` shares with the input.
     ///
     /// A table message may hold any other message, including as a member of a
-    /// `oneof`. It reaches a child that is a table message through the
-    /// child's table, and any other child, whether it is
+    /// `oneof` and as a `map` value. It reaches a child that is a table
+    /// message through the child's table, and any other child, whether it is
     /// [`Unrolled`](Self::Unrolled), generated by another crate, or a
     /// well-known type, through the child's `Message` impl, which costs a
     /// function call per child. Selecting a message does not select the
@@ -1124,7 +1126,8 @@ pub enum CodecStrategy {
     /// different error for the same rejected input; a child reached through its
     /// `Message` impl is read from the slice of the nearest enclosing table
     /// message, so it is bounded there. `clear()` resets the message to its
-    /// default, which releases the capacity of its strings and vectors.
+    /// default, which releases the capacity of its strings, vectors and maps,
+    /// where unrolled code keeps it.
     /// Decoding gathers a `Buf` that is not contiguous into one buffer first.
     /// Encoding into any sink other than the cursor that `Message::encode` and
     /// its siblings write a `BufMut` through, meaning a `Rope`, a sink defined
@@ -1157,7 +1160,7 @@ pub enum CodecStrategy {
 #[derive(Debug, Clone, PartialEq, Eq)]
 #[non_exhaustive]
 pub struct TableCodecFallbackReason {
-    /// The reason as a predicate, such as `has a map field`. The wording is for
+    /// The reason as a predicate, such as `has a group field`. The wording is for
     /// people and may change between releases.
     pub reason: String,
     /// The proto paths of all of them, with a leading dot, in declaration

buffa-codegen/src/table_codec.rs

@@ -14,7 +14,9 @@ use crate::generated::descriptor::field_descriptor_proto::Type;
 use crate::generated::descriptor::{DescriptorProto, FieldDescriptorProto};
 use crate::idents::rust_path_to_tokens;
 use crate::message::classify_field;
-use crate::table_plan::{table_fields, Card, OneofMembership, TableField};
+use crate::table_plan::{
+    table_fields, type_stem, Card, MapField, OneofMembership, PlainField, Shape, TableField,
+};
 use crate::CodeGenError;
 
 /// The name of the static table of the message struct `rust_name`.
@@ -77,8 +79,14 @@ pub(crate) fn generate_table_impl(
     let mut aux: Vec<TokenStream> = Vec::new();
     let mut oneofs = Oneofs::new(scope, msg, &fields)?;
     for f in &fields {
-        if let Some(member) = &f.oneof {
-            entries.push(oneofs.member_entry(scope, &name, f, member, &mut aux)?);
+        if let Shape::Plain(
+            plain @ PlainField {
+                oneof: Some(member),
+                ..
+            },
+        ) = &f.shape
+        {
+            entries.push(oneofs.member_entry(scope, &name, f, plain, member, &mut aux)?);
             continue;
         }
         let (entry, aux_item) = field_entry(scope, msg, &name, f, aux.len(), resolver)?;
@@ -191,15 +199,30 @@ fn field_entry(
     let field = f.field;
     let field_name = field.name.as_deref().unwrap_or("");
     let ident = ctx.field_ident(field_name, field.number.unwrap_or(0));
-    let kind = format_ident!("{}", f.kind);
     let number = f.number;
 
     let aux_u16 = || u16::try_from(aux_index).map_err(|_| too_many_descriptors(scope));
     let type_path = |what: &str| type_path(scope, field, what);
     let unshortened_path = || unshortened_path(scope, field);
     let type_name = || field_type_name(field);
 
-    match f.ty {
+    let plain = match &f.shape {
+        Shape::Map(map) => {
+            let aux = aux_u16()?;
+            let map_ty = classify_field(scope, msg, field, resolver)?.rust_type;
+            let vt = map_descriptor(scope, &map_ty, map)?;
+            return Ok((
+                quote! {
+                    ::buffa::__table_entry!(#name, #ident, Map, #number, aux = #aux, slot = #map_ty)
+                },
+                Some(quote! { ::buffa::table::Aux::Map(&#vt) }),
+            ));
+        }
+        Shape::Plain(plain) => plain,
+    };
+    let kind = format_ident!("{}", plain.kind);
+
+    match plain.ty {
         Type::TYPE_MESSAGE => {
             let child = type_path("message")?;
             let child_ty = rust_path_to_tokens(&child);
@@ -210,7 +233,7 @@ fn field_entry(
             } else {
                 None
             };
-            let (slot, aux_item) = if f.card == Card::Repeated {
+            let (slot, aux_item) = if plain.card == Card::Repeated {
                 let vt = match &child_table {
                     Some(table) => quote! { ::buffa::table::RepVt::new::<#child_ty>(&#table) },
                     None => quote! { ::buffa::table::RepVt::new_via_message::<#child_ty>() },
@@ -237,8 +260,8 @@ fn field_entry(
         }
         Type::TYPE_ENUM => {
             let enum_ty = rust_path_to_tokens(&type_path("enum")?);
-            let repeated = matches!(f.card, Card::Repeated | Card::Packed);
-            let shape = match (repeated, f.card == Card::Optional, f.closed_enum) {
+            let repeated = matches!(plain.card, Card::Repeated | Card::Packed);
+            let shape = match (repeated, plain.card == Card::Optional, plain.closed_enum) {
                 (true, _, true) => quote! { RepeatedClosed },
                 (true, _, false) => quote! { RepeatedOpen },
                 (false, true, true) => quote! { OptionalClosed },
@@ -268,6 +291,57 @@ fn field_entry(
     }
 }
 
+/// The `MapVt` expression for the map field of Rust type `map_ty`.
+fn map_descriptor(
+    scope: MessageScope<'_>,
+    map_ty: &TokenStream,
+    map: &MapField<'_>,
+) -> Result<TokenStream, CodeGenError> {
+    let marker = |ty: Type, slot: &str| -> Result<TokenStream, CodeGenError> {
+        match type_stem(ty, Card::Required) {
+            Some(stem) if !matches!(ty, Type::TYPE_ENUM | Type::TYPE_MESSAGE) => {
+                let kind = format_ident!("{}Required", stem);
+                Ok(quote! { ::buffa::table::kinds::#kind })
+            }
+            _ => Err(CodeGenError::Other(format!(
+                "table codec: a map {slot} of type {ty:?} has no scalar kind"
+            ))),
+        }
+    };
+    let key = marker(map.key_ty, "key")?;
+    Ok(match map.val_ty {
+        Type::TYPE_ENUM => {
+            let enum_ty = rust_path_to_tokens(&type_path(scope, map.val_field, "enum")?);
+            let shape = if map.closed_enum {
+                quote! { ImplicitClosed }
+            } else {
+                quote! { ImplicitOpen }
+            };
+            quote! {
+                ::buffa::table::MapVt::with_enum::<#map_ty, #key, ::buffa::table::#shape<#enum_ty>>()
+            }
+        }
+        Type::TYPE_MESSAGE => {
+            let child = rust_path_to_tokens(&type_path(scope, map.val_field, "message")?);
+            // A value without a table here is reached through its `Message`
+            // impl.
+            let msg_vt = if scope.ctx.uses_table_codec(field_type_name(map.val_field)?) {
+                let child_table = table_path(&unshortened_path(scope, map.val_field)?)?;
+                quote! { ::buffa::table::DirectMsgVt::new(&#child_table) }
+            } else {
+                quote! { ::buffa::table::DirectMsgVt::<#child>::via_message() }
+            };
+            quote! {
+                ::buffa::table::MapVt::with_msg::<#map_ty, #key, #child>(&#msg_vt)
+            }
+        }
+        ty => {
+            let value = marker(ty, "value")?;
+            quote! { ::buffa::table::MapVt::new::<#map_ty, #key, #value>() }
+        }
+    })
+}
+
 /// The proto path of the message or enum type of `field`.
 fn field_type_name(field: &FieldDescriptorProto) -> Result<&str, CodeGenError> {
     field
@@ -355,13 +429,13 @@ impl Oneofs {
     ) -> Result<Self, CodeGenError> {
         let mut first: HashMap<usize, u32> = HashMap::new();
         let mut with_messages = HashSet::new();
-        for (oneof, f) in fields
-            .iter()
-            .filter_map(|f| f.oneof.as_ref().map(|oneof| (oneof, f)))
-        {
+        for (oneof, f, plain) in fields.iter().filter_map(|f| match &f.shape {
+            Shape::Plain(plain) => plain.oneof.as_ref().map(|oneof| (oneof, f, plain)),
+            Shape::Map(_) => None,
+        }) {
             let lowest = first.entry(oneof.index).or_insert(f.number);
             *lowest = (*lowest).min(f.number);
-            if f.ty == Type::TYPE_MESSAGE {
+            if plain.ty == Type::TYPE_MESSAGE {
                 with_messages.insert(oneof.index);
             }
         }
@@ -399,6 +473,7 @@ impl Oneofs {
         scope: MessageScope<'_>,
         message: &proc_macro2::Ident,
         f: &TableField<'_>,
+        plain: &PlainField<'_>,
         member: &OneofMembership<'_>,
         aux: &mut Vec<TokenStream>,
     ) -> Result<TokenStream, CodeGenError> {
@@ -417,7 +492,7 @@ impl Oneofs {
         let oneof_field = ctx.oneof_ident(member.name);
         let variant = crate::oneof::oneof_variant_ident(field_name);
         let number = f.number;
-        let payload_kind = format_ident!("{}", f.kind);
+        let payload_kind = format_ident!("{}", plain.kind);
         let first = self.first[&member.index];
 
         // The oneof's descriptor, made when its first member is met.
@@ -445,7 +520,7 @@ impl Oneofs {
         // is first set.
         let variant_fqn = format!(".{}.{}.{field_name}", scope.proto_fqn, member.name);
         let default = quote! { ::core::default::Default::default() };
-        let (slot, value_aux, new) = match f.ty {
+        let (slot, value_aux, new) = match plain.ty {
             Type::TYPE_MESSAGE => {
                 let child = rust_path_to_tokens(&type_path(scope, field, "message")?);
                 // A child without a table here is reached through its
@@ -456,7 +531,7 @@ impl Oneofs {
                 } else {
                     quote! { ::buffa::table::MsgVt::direct_via_message::<#child>() }
                 };
-                let new = if crate::oneof::variant_boxed(ctx, f.ty, &variant_fqn) {
+                let new = if crate::oneof::variant_boxed(ctx, plain.ty, &variant_fqn) {
                     match ctx.pointer_repr(&variant_fqn) {
                         crate::PointerRepr::Box => quote! { ::buffa::alloc::boxed::Box::default() },
                         repr => repr.pointer_new(&child, &default)?,
@@ -472,7 +547,7 @@ impl Oneofs {
             }
             Type::TYPE_ENUM => {
                 let enum_ty = rust_path_to_tokens(&type_path(scope, field, "enum")?);
-                let shape = if f.closed_enum {
+                let shape = if plain.closed_enum {
                     quote! { ::buffa::table::ImplicitClosed<#enum_ty> }
                 } else {
                     quote! { ::buffa::table::ImplicitOpen<#enum_ty> }