Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
kind: Added
body: |-
**`Any::pack_message`, `try_pack_message`, `unpack_message`, `is_message` and `type_name`** (#499). The first four pack and match an `Any` by message type in place of a type URL string. `pack_message` and `try_pack_message` store `MessageName::TYPE_URL`. `is_message::<T>()` and `unpack_message::<T>()` compare the name after the last `/` of the URL with `MessageName::FULL_NAME`, so they accept an `Any` packed under any URL prefix; the JSON and text registries match by the same name after trying the exact URL first. `type_name()` returns that name. `is_type` and `unpack_if` still compare the whole URL.
**`Any::pack_message`, `try_pack_message`, `unpack_message`, `is_message` and `type_name`** (#499). The first four pack and match an `Any` by message type in place of a type URL string. `pack_message` and `try_pack_message` store `MessageName::TYPE_URL`. `is_message::<T>()` and `unpack_message::<T>()` compare the name after the last `/` of the URL with `MessageName::FULL_NAME`, so they accept an `Any` packed under any URL prefix; the JSON and text registries match by the same name after trying the exact URL first. `type_name()` returns that name. A URL without a `/` is read as a bare name (#692). `is_type` and `unpack_if` still compare the whole URL.
time: 2026-10-01T23:58:11+02:00
3 changes: 3 additions & 0 deletions .changes/unreleased/added-20261010-101508.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
kind: Added
body: '**`JsonParseOptions::strict_any_type_urls` and `DynamicMessageSeed::strict_any_type_urls`** (#692) reject a `google.protobuf.Any` `@type` that has no `/` when parsing JSON, as C++ and Java do. Both are off by default, and the default accepts a bare message full name that resolves. The first covers generated messages and the second a `DynamicMessage`; each covers an `Any` nested at any depth.'
time: 2026-10-10T10:15:08.176026492-07:00
5 changes: 5 additions & 0 deletions .changes/unreleased/changed-20261010-101508.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
kind: Changed
body: |-
**A `google.protobuf.Any` whose `type_url` is a bare message full name is read as that message** (#692). `any.proto` requires a `/` in a type URL, but protobuf-go and Python resolve a string without one as a full name, and so did `DynamicMessage`. The generated `Any` now does the same. Its JSON parser accepts `{"@type": "pkg.Message", ...}` when the type registry has a JSON entry for `pkg.Message`, `AnyRegistry::lookup` and `TypeRegistry::json_any_by_url` find a registered message by its bare name, and an entry registered under a bare name is found under any URL prefix.
This changes JSON output. An `Any` with a bare name and a registered message was written as `{"@type": "pkg.Message", "value": "<base64>"}`, which the generated parser rejected, and is now written with the fields of the message; a payload that does not decode as that message now fails to serialize. A bare name without a JSON entry is still a parse error, and so is an `@type` that is empty or ends in `/`. Text format is unchanged: it does not resolve a bare name to a registered message, because `[pkg.Message] { ... }` is the syntax of an extension field.
time: 2026-10-10T10:15:08.164120259-07:00
3 changes: 2 additions & 1 deletion buffa-descriptor/src/reflect/dynamic.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1510,7 +1510,8 @@ impl DynamicMessage {
/// Decode the message wrapped in this `google.protobuf.Any`.
///
/// Reads `type_url` (field 1), resolves the type name (the segment after
/// the last `/`) against this message's pool, and decodes `value`
/// the last `/`, or the whole string when it has no `/`) against this
/// message's pool, and decodes `value`
/// (field 2) into a [`DynamicMessage`] of that type. This is the binary
/// counterpart of the `Any` JSON `@type` expansion; CEL `dyn` evaluation
/// unpacks `Any` values this way.
Expand Down
80 changes: 51 additions & 29 deletions buffa-descriptor/src/reflect/json.rs
Original file line number Diff line number Diff line change
Expand Up @@ -556,7 +556,7 @@ impl DynamicMessage {
pub struct DynamicMessageSeed {
pool: Arc<DescriptorPool>,
msg_idx: MessageIndex,
ignore_unknown: bool,
flags: ParseFlags,
element_memory_limit: usize,
}

Expand All @@ -567,7 +567,7 @@ impl DynamicMessageSeed {
Self {
pool,
msg_idx,
ignore_unknown: false,
flags: ParseFlags::default(),
element_memory_limit: buffa::DEFAULT_ELEMENT_MEMORY_LIMIT,
}
}
Expand All @@ -583,7 +583,25 @@ impl DynamicMessageSeed {
/// [element-memory limit](Self::with_element_memory_limit).
#[must_use]
pub fn ignore_unknown_fields(mut self, ignore: bool) -> Self {
self.ignore_unknown = ignore;
self.flags.ignore_unknown = ignore;
self
}

/// Require a `/` in the `@type` of a `google.protobuf.Any` (default:
/// accept a bare full name too).
///
/// By default `"@type": "my.pkg.Request"` resolves like
/// `"@type": "type.googleapis.com/my.pkg.Request"`, as it does in
/// protobuf-go and Python. With this set, an `@type` without a `/` is a
/// parse error, as it is in C++ and Java; `any.proto` says that a type
/// URL contains at least one `/`. The setting propagates to every nested
/// `Any`, including one inside another `Any`'s payload.
///
/// The setting covers parsing only: [`DynamicMessage::to_json`] and
/// [`DynamicMessage::unpack_any`] resolve a bare full name.
#[must_use]
pub fn strict_any_type_urls(mut self, strict: bool) -> Self {
self.flags.strict_any_type_urls = strict;
self
}

Expand Down Expand Up @@ -706,13 +724,23 @@ impl<'de> DeserializeSeed<'de> for DynamicMessageSeed {
NestedSeed {
pool: self.pool,
msg_idx: self.msg_idx,
ignore_unknown: self.ignore_unknown,
flags: self.flags,
budget: &budget,
}
.deserialize(d)
}
}

/// The parse options that [`DynamicMessageSeed`] sets and every nested seed
/// carries.
#[derive(Clone, Copy, Default)]
struct ParseFlags {
/// Discard unknown fields instead of erroring.
ignore_unknown: bool,
/// Reject an `Any` `@type` that has no `/`.
strict_any_type_urls: bool,
}

/// The internal twin of [`DynamicMessageSeed`] that borrows the parse's
/// element-memory budget instead of owning a limit.
///
Expand All @@ -722,7 +750,7 @@ impl<'de> DeserializeSeed<'de> for DynamicMessageSeed {
struct NestedSeed<'a> {
pool: Arc<DescriptorPool>,
msg_idx: MessageIndex,
ignore_unknown: bool,
flags: ParseFlags,
budget: &'a Cell<usize>,
}

Expand All @@ -732,18 +760,12 @@ impl<'de> DeserializeSeed<'de> for NestedSeed<'_> {
fn deserialize<D: Deserializer<'de>>(self, d: D) -> Result<Self::Value, D::Error> {
let md = self.pool.message(self.msg_idx);
if let Some(wkt) = WktKind::from_full_name(&md.full_name) {
return wkt.deserialize_message(
self.pool,
self.msg_idx,
d,
self.ignore_unknown,
self.budget,
);
return wkt.deserialize_message(self.pool, self.msg_idx, d, self.flags, self.budget);
}
d.deserialize_map(MessageVisitor {
pool: self.pool,
msg_idx: self.msg_idx,
ignore_unknown: self.ignore_unknown,
flags: self.flags,
budget: self.budget,
})
}
Expand All @@ -752,7 +774,7 @@ impl<'de> DeserializeSeed<'de> for NestedSeed<'_> {
struct MessageVisitor<'a> {
pool: Arc<DescriptorPool>,
msg_idx: MessageIndex,
ignore_unknown: bool,
flags: ParseFlags,
budget: &'a Cell<usize>,
}

Expand Down Expand Up @@ -814,7 +836,7 @@ impl<'de> Visitor<'de> for MessageVisitor<'_> {
// lenient mode. There is no descriptor to deduplicate
// against, and the spec's no-duplicates rule is in terms
// of fields, not arbitrary keys.
if self.ignore_unknown {
if self.flags.ignore_unknown {
map.next_value::<de::IgnoredAny>()?;
continue;
}
Expand Down Expand Up @@ -842,7 +864,7 @@ impl<'de> Visitor<'de> for MessageVisitor<'_> {
pool: &self.pool,
kind,
enum_type,
ignore_unknown: self.ignore_unknown,
flags: self.flags,
budget: self.budget,
})?;
// null → leave the field unset (per spec, except NullValue which
Expand Down Expand Up @@ -904,7 +926,7 @@ struct FieldSeed<'a> {
pool: &'a Arc<DescriptorPool>,
kind: FieldKind,
enum_type: Option<EnumType>,
ignore_unknown: bool,
flags: ParseFlags,
budget: &'a Cell<usize>,
}

Expand All @@ -920,23 +942,23 @@ impl<'de> DeserializeSeed<'de> for FieldSeed<'_> {
pool: self.pool,
kind: sk,
enum_type: self.enum_type,
ignore_unknown: self.ignore_unknown,
flags: self.flags,
budget: self.budget,
}
.deserialize(d),
FieldKind::List(sk) => d.deserialize_any(ListVisitor {
pool: self.pool,
kind: sk,
enum_type: self.enum_type,
ignore_unknown: self.ignore_unknown,
flags: self.flags,
budget: self.budget,
}),
FieldKind::Map { key, value } => d.deserialize_any(MapFieldVisitor {
pool: self.pool,
key,
value,
enum_type: self.enum_type,
ignore_unknown: self.ignore_unknown,
flags: self.flags,
budget: self.budget,
}),
}
Expand All @@ -947,7 +969,7 @@ struct SingularSeed<'a> {
pool: &'a Arc<DescriptorPool>,
kind: SingularKind,
enum_type: Option<EnumType>,
ignore_unknown: bool,
flags: ParseFlags,
budget: &'a Cell<usize>,
}

Expand All @@ -966,7 +988,7 @@ impl<'de> DeserializeSeed<'de> for SingularSeed<'_> {
return NestedSeed {
pool: Arc::clone(self.pool),
msg_idx: midx,
ignore_unknown: self.ignore_unknown,
flags: self.flags,
budget: self.budget,
}
.deserialize(d)
Expand All @@ -976,7 +998,7 @@ impl<'de> DeserializeSeed<'de> for SingularSeed<'_> {
d.deserialize_option(NestedMessageVisitor {
pool: self.pool,
midx,
ignore_unknown: self.ignore_unknown,
flags: self.flags,
budget: self.budget,
})
}
Expand Down Expand Up @@ -1011,7 +1033,7 @@ fn deserialize_optional_scalar<'de, D: Deserializer<'de>>(
struct NestedMessageVisitor<'a> {
pool: &'a Arc<DescriptorPool>,
midx: MessageIndex,
ignore_unknown: bool,
flags: ParseFlags,
budget: &'a Cell<usize>,
}

Expand All @@ -1034,7 +1056,7 @@ impl<'de> Visitor<'de> for NestedMessageVisitor<'_> {
NestedSeed {
pool: Arc::clone(self.pool),
msg_idx: self.midx,
ignore_unknown: self.ignore_unknown,
flags: self.flags,
budget: self.budget,
}
.deserialize(d)
Expand Down Expand Up @@ -1238,7 +1260,7 @@ struct ListVisitor<'a> {
pool: &'a Arc<DescriptorPool>,
kind: SingularKind,
enum_type: Option<EnumType>,
ignore_unknown: bool,
flags: ParseFlags,
budget: &'a Cell<usize>,
}

Expand All @@ -1256,7 +1278,7 @@ impl<'de> Visitor<'de> for ListVisitor<'_> {
pool: self.pool,
kind: self.kind,
enum_type: self.enum_type,
ignore_unknown: self.ignore_unknown,
flags: self.flags,
budget: self.budget,
})? {
// Per the spec, repeated fields cannot contain null elements.
Expand All @@ -1276,7 +1298,7 @@ struct MapFieldVisitor<'a> {
key: ScalarType,
value: SingularKind,
enum_type: Option<EnumType>,
ignore_unknown: bool,
flags: ParseFlags,
budget: &'a Cell<usize>,
}

Expand All @@ -1298,7 +1320,7 @@ impl<'de> Visitor<'de> for MapFieldVisitor<'_> {
pool: self.pool,
kind: self.value,
enum_type: self.enum_type,
ignore_unknown: self.ignore_unknown,
flags: self.flags,
budget: self.budget,
})?;
let v = v.ok_or_else(|| de::Error::custom("null value in map field"))?;
Expand Down
35 changes: 22 additions & 13 deletions buffa-descriptor/src/reflect/json_wkt.rs
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ impl WktKind {
pool: Arc<DescriptorPool>,
midx: MessageIndex,
d: D,
ignore_unknown: bool,
flags: ParseFlags,
budget: &Cell<usize>,
) -> Result<DynamicMessage, D::Error> {
// Two WKTs consult `ignore_unknown`: `Any` recurses into a
Expand All @@ -131,7 +131,7 @@ impl WktKind {
// scalar (a type error, never an unknown field) or are open schemas
// that accept any member by construction.
match self {
Self::Any => deserialize_any(pool, midx, d, ignore_unknown, budget),
Self::Any => deserialize_any(pool, midx, d, flags, budget),
Self::Timestamp => {
let s: String = String::deserialize(d)?;
let (secs, nanos) = parse_rfc3339(&s).map_err(de::Error::custom)?;
Expand Down Expand Up @@ -191,7 +191,9 @@ impl WktKind {
Ok(())
}
}
d.deserialize_map(EmptyVisitor { ignore_unknown })?;
d.deserialize_map(EmptyVisitor {
ignore_unknown: flags.ignore_unknown,
})?;
Ok(DynamicMessage::new(pool, midx))
}
Self::Wrapper(sc) => {
Expand Down Expand Up @@ -399,14 +401,15 @@ impl<'de> Visitor<'de> for StructVisitor<'_> {
.ok_or_else(|| de::Error::custom("Value not in pool"))?;
let mut fields: Vec<(MapKey, Value)> = Vec::new();
while let Some(key) = map.next_key::<String>()? {
// The Value seed deliberately doesn't carry `ignore_unknown`:
// The Value seed deliberately doesn't carry the parse flags:
// `google.protobuf.Value` is a closed schema (null/bool/number/
// string/Struct/ListValue) that cannot recurse into a
// user-defined message where unknown fields could appear.
// user-defined message where unknown fields could appear, or
// into an `Any`.
let v = map.next_value_seed(NestedSeed {
pool: Arc::clone(&self.pool),
msg_idx: value_idx,
ignore_unknown: false,
flags: ParseFlags::default(),
budget: self.budget,
})?;
// `Struct.fields` is `map<string, Value>`.
Expand Down Expand Up @@ -449,11 +452,11 @@ impl<'de> Visitor<'de> for ListValueVisitor<'_> {
.ok_or_else(|| de::Error::custom("Value not in pool"))?;
let mut items = Vec::new();
// See `StructVisitor::visit_map` for why the Value seed doesn't
// carry `ignore_unknown`.
// carry the parse flags.
while let Some(v) = seq.next_element_seed(NestedSeed {
pool: Arc::clone(&self.pool),
msg_idx: value_idx,
ignore_unknown: false,
flags: ParseFlags::default(),
budget: self.budget,
})? {
// `ListValue.values` is `repeated Value`.
Expand Down Expand Up @@ -559,7 +562,7 @@ fn deserialize_any<'de, D: Deserializer<'de>>(
pool: Arc<DescriptorPool>,
midx: MessageIndex,
d: D,
ignore_unknown: bool,
flags: ParseFlags,
budget: &Cell<usize>,
) -> Result<DynamicMessage, D::Error> {
use buffa::json_helpers::buffered;
Expand All @@ -580,6 +583,11 @@ fn deserialize_any<'de, D: Deserializer<'de>>(
let Some(serde_json::Value::String(type_url)) = obj.remove("@type") else {
return Err(D::Error::custom("Any object missing string \"@type\""));
};
if flags.strict_any_type_urls && !type_url.contains('/') {
return Err(D::Error::custom(format!(
"Any type_url {type_url:?} must contain a '/' (e.g. type.googleapis.com/pkg.Type)"
)));
}
let Some(inner_idx) = resolve_any_type(&pool, &type_url) else {
return Err(D::Error::custom(format!(
"Any type_url {type_url:?} not registered in the descriptor pool"
Expand All @@ -596,7 +604,7 @@ fn deserialize_any<'de, D: Deserializer<'de>>(
"Any with WKT type {type_url:?} requires a \"value\" key"
))
})?;
if !ignore_unknown {
if !flags.ignore_unknown {
if let Some(key) = obj.keys().next() {
return Err(D::Error::custom(format!(
"unknown field {key:?} in Any wrapper for {type_url:?}"
Expand All @@ -616,7 +624,7 @@ fn deserialize_any<'de, D: Deserializer<'de>>(
let inner = NestedSeed {
pool: Arc::clone(&pool),
msg_idx: inner_idx,
ignore_unknown,
flags,
budget,
}
.deserialize(inner_json)
Expand Down Expand Up @@ -645,7 +653,7 @@ fn deserialize_any<'de, D: Deserializer<'de>>(
_pool: Arc<DescriptorPool>,
_midx: MessageIndex,
_d: D,
_ignore_unknown: bool,
_flags: ParseFlags,
_budget: &Cell<usize>,
) -> Result<DynamicMessage, D::Error> {
Err(de::Error::custom(
Expand All @@ -654,7 +662,8 @@ fn deserialize_any<'de, D: Deserializer<'de>>(
}

/// Resolve a `type_url` to a [`MessageIndex`]. Accepts `type.googleapis.com/`
/// and any other prefix; the type name is the segment after the last `/`.
/// and any other prefix; the type name is the segment after the last `/`, or
/// the whole string when it has no `/`.
fn resolve_any_type(pool: &DescriptorPool, type_url: &str) -> Option<MessageIndex> {
let name = type_url.rsplit('/').next()?;
pool.message_index(name)
Expand Down
Loading
Loading