Repository navigation
Commit 4641a3c
codegen: floor descriptor_pool()'s scaled element-memory bound at the default (#360)
Fixes #336.
## The problem
Generated `descriptor_pool()` scales its decode bound to
`FILE_DESCRIPTOR_SET_BYTES.len() * 64`. For descriptor sets built mostly
from short single-character type names, the real per-byte amplification
runs above 64x — up to 352x for an empty `FileDescriptorProto` element,
which is the tightest possible case for `descriptor.proto` on 64-bit
targets. So the scaled bound can come out *smaller* than the flat 32 MiB
`DEFAULT_ELEMENT_MEMORY_LIMIT` it replaced. The issue's own trigger
shape
(50 outer messages × 26 one-character nested messages) reproduces this:
it's well under 1 KiB, needs ~71.5x to decode, and the 64x-scaled bound
rejects it even though the pre-0.9.1 flat default accepted it fine.
(Small correction while reproducing: the issue's own repro table reports
68-69x for this shape; the actual measured ratio is 71.5x. Traced to a
`format!("{:0width$}", i, width=1)` name-padding bug in whatever
generated
that table, not a discrepancy in the shape itself — doesn't change the
conclusion, just flagging it since the corrected number is what this
PR's
test locks in.)
## The fix
```rust
FILE_DESCRIPTOR_SET_BYTES
.len()
.saturating_mul(64)
.max(::buffa::DEFAULT_ELEMENT_MEMORY_LIMIT)
```
Floors the scaled bound at the old default. This makes the emitted bound
provably monotonic relative to what it replaced — it can only loosen,
never tighten — which is the specific property missing today.
This intentionally does **not** try to resolve whether 64x is the right
multiplier in general. It isn't a proven bound; 352x is, for
`descriptor.proto` on 64-bit targets, today (32-bit wasn't measured and
would need re-deriving — `size_of` scales with pointer width). Landing
the
floor now doesn't foreclose adopting 352, or an exact/trusted-input
budget, later; all of those remain `.max()`-compatible with this floor
or
supersede it cleanly.
Worth noting: @iainmcgin's own PR description for #332 — the commit that
introduced this bound — already named the real fix and deliberately
deferred it:
> the durable fix is a bound expressed as a multiple of the encoded
input
> length, which would need no per-call-site exemption and would
> self-tune... That is an API change to `DecodeOptions` touching the
> untrusted-input posture, so it does not belong in a regression fix.
This PR doesn't attempt that larger API change either — it's scoped the
same way #332 was, to the narrow regression this issue reports. Picking
a
tighter proven multiplier (or the exact-budget/trusted-input
alternatives)
is a separate, larger decision for whoever owns that tradeoff.
## What else this touches
`buffa-types/src/generated/google.protobuf.mod.rs` is regenerated via
`task gen-wkt-types` so the bundled well-known-types pool picks up the
same fix — confirmed the regen is idempotent (ran it twice, diffed the
output directly, byte-identical).
## Testing
New `buffa-descriptor/tests/issue_336_floor.rs`:
- `old_formula_rejects_the_trigger_shape` — documents the bug on the
issue's own trigger shape.
- `floored_formula_accepts_the_trigger_shape` — proves the fix on that
same shape, and that it's the floor (not the scaled term) doing the
work.
- `floor_is_monotonic_never_below_default` — sweeps several input
lengths
proving the floor never dips below the default, and checks the
crossover point (where scaling starts to dominate the floor) lands at
exactly 512 KiB, matching the issue's own math.
Strengthened the existing `reflect_pool_module_emits_well_formed_tokens`
codegen test to assert the emitted tokens reference
`DEFAULT_ELEMENT_MEMORY_LIMIT` — it previously only checked that the
module parses and mentions the bytes constant, so it wouldn't have
caught
a future edit silently dropping the floor.
`cargo test --workspace`, `cargo fmt --all --check`, and `cargo clippy
--workspace --all-targets -- -D warnings` all pass.
---------
Co-authored-by: Iain McGinniss <309153+iainmcgin@users.noreply.github.com>1 parent 2827cbf commit 4641a3c
4 files changed
Lines changed: 102 additions & 2 deletions
File tree
- .changes/unreleased
- buffa-codegen/src
- buffa-descriptor/tests
- buffa-types/src/generated
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
209 | 209 | | |
210 | 210 | | |
211 | 211 | | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
212 | 216 | | |
213 | 217 | | |
214 | 218 | | |
| |||
222 | 226 | | |
223 | 227 | | |
224 | 228 | | |
225 | | - | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
226 | 233 | | |
227 | 234 | | |
228 | 235 | | |
| |||
488 | 495 | | |
489 | 496 | | |
490 | 497 | | |
| 498 | + | |
| 499 | + | |
| 500 | + | |
| 501 | + | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
491 | 508 | | |
492 | 509 | | |
493 | 510 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
204 | 204 | | |
205 | 205 | | |
206 | 206 | | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
0 commit comments