feat: add blob encoding benchmark and profiling harness#3085
feat: add blob encoding benchmark and profiling harness#3085
Conversation
wbbradley
force-pushed
the
wbbradley/profiling-encoding
branch
from
89f3b76 to
19db448
Compare
mlegner
left a comment
mlegner
left a comment
There was a problem hiding this comment.
Thanks a lot for expanding our benchmarking/profiling toolbox. A few questions mainly about code duplication.
Verified compilation via
chk(formatting, clippy).
Where is that chk defined?
| [[bench]] | ||
| name = "blob_encoding" | ||
| harness = false | ||
|
|
||
| [[bench]] | ||
| name = "encoding_phases" | ||
| harness = false |
There was a problem hiding this comment.
Question: What is the relationship to the existing benchmarks? Can we combine them?
| fn parse_size(s: &str) -> Result<usize, String> { | ||
| let s = s.to_lowercase(); | ||
| let (num, mult) = if let Some(n) = s.strip_suffix('g') { | ||
| (n, 1 << 30) | ||
| } else if let Some(n) = s.strip_suffix('m') { | ||
| (n, 1 << 20) | ||
| } else if let Some(n) = s.strip_suffix('k') { | ||
| (n, 1 << 10) | ||
| } else { | ||
| (s.as_str(), 1) | ||
| }; | ||
| let n: usize = num.parse().map_err(|e| format!("invalid size: {e}"))?; | ||
| Ok(n * mult) | ||
| } |
There was a problem hiding this comment.
Hint: We already have a struct that does this in the walrus-service crate. That could be moved to walrus-core or walrus-utils.
There was a problem hiding this comment.
AFAICT, many of the sub-benchmarks here mostly copy some code from crates/walrus-core/src/encoding/blob_encoding.rs. Can we instead create some functions there that are called in both the production code and here in the benchmarks? In that case we probably also don't have to export the leaf_hash function.
Add phase-level criterion benchmarks (encoding_phases) that measure secondary encoding, primary encoding, hashing, and metadata construction independently at production parameters (n_shards=1000). Add a standalone profiling binary (profile_encoding) designed for use with samply/flamegraph without criterion overhead. Make leaf_hash public to support external benchmarking of hashing costs.
wbbradley
force-pushed
the
wbbradley/profiling-encoding
branch
from
19db448 to
f13ab19
Compare
Add heap peak tracking via peakmem-alloc and RSS peak via libc::getrusage() to the profiling binary. Each iteration now reports peak_heap, peak_rss, and heap expansion ratio. Multi-iteration runs report max_peak_heap in the summary.
Add --concurrent-blobs N flag that encodes N blobs simultaneously using std::thread::scope, simulating multi-blob uploads. Reports per-blob latency, total wall time, and peak memory with per-blob expansion ratio for direct comparison with single-blob runs.
2 participants
Description
Adds phase-level benchmarking and profiling infrastructure for the blob encoding pipeline (
encode_with_metadata()) at production parameters (n_shards=1000).examples/profile_encoding.rs— Standalone profiling binary for use withsamply recordorcargo flamegraph. Accepts--size,--shards,--iterationsflags. Reports wall-clock time and throughput (MiB/s).benches/encoding_phases.rs— Criterion benchmark measuring individual phases: secondary encoding, primary encoding, primary encoding + hashing, metadata/Merkle tree construction, and full pipeline. Blob sizes: 1MiB, 32MiB, 256MiB.leaf_hashpublic inmerkle.rsso benchmarks can measure hashing independently.clapdev-dependency for the profiling binary's CLI.Test plan
chk(formatting, clippy).cargo nextest run -p walrus-core(232 tests pass).cargo bench -p walrus-core --bench encoding_phasesto verify benchmarks execute correctly.cargo build --release --example profile_encoding && ./target/release/examples/profile_encodingto verify the profiling binary works.Release notes