🤖 Update Mock Benchmark Results#6012
Conversation
There was a problem hiding this comment.
Automated Benchmark Update Review
This is a routine automated data refresh — no functional code changes, just updated benchmark numbers and timestamps. The automation plumbing looks correct: JSON data files, rendered Markdown tables, and Mermaid chart axis ranges are all consistently updated together.
Notable observation: CombinedWorkflow regression
The CombinedWorkflow benchmark shows a notable day-over-day increase for TUnit.Mocks:
| Run | Mean |
|---|---|
| 2026-05-25 | 1.459 μs |
| 2026-05-26 | 1.865 μs |
That's a ~28% increase in a single day on the same environment. The other benchmarks are relatively stable (sub-5% variance), so this stands out. It's almost certainly CI runner noise — cloud VMs are notoriously inconsistent for microbenchmarks — but if this kind of variance is common in the combined workflow, it may be worth investigating whether the benchmark is running with [IterationTime]/warmup counts sufficient to stabilize results on shared runners.
Suggestion (not blocking)
If day-to-day variance in the combined benchmark continues to be this high, consider either:
- Increasing iteration count for the
CombinedWorkflowbenchmark specifically, or - Running benchmarks on self-hosted or dedicated hardware to reduce runner noise
The relative ranking of libraries hasn't changed meaningfully, so this is fine to merge as-is. ✅
Up to standards ✅🟢 Issues
|
Automated Mock Benchmark Update
This PR updates the mock benchmark documentation with the latest results from the Mock Benchmarks workflow.
Benchmarks Produced
Individual benchmark artifacts are available for download:
mock-benchmark-MockCreationmock-benchmark-Setupmock-benchmark-Invocationmock-benchmark-Verificationmock-benchmark-Callbackmock-benchmark-CombinedWorkflowmock-benchmark-summary(aggregated overview)Libraries Compared
Libraries are discovered dynamically from the benchmark output. See the generated
summary.jsonfor the full list.Changes
docs/static/benchmarks/mocks/latest.jsondocs/docs/benchmarks/mocks/docs/static/benchmarks/mocks/summary.jsonWorkflow Run
🤖 This PR was automatically created and will be merged automatically once CI checks pass.