Conversation
Up to standards ✅🟢 Issues
|
There was a problem hiding this comment.
Mock Benchmark Update Review
This is a clean, automated benchmark data update with no code changes — just refreshed timings, dates, and JSON data. No issues with the automation itself.
Performance Highlights
TUnit.Mocks improvements (good):
- Callback: 731 ns → 612 ns (~16% faster)
- Invocation (void): 255 ns → 233 ns (~9% faster)
TUnit.Mocks regressions (minor, within CI noise):
- CombinedWorkflow: 1.806 μs → 1.941 μs (~7.5% slower) — still ~160× faster than Moq in this benchmark
Competitor movements:
- Moq Callback notably regressed: 140k ns → 180k ns (+28%) — likely a CI environment variance rather than a real change
- Moq Verification (Multiple) spiked: 346k ns → 467k ns (+35%) — same likely cause
Minor Structural Note
In the updated JSON files, several entries that previously included a field no longer have it (e.g., Verification and Callback entries). This is a schema inconsistency between the old and new benchmark output format. If the documentation renderer or any tooling depends on the field being present, it's worth verifying. That said, this appears to be a change in what BenchmarkDotNet emits, not a bug in the automation.
Missing EOF Newline
now lacks a trailing newline (). Minor, but worth fixing in the generation script if it's not intentional.
Overall this is a good, routine update — the automation is working as intended, and TUnit.Mocks continues to compare favourably across most benchmark categories.
There was a problem hiding this comment.
Mock Benchmark Update Review
This is a clean, automated benchmark data update with no code changes — just refreshed timings, dates, and JSON data.
Performance Highlights
TUnit.Mocks improvements:
- Callback: 731 ns -> 612 ns (~16% faster)
- Invocation (void): 255 ns -> 233 ns (~9% faster)
TUnit.Mocks minor regressions (within CI variance):
- CombinedWorkflow: 1.806 us -> 1.941 us (~7.5% slower) — still ~160x faster than Moq in this benchmark
Competitor movements:
- Moq Callback regressed: 140k ns -> 180k ns (+28%) — likely CI environment variance
- Moq Verification (Multiple): 346k ns -> 467k ns (+35%) — same likely cause
Minor Structural Note
Several JSON entries that previously included a 'Median' field no longer have it in the new benchmark output. This is a schema change in what BenchmarkDotNet emits. If any tooling depends on the Median field being present, it is worth verifying. The generation script should handle this gracefully.
Missing EOF Newline
docs/static/benchmarks/mocks/latest.json now lacks a trailing newline. Minor, but worth fixing in the generation script if unintentional.
Overall this is a routine, well-structured update. The automation is working as intended, and TUnit.Mocks continues to compare favourably across most benchmark categories.
Automated Mock Benchmark Update
This PR updates the mock benchmark documentation with the latest results from the Mock Benchmarks workflow.
Benchmarks Produced
Individual benchmark artifacts are available for download:
mock-benchmark-MockCreationmock-benchmark-Setupmock-benchmark-Invocationmock-benchmark-Verificationmock-benchmark-Callbackmock-benchmark-CombinedWorkflowmock-benchmark-summary(aggregated overview)Libraries Compared
Libraries are discovered dynamically from the benchmark output. See the generated
summary.jsonfor the full list.Changes
docs/static/benchmarks/mocks/latest.jsondocs/docs/benchmarks/mocks/docs/static/benchmarks/mocks/summary.jsonWorkflow Run
🤖 This PR was automatically created and will be merged automatically once CI checks pass.