Skip to content

🤖 Update Mock Benchmark Results#6018

Merged
thomhurst merged 1 commit into
mainfrom
automated-mock-benchmarks-update
May 27, 2026
Merged

🤖 Update Mock Benchmark Results#6018
thomhurst merged 1 commit into
mainfrom
automated-mock-benchmarks-update

Conversation

@thomhurst
Copy link
Copy Markdown
Owner

Automated Mock Benchmark Update

This PR updates the mock benchmark documentation with the latest results from the Mock Benchmarks workflow.

Benchmarks Produced

Individual benchmark artifacts are available for download:

  • mock-benchmark-MockCreation
  • mock-benchmark-Setup
  • mock-benchmark-Invocation
  • mock-benchmark-Verification
  • mock-benchmark-Callback
  • mock-benchmark-CombinedWorkflow
  • mock-benchmark-summary (aggregated overview)

Libraries Compared

Libraries are discovered dynamically from the benchmark output. See the generated summary.json for the full list.

Changes

  • Updated mock benchmark data in docs/static/benchmarks/mocks/latest.json
  • Regenerated mock benchmark documentation in docs/docs/benchmarks/mocks/
  • Updated mock benchmark summary in docs/static/benchmarks/mocks/summary.json

Workflow Run

  • Run ID: 26488547632
  • Triggered: schedule

🤖 This PR was automatically created and will be merged automatically once CI checks pass.

@thomhurst thomhurst added automated benchmarks documentation Improvements or additions to documentation ignore-for-release labels May 27, 2026
@thomhurst thomhurst merged commit 7a34c5d into main May 27, 2026
12 of 13 checks passed
@thomhurst thomhurst deleted the automated-mock-benchmarks-update branch May 27, 2026 03:29
@codacy-production
Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

Copy link
Copy Markdown
Contributor

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: Automated Mock Benchmark Update

This is a clean, automated documentation-only PR with no code changes — just refreshed benchmark data and timestamps. The automation pipeline is working as expected.

Overall Assessment

The vast majority of benchmark delta falls within normal run-to-run variance (1–5%). The data format, chart axes, and file structure are all consistent.

One Item Worth Investigating

In the Callback (with args) benchmark, TUnit.Mocks regressed ~13.7%:

Previous Current Delta
TUnit.Mocks (Callback with args) 755.9 ns 859.4 ns +13.7%

Compare that to the other libraries in the same run, which moved at most 2–3%:

  • Imposter: 544.2 → 533.2 ns (-2%)
  • NSubstitute: 4,686.3 → 4,704.6 ns (+0.4%)

A 13.7% swing for TUnit.Mocks while competitors are flat is larger than benchmark noise typically produces. It may be:

  1. A genuine performance regression introduced recently
  2. A one-off outlier due to CI machine load

Suggestion: If a second benchmark run shows the same pattern, it's worth investigating the callback-with-args path in TUnit.Mocks source generation for unintended overhead introduced since the last run.

Everything Else

All other metrics look within expected variance. The automation, file paths, chart regeneration, and JSON schema updates are correct. No structural concerns with the PR itself — safe to merge once CI passes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automated benchmarks documentation Improvements or additions to documentation ignore-for-release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant