Skip to content

🤖 Update Mock Benchmark Results#6012

Merged
thomhurst merged 1 commit into
mainfrom
automated-mock-benchmarks-update
May 26, 2026
Merged

🤖 Update Mock Benchmark Results#6012
thomhurst merged 1 commit into
mainfrom
automated-mock-benchmarks-update

Conversation

@thomhurst
Copy link
Copy Markdown
Owner

Automated Mock Benchmark Update

This PR updates the mock benchmark documentation with the latest results from the Mock Benchmarks workflow.

Benchmarks Produced

Individual benchmark artifacts are available for download:

  • mock-benchmark-MockCreation
  • mock-benchmark-Setup
  • mock-benchmark-Invocation
  • mock-benchmark-Verification
  • mock-benchmark-Callback
  • mock-benchmark-CombinedWorkflow
  • mock-benchmark-summary (aggregated overview)

Libraries Compared

Libraries are discovered dynamically from the benchmark output. See the generated summary.json for the full list.

Changes

  • Updated mock benchmark data in docs/static/benchmarks/mocks/latest.json
  • Regenerated mock benchmark documentation in docs/docs/benchmarks/mocks/
  • Updated mock benchmark summary in docs/static/benchmarks/mocks/summary.json

Workflow Run

  • Run ID: 26430134729
  • Triggered: schedule

🤖 This PR was automatically created and will be merged automatically once CI checks pass.

@thomhurst thomhurst added automated benchmarks documentation Improvements or additions to documentation ignore-for-release labels May 26, 2026
@thomhurst thomhurst merged commit defb022 into main May 26, 2026
12 of 13 checks passed
@thomhurst thomhurst deleted the automated-mock-benchmarks-update branch May 26, 2026 03:28
Copy link
Copy Markdown
Contributor

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Benchmark Update Review

This is a routine automated data refresh — no functional code changes, just updated benchmark numbers and timestamps. The automation plumbing looks correct: JSON data files, rendered Markdown tables, and Mermaid chart axis ranges are all consistently updated together.

Notable observation: CombinedWorkflow regression

The CombinedWorkflow benchmark shows a notable day-over-day increase for TUnit.Mocks:

Run Mean
2026-05-25 1.459 μs
2026-05-26 1.865 μs

That's a ~28% increase in a single day on the same environment. The other benchmarks are relatively stable (sub-5% variance), so this stands out. It's almost certainly CI runner noise — cloud VMs are notoriously inconsistent for microbenchmarks — but if this kind of variance is common in the combined workflow, it may be worth investigating whether the benchmark is running with [IterationTime]/warmup counts sufficient to stabilize results on shared runners.

Suggestion (not blocking)

If day-to-day variance in the combined benchmark continues to be this high, consider either:

  • Increasing iteration count for the CombinedWorkflow benchmark specifically, or
  • Running benchmarks on self-hosted or dedicated hardware to reduce runner noise

The relative ranking of libraries hasn't changed meaningfully, so this is fine to merge as-is. ✅

@codacy-production
Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automated benchmarks documentation Improvements or additions to documentation ignore-for-release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant