Skip to content

ci: Run benchmarks on CodSpeed Graviton runners - #592

Merged
robjtede merged 2 commits into
mainfrom
t3/benchmark-github-runners
Nov 7, 2026
Merged

robjtede merged 2 commits into
mainfrom
t3/benchmark-github-runners

Conversation

@robjtede

@robjtede robjtede commented Nov 7, 2026 •

Copy link
Copy Markdown
Member

Run walltime and memory benchmarks on CodSpeed’s ARM64 Graviton runner. Use https://github.com/x52dev/httpea/blob/main/.github/workflows/benchmark.yml as a reference while retaining memory measurements.

Keep the 10-minute job timeout and one-second minimum and maximum sampling times. Let Divan tune sample sizes per benchmark instead of fixing each sample at 10,000 iterations. Divan checks its time budget between samples.

Use actions/checkout@v7.0.1 and update the Actions lockfile. Run on pushes to main and default pull request events; remove the manual workflow trigger. Retain the locked Cargo build and repository action versions.

Validation on 8393620: all 15 checks passed. Walltime and memory measurements completed and uploaded. The benchmark job took 8m 00s (build: 4m 33s; measurement action: 2m 30s). The next slowest job, Rust analysis, took 4m 16s. The benchmark job was 1.875 times as long, within the two-times limit of 8m 32s. This confirms the target for this run; future cache and runner conditions can change the ratio.

Prettier passed for the benchmark workflow. Actions lockfile coverage was verified for all five workflows.

@robjtede
robjtede force-pushed the t3/benchmark-github-runners branch from f133b79 to d942b9d Compare November 7, 2026 15:03
@codspeed

codspeed Bot commented Nov 7, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 68 untouched benchmarks
🆕 68 new benchmarks
⏩ 68 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
🆕 Walltime encode_codec_frame_size_new_buffer[1024] N/A 439 ns N/A
🆕 Walltime encode_codec_frame_size_new_buffer[32] N/A 216 ns N/A
🆕 Walltime encode_codec_frame_size_new_buffer[65536] N/A 17.2 µs N/A
🆕 Walltime encode_codec_message_new_buffer[1000] N/A 436 ns N/A
🆕 Walltime encode_codec_message_new_buffer[1024] N/A 454 ns N/A
🆕 Walltime encode_codec_message_new_buffer[32] N/A 214 ns N/A
🆕 Walltime encode_codec_message_new_buffer[60000] N/A 15.8 µs N/A
🆕 Walltime encode_codec_message_new_buffer[65536] N/A 17.1 µs N/A
🆕 Walltime encode_codec_message_reused_buffer[1000] N/A 334 ns N/A
🆕 Walltime encode_codec_message_reused_buffer[1024] N/A 347 ns N/A
🆕 Walltime encode_codec_message_reused_buffer[32] N/A 94 ns N/A
🆕 Walltime encode_codec_message_reused_buffer[60000] N/A 15.3 µs N/A
🆕 Walltime encode_codec_message_reused_buffer[65536] N/A 16.7 µs N/A
🆕 Walltime encode_into_bytestring_comment N/A 170 ns N/A
🆕 Walltime encode_into_bytestring_message[1000] N/A 435 ns N/A
🆕 Walltime encode_into_bytestring_message[1024] N/A 442 ns N/A
🆕 Walltime encode_into_bytestring_message[32] N/A 180 ns N/A
🆕 Walltime encode_into_bytestring_message[60000] N/A 15.6 µs N/A
🆕 Walltime encode_into_bytestring_message[65536] N/A 16.9 µs N/A
🆕 Walltime encode_into_bytestring_multiline_message N/A 8.5 µs N/A
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.


Comparing t3/benchmark-github-runners (8393620) with main (a33191f)

Open in CodSpeed

Footnotes

  1. 68 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@robjtede
robjtede force-pushed the t3/benchmark-github-runners branch 3 times, most recently from 235a4e0 to f1a31d1 Compare November 7, 2026 15:20
@robjtede
robjtede force-pushed the t3/benchmark-github-runners branch from f1a31d1 to 2736e7d Compare November 7, 2026 15:20
@robjtede
robjtede added this pull request to the merge queue Nov 7, 2026
Merged via the queue into main with commit d14247a Nov 7, 2026
16 checks passed
@robjtede
robjtede deleted the t3/benchmark-github-runners branch November 7, 2026 15:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant