Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: test larger gh runners #8545

Merged
merged 2 commits into from
Feb 7, 2025
Merged

chore: test larger gh runners #8545

merged 2 commits into from
Feb 7, 2025

Conversation

stipsan
Copy link
Member

@stipsan stipsan commented Feb 7, 2025

Description

We often get congestion and bottlenecks if there's a lot of PRs being worked on, as the default ubuntu-latest runner has a 60 concurrent builds limit. Current queue status is visible here: https://github.com/organizations/sanity-io/settings/actions/hosted-runners

image

Based on the stats here, I picked the jobs that use the most resources on two new images, ubuntu-latest-m and ubuntu-22.04-m, which have their own separate 60 concurrent builds limits, and have more memory and more CPU cores (as well as faster ones).

What to review

Everything should work like before, only faster and less bottlenecks on the PR checks.

Testing

If the status checks behave exactly like before (but faster) then we're good to go 👍

Notes for release

N/A

Copy link

vercel bot commented Feb 7, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
page-building-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Feb 7, 2025 2:23pm
performance-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Feb 7, 2025 2:23pm
test-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Feb 7, 2025 2:23pm
2 Skipped Deployments
Name Status Preview Comments Updated (UTC)
studio-workshop ⬜️ Ignored (Inspect) Visit Preview Feb 7, 2025 2:23pm
test-next-studio ⬜️ Ignored (Inspect) Feb 7, 2025 2:23pm

Copy link
Contributor

github-actions bot commented Feb 7, 2025

No changes to documentation

Copy link
Contributor

github-actions bot commented Feb 7, 2025

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 42.45% 53812 / 126761
🔵 Statements 42.45% 53812 / 126761
🔵 Functions 47.94% 2772 / 5782
🔵 Branches 78.71% 10274 / 13052
File CoverageNo changed files found.
Generated in workflow #30016 for commit a93eaeb by the Vitest Coverage Report Action

Copy link
Contributor

github-actions bot commented Feb 7, 2025

⚡️ Editor Performance Report

Updated Fri, 07 Feb 2025 14:32:01 GMT

Benchmark reference
latency of sanity@latest
experiment
latency of this branch
Δ (%)
latency difference
article (title) 22.7 efps (44ms) 23.3 efps (43ms) -1ms (-2.3%)
article (body) 59.5 efps (17ms) 63.7 efps (16ms) -1ms (-6.5%)
article (string inside object) 24.4 efps (41ms) 23.8 efps (42ms) +1ms (+2.4%)
article (string inside array) 20.4 efps (49ms) 20.8 efps (48ms) -1ms (-2.0%)
recipe (name) 52.6 efps (19ms) 50.0 efps (20ms) +1ms (+5.3%)
recipe (description) 55.6 efps (18ms) 55.6 efps (18ms) +0ms (-/-%)
recipe (instructions) 99.9+ efps (5ms) 99.9+ efps (5ms) +0ms (-/-%)
synthetic (title) 18.2 efps (55ms) 18.0 efps (56ms) +1ms (+0.9%)
synthetic (string inside object) 18.2 efps (55ms) 18.9 efps (53ms) -2ms (-3.6%)

efps — editor "frames per second". The number of updates assumed to be possible within a second.

Derived from input latency. efps = 1000 / input_latency

Detailed information

🏠 Reference result

The performance result of sanity@latest

Benchmark latency p75 p90 p99 blocking time test duration
article (title) 44ms 50ms 82ms 460ms 944ms 11.6s
article (body) 17ms 21ms 34ms 198ms 573ms 6.0s
article (string inside object) 41ms 43ms 49ms 149ms 157ms 6.8s
article (string inside array) 49ms 52ms 59ms 255ms 494ms 8.0s
recipe (name) 19ms 20ms 22ms 43ms 0ms 8.2s
recipe (description) 18ms 19ms 21ms 45ms 0ms 4.5s
recipe (instructions) 5ms 7ms 9ms 18ms 0ms 3.1s
synthetic (title) 55ms 57ms 60ms 320ms 937ms 13.5s
synthetic (string inside object) 55ms 60ms 69ms 498ms 1607ms 9.1s

🧪 Experiment result

The performance result of this branch

Benchmark latency p75 p90 p99 blocking time test duration
article (title) 43ms 69ms 87ms 392ms 929ms 12.6s
article (body) 16ms 18ms 24ms 80ms 62ms 5.2s
article (string inside object) 42ms 44ms 47ms 151ms 170ms 7.3s
article (string inside array) 48ms 51ms 56ms 170ms 303ms 7.3s
recipe (name) 20ms 22ms 23ms 40ms 0ms 7.6s
recipe (description) 18ms 19ms 21ms 36ms 0ms 4.5s
recipe (instructions) 5ms 6ms 7ms 19ms 0ms 3.0s
synthetic (title) 56ms 58ms 61ms 282ms 1022ms 13.2s
synthetic (string inside object) 53ms 55ms 61ms 548ms 1301ms 8.5s

📚 Glossary

column definitions

  • benchmark — the name of the test, e.g. "article", followed by the label of the field being measured, e.g. "(title)".
  • latency — the time between when a key was pressed and when it was rendered. derived from a set of samples. the median (p50) is shown to show the most common latency.
  • p75 — the 75th percentile of the input latency in the test run. 75% of the sampled inputs in this benchmark were processed faster than this value. this provides insight into the upper range of typical performance.
  • p90 — the 90th percentile of the input latency in the test run. 90% of the sampled inputs were faster than this. this metric helps identify slower interactions that occurred less frequently during the benchmark.
  • p99 — the 99th percentile of the input latency in the test run. only 1% of sampled inputs were slower than this. this represents the worst-case scenarios encountered during the benchmark, useful for identifying potential performance outliers.
  • blocking time — the total time during which the main thread was blocked, preventing user input and UI updates. this metric helps identify performance bottlenecks that may cause the interface to feel unresponsive.
  • test duration — how long the test run took to complete.

@stipsan stipsan marked this pull request as ready for review February 7, 2025 14:20
@stipsan stipsan requested a review from a team as a code owner February 7, 2025 14:20
@stipsan stipsan requested review from pedrobonamin and removed request for a team February 7, 2025 14:20
Copy link
Contributor

github-actions bot commented Feb 7, 2025

Component Testing Report Updated Feb 7, 2025 2:31 PM (UTC)

❌ Failed Tests (2) -- expand for details
File Status Duration Passed Skipped Failed
comments/CommentInput.spec.tsx ✅ Passed (Inspect) 1m 10s 15 0 0
formBuilder/ArrayInput.spec.tsx ✅ Passed (Inspect) 12s 3 0 0
formBuilder/inputs/PortableText/Annotations.spec.tsx ❌ Failed (Inspect) 1m 57s 4 0 2
formBuilder/inputs/PortableText/copyPaste/CopyPaste.spec.tsx ✅ Passed (Inspect) 51s 11 7 0
formBuilder/inputs/PortableText/copyPaste/CopyPasteFields.spec.tsx ✅ Passed (Inspect) 0s 0 12 0
formBuilder/inputs/PortableText/Decorators.spec.tsx ✅ Passed (Inspect) 26s 6 0 0
formBuilder/inputs/PortableText/DisableFocusAndUnset.spec.tsx ✅ Passed (Inspect) 14s 3 0 0
formBuilder/inputs/PortableText/DragAndDrop.spec.tsx ✅ Passed (Inspect) 28s 6 0 0
formBuilder/inputs/PortableText/FocusTracking.spec.tsx ✅ Passed (Inspect) 1m 8s 15 0 0
formBuilder/inputs/PortableText/Input.spec.tsx ✅ Passed (Inspect) 1m 39s 21 0 0
formBuilder/inputs/PortableText/ObjectBlock.spec.tsx ✅ Passed (Inspect) 2m 3s 21 0 0
formBuilder/inputs/PortableText/PresenceCursors.spec.tsx ✅ Passed (Inspect) 13s 3 9 0
formBuilder/inputs/PortableText/Styles.spec.tsx ✅ Passed (Inspect) 26s 6 0 0
formBuilder/inputs/PortableText/Toolbar.spec.tsx ✅ Passed (Inspect) 1m 44s 21 0 0
formBuilder/tree-editing/TreeEditing.spec.tsx ✅ Passed (Inspect) 0s 0 3 0
formBuilder/tree-editing/TreeEditingNestedObjects.spec.tsx ✅ Passed (Inspect) 0s 0 3 0

@stipsan stipsan added this pull request to the merge queue Feb 7, 2025
Merged via the queue into next with commit d8b4e6c Feb 7, 2025
62 checks passed
@stipsan stipsan deleted the use-larger-runner-for-e2e branch February 7, 2025 14:46
juice49 pushed a commit that referenced this pull request Feb 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants