Skip to content

chore: add --changedSince base SHA to Jest in CI#110568

Merged
JoshuaKGoldberg merged 4 commits into
masterfrom
frontend-jest-ci-changed-since
Mar 30, 2026
Merged

chore: add --changedSince base SHA to Jest in CI#110568
JoshuaKGoldberg merged 4 commits into
masterfrom
frontend-jest-ci-changed-since

Conversation

@JoshuaKGoldberg

@JoshuaKGoldberg JoshuaKGoldberg commented Mar 12, 2026

Copy link
Copy Markdown
Member

"Why run many unit test when few do trick?" – Kevin Malone, Senior Frontend Developer

Changes the Frontend > Jest jobs to use Jest's --changedSince option when possible. That has them only run test suites that are impacted by the changes in a PR based on the module dependency graph. Jest requires the commit history since the base commit, so this adds a second "run only necessary Jest tests" flow in frontend-optional.yml. Specifically, that:

  1. Increases the frontend-optional fetch-depth in PRs to 100, which should get most PRs' commits pretty quickly
  2. Tries to compute a MERGE_BASE using git merge-base
    • Note that this also clears MERGE_BASE if the PR touches any non-static/ files.
  3. If MERGE_BASE could be computed, passes it as a process env var to test scripts for use as Jest's changedSince
    • If MERGE_BASE couldn't be computed, the new "run only necessary Jest tests" flow is skipped

My plan (thanks to reviewer feedback) is to keep this running as a secondary, optional test flow on master for a couple weeks. If all seems well, we can remove the original "run all Jest tests" logic and only run the "run only necessary Jest tests" logic.

The following PRs exercise different scales of changes:

PR Source Files Test Files Interconnectivity Ran Suites / 1,881 suites
#110624 0 1 ⬜️ None 1 0.05%
#111073 0 6 ⬜️ None 6 0.32%
#111122 0 100 ⬜️ None 100 5.3%
#111074 1 0 🟦 Low 5 0.27%
#111075 4 0 🟦 Low 7 0.37%
#111076 3 3 🟦 Low 13 0.69%
#111110 20 20 🟦 Low 74 3.9%
#111111 100 100 🟦 Low 196 10.4%
#111119 1 1 🟨 Medium 30 1.6%
#111628 3 3 🟨 Medium 78 4.1%
#111629 10 10 🟨 Medium 167 8.9%
#111633 25 25 🟨 Medium 227 12.1%
#111652 100 100 🟨 Medium 549 29.2%
#111115 3 3 🟧 High 1,652 87.8%
#111083 20 20 🟧 High 1,660 88.3%
#111086 100 100 🟧 High 1,721 91.5%
#110568 0 0 🟥 Config (Full) 1,881 100%

For low-connectivity changes (isolated components, feature-specific views), --changedSince provides pretty great savings: even 100 low-connectivity files only trigger ~10% of suites. Medium-connectivity files still scale pretty well, with 100 files hitting ~29%. High-connectivity files (widely-imported utils, core components) sadly trigger 85-90% of suites, which is inevitable given Sentry's highly connected module graph.

Fixes ENG-7103

@github-actions github-actions Bot added the Scope: Frontend Automatically applied to PRs that change frontend components label Mar 12, 2026
@JoshuaKGoldberg JoshuaKGoldberg force-pushed the frontend-jest-ci-changed-since branch from 4ac580b to e489c3c Compare March 13, 2026 13:40
@JoshuaKGoldberg JoshuaKGoldberg force-pushed the frontend-jest-ci-changed-since branch from e489c3c to 2f37360 Compare March 19, 2026 12:53
@linear-code

linear-code Bot commented Mar 19, 2026

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Comment thread jest.config.ts
if (allTests.length === 0) {
return [];
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shard balancing uses unfiltered total, breaking distribution

Medium Severity

speculatedSuiteDuration is computed from the full testStats (all ~1,861 suites), but the new filtering at lines 143–146 correctly restricts the tests Map to only the allTests subset from --changedSince. Since targetDuration is massively inflated relative to the actual filtered tests' total duration, the balancing loop stuffs every test into the first shard while shards 1–3 get nothing. This defeats parallelization for any small-to-medium --changedSince result set.

Additional Locations (1)
Fix in Cursor Fix in Web

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, interesting. I don't think parallelization at those result set scales really matters much, right? Running 10 suites vs 100 is pretty quick.

Comment thread .github/workflows/frontend.yml
@scttcper

Copy link
Copy Markdown
Member

on package.json or those kinds of changes would it know to run the entire suite?

Comment thread .github/workflows/frontend.yml Outdated
Comment thread .github/workflows/frontend-optional.yml Outdated
The git diff for the non-frontend file guard was comparing against HEAD
(the synthetic merge commit) instead of HEAD^2 (the PR branch tip).
This included base branch changes in the diff, causing false positives
that unnecessarily forced full test runs.

Made-with: Cursor
@JoshuaKGoldberg JoshuaKGoldberg force-pushed the frontend-jest-ci-changed-since branch from 14d8c0e to 5c9ae7e Compare March 26, 2026 20:13
@JoshuaKGoldberg JoshuaKGoldberg marked this pull request as ready for review March 26, 2026 22:20
@JoshuaKGoldberg JoshuaKGoldberg merged commit d9cf05d into master Mar 30, 2026
88 checks passed
@JoshuaKGoldberg JoshuaKGoldberg deleted the frontend-jest-ci-changed-since branch March 30, 2026 14:17
@github-actions github-actions Bot locked and limited conversation to collaborators Apr 15, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Scope: Frontend Automatically applied to PRs that change frontend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants