Skip to content

fix(process): judge the pull request in CI fix: commit check (#1654) - #1656

Open
Mohit-Gaur75 wants to merge 2 commits into
pyrite-wiki:devfrom
Mohit-Gaur75:fix/1654-fix-check-judges-pull-request
Open

Mohit-Gaur75 wants to merge 2 commits into
pyrite-wiki:devfrom
Mohit-Gaur75:fix/1654-fix-check-judges-pull-request

Conversation

@Mohit-Gaur75

Copy link
Copy Markdown
Contributor

Summary

In CI --range mode, judge the pull request as a whole rather than judging each commit individually. Because PRs are squashed upon merge to dev, the unit that lands is the pull request. If the range holds any fix: commit and at least one commit in the range touches a test path (such as in a test: then fix: branch), the check passes. If no commit touches a test path, each fix: commit is reported and the check fails. Local commit-msg hook remains per-commit.

Fixes #1654

Plan / claim

Claim comment: #1654 (comment)

Testing

  • pytest tests/ extensions/ -n auto passes locally (or: the pre-push hook ran it)
  • ruff check pyrite/ tests/ extensions/ && ruff format --check pyrite/ tests/ extensions/ is clean
  • A fix: change includes a test that fails without the fix (the commit-msg hook and CI check that a fix touches tests/)
  • Frontend change: cd web && npm run check && npm run test:unit

Changelog: none

Notes for the reviewer

  • Parametrised TestRangeMode.test_range_shapes covers all requested shapes: test_then_fix, fix_then_test, fix_with_its_test, fix_alone, two_fixes_and_one_test, two_fixes_and_no_test, scoped_breaking_fix_with_test, scoped_breaking_fix_alone, docs_only_fix_prefix, docs_commit_without_fix_prefix, frontend_test_path, and no_fix_commit.
  • Dedicated tests cover the regime of a PR that merged dev into itself with and without tests under synthetic merge commits (HEAD^1 HEAD).
  • Dedicated tests cover TestLocalCommitMsgHook for staged commit behavior.
  • Updated CONTRIBUTING.md and .github/PULL_REQUEST_TEMPLATE.md to reflect the pull-request level check in CI and squash merge queue behavior.

@markramm markramm left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of 8e1993b. The behaviour matches #1654. check_range in scripts/check_fix_commit_has_tests.py judges the whole pull request: it fails only when the range holds a fix: commit and no commit touches a test path, and then it names every fix commit. The local commit-msg path (main()) is unchanged. With HEAD^1..HEAD, dev's commits stay out of the range, and a merge commit reports no paths without -m, so a merge of dev neither supplies a test nor counts as a fix. All seven shapes and the three regimes in the groom are covered. test_then_fix, fix_then_test, two_fixes_and_one_test, scoped_breaking_fix_with_test, frontend_test_path and the merge-with-test case are red on dev, as they should be. Deleting test_range_checks_every_commit_not_just_the_last is right, because it pinned the opposite rule.

Blocking (text only):

  1. .github/workflows/ci.yml, the comment above the step "Check fix: commits have tests (pull_request only)" (lines 394-396 on dev). It still says "--range walks each commit individually (not the range's cumulative diff), so an earlier offending commit isn't hidden by a later, unrelated commit that happens to touch tests/". That is now the opposite of what the step does. Please reword it: CI judges the PR as a whole, a fix: passes if any commit in the range touches a test path, and the reason is that the queue squashes. The step command stays as it is. The file is not in #1654's Touches list, but the issue asks for the rule to be stated truly everywhere.

Non-blocking:
2. CONTRIBUTING.md, the new merge-method sentence ("The merge queue squashes each pull request into a single commit on dev"). Add "by default" so it agrees with "Maintainer-only landing with a merge commit" further down the same file.
3. _NO_TEST_MESSAGE is shared with the local hook. The trimmed text drops "put the test in the same commit as the fix", which is still the right advice for a local commit. Consider keeping one sentence that only the hook prints.
4. No test pins the empty range (it returns 0, which is correct).

This touches .github/, so the maintainer merges it once item 1 is in.

Recommendation: request changes (item 1 only; 2-4 are optional).

@markramm markramm added state: needs-review No review on the current head: the next move is ours state: needs-author Changes requested on the current head: the next move is the author's and removed state: needs-review No review on the current head: the next move is ours labels Nov 17, 2026
@Mohit-Gaur75

Copy link
Copy Markdown
Contributor Author

Thank you for the review! All 4 items have been addressed in 374dc0c8:

  1. .github/workflows/ci.yml: Reworded the step comment above "Check fix: commits have tests (pull_request only)" to state that CI judges the PR as a whole, a fix: passes if any commit in the range touches a test path, and the reason is that the queue squashes.
  2. CONTRIBUTING.md: Added "by default" to the merge queue squash sentence ("The merge queue squashes each pull request into a single commit on dev by default") so it agrees with the maintainer landing section.
  3. scripts/check_fix_commit_has_tests.py: Added "Put the test in the same commit as the fix." back to the message printed exclusively by the local commit-msg hook (main()).
  4. tests/test_check_fix_commit_has_tests.py: Added test_range_empty_range_returns_zero asserting returncode 0 on an empty range, and added assertion for the hook sentence in test_local_hook_fails_when_fix_has_no_staged_test. All 49 tests pass cleanly.

@markramm markramm left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated delta review of head 374dc0c (previous review on 8e1993b). Comment only.

Earlier item Status Where
BLOCKING: ci.yml comment said --range walks each commit Fixed. New comment says CI judges the PR as a whole, a fix: passes if any commit in the range touches a test path, because the merge queue squashes each PR. .github/workflows/ci.yml:394-396
Drop "by default" on the squash sentence Not fixed. The delta adds "by default" (the 8e1993b text lacked it). The sentence now reads "...into a single commit on dev by default." CONTRIBUTING.md:557
Keep a hook-only "same commit" sentence Fixed. The local hook message ends "Put the test in the same commit as the fix." and the hook test asserts it. scripts/check_fix_commit_has_tests.py:~168; tests/test_check_fix_commit_has_tests.py:~441
Add an empty-range test Fixed. test_range_empty_range_returns_zero runs --range HEAD HEAD, asserts rc 0 and empty stderr. tests/test_check_fix_commit_has_tests.py:~360

New finding (non-blocking): the "by default" wording at CONTRIBUTING.md:557 contradicts the squash-always claim made in ci.yml and in the new CONTRIBUTING/PR-template text. If squash is not guaranteed, the whole-PR check is unjustified; if it is, the qualifier is wrong. Drop the two words. I read the diff only and ran no tests.

CI on 374dc0c: pending. collect 3.11/3.13, frontend, lock, mypy and changes pass; the test shards (3.11 to 3.13) and experimental are still running; coverage, e2e, kb and smoke skipped.

Because this touches .github/, the maintainer merges it.

Recommendation: merge.

@markramm markramm added state: needs-maintainer Reviewed clean; the maintainer merges (storage, server, schema or a public shape) and removed state: needs-author Changes requested on the current head: the next move is the author's labels Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

state: needs-maintainer Reviewed clean; the maintainer merges (storage, server, schema or a public shape)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The fix: check judges the pull request, not each commit

2 participants