The different jobs in test_pull_request have two types of behavior in terms of which HF adapters commit they run on:
- The regular jobs (load, smoke etc.) use
build-hf-adapters, which does checkout of the PR-head commit (PR_HEAD_SHA)
- Other jobs (generating test matrices, pre-commit, adapter coverage etc.) use just
actions/checkout, which uses GITHUB_SHA - a synthetic merge commit of the PR with the latest main.
So whenever the PR is not up-to-date with main, this can lead to strange discrepancies - for example, the test matrix includes models with adapters that are not yet in the PR code (because in the GitHub synthetic head they exist), and then these matrix entries try to run in the PR code and naturally fail.
The different jobs in
test_pull_requesthave two types of behavior in terms of which HF adapters commit they run on:build-hf-adapters, which does checkout of the PR-head commit (PR_HEAD_SHA)actions/checkout, which usesGITHUB_SHA- a synthetic merge commit of the PR with the latest main.So whenever the PR is not up-to-date with main, this can lead to strange discrepancies - for example, the test matrix includes models with adapters that are not yet in the PR code (because in the GitHub synthetic head they exist), and then these matrix entries try to run in the PR code and naturally fail.