sig-testing: add alpha/beta enabled and conformance periodic jobs #35621
+203
−4
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This mirrors the presubmit counterparts that were added in #35534 and tested in kubernetes/kubernetes#134250.
I can help monitor the existing alpha/beta features jobs. Those need to be renamed in testgrid to avoid confusion. Other than that they remain unchanged for now.
Potential future work:
Changes related to serial and/or slow jobs.
Serial tests are excluded implicitly by e2e-k8s.sh because the jobs enables PARALLEL (kind jobs: stop relying on implicit SKIP=[Serial] #35594).
Slow jobs are disabled in LABEL_FILTER because that is what the existing periodics did. We might be able to run them because as long as they overlap with other tests there shouldn't be much impact on overall job duration (same applies to presubmits!). Scheduling of slow tests may be relevant (prioritize tests during parallel execution onsi/ginkgo#1599).
Release informing/blocking.
The existing jobs are release informing. alpha-beta-features shouldn't be because breaking alpha tests is not something that the release team should have to deal with. Instead, the new jobs should get promoted once they are known to be stable. beta-features can remain release informing, tests for beta features (even if off-by-default) need to be stable.
Decision about "enabled-conformance".
The conformance jobs got included because it was suggested on Slack. They run a subset of the tests run by their "enabled" counterparts. It remains to be seen whether having two jobs instead of one really provides a better release signal.
/assign @aojea @BenTheElder