fix(github-actions): continue Docker compatibility tests after failures - #3351
Conversation
|
🎉 Thanks for your first pull request to Floci! Your CI checks need a maintainer to approve them before they run. That is GitHub's standard gate on first-time contributors, not a problem with your PR — so if the checks look like they are doing nothing, that is why. Once a maintainer approves, CI and the compatibility suite start automatically. Nothing is needed from you in the meantime. While you wait, a couple of things that make review faster:
Come join us in Slack — it is the fastest way to reach maintainers if you get stuck, or want feedback on an approach before investing more time in it. |
|
| Filename | Overview |
|---|---|
| docker/run-docker-tests.sh | Aggregates Docker build and suite failures, continues remaining suites, and reports the final status. |
| docker/test-run-docker-tests.sh | Tests run-failure aggregation, build-failure handling, continuation behavior, and successful completion. |
| .github/workflows/compatibility.yml | Adds automated execution of the Docker runner regression test and relevant path triggers. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Start compatibility runner] --> B[Build next suite image]
B -->|Build fails| C[Record failure]
B -->|Build succeeds| D[Run suite container]
D -->|Run fails| C
D -->|Run succeeds| E{More suites?}
C --> E
E -->|Yes| B
E -->|No| F{Any failure recorded?}
F -->|Yes| G[Exit non-zero]
F -->|No| H[Exit zero]
Reviews (2): Last reviewed commit: "test: run Docker failure aggregation in ..." | Re-trigger Greptile
pgermosen
left a comment
There was a problem hiding this comment.
Traced this against the actual script on main rather than just the description, and it's fixing two real bugs, not one. The old docker build call had no failure handling under this script's set -e, so a build failure would silently kill the whole run before later suites ever executed. Separately, the docker run failure was only ever echoed, never tracked, so the script always exited 0 regardless, meaning CI never actually failed on a real test failure. Both are fixed here.
The test harness is genuinely convincing too, not just exercising the code. I checked that the hardcoded suite list in the test matches the real SUITES array exactly, so it'd actually catch drift instead of silently passing on a stale assumption.
Summary
The Docker compatibility runner now executes every remaining suite after an image build or test failure, then returns a non-zero status so CI reports the failure without hiding results from later suites.
Type of change
fix:)feat:)feat!:orfix!:)AWS Compatibility
A failed compatibility suite no longer stops the other independent suites from running. The workflow still fails at the end when any suite failed, so CI keeps its failure signal while producing a complete result set.
Checklist
./mvnw testpasses locally