Skip to content

fix(github-actions): continue Docker compatibility tests after failures - #3351

Merged
pgermosen merged 3 commits into
floci-io:mainfrom
nkootstra:fix/docker-compatibility-failure-status
Sep 10, 2026
Merged

fix(github-actions): continue Docker compatibility tests after failures#3351
pgermosen merged 3 commits into
floci-io:mainfrom
nkootstra:fix/docker-compatibility-failure-status

Conversation

@nkootstra

Copy link
Copy Markdown
Contributor

Summary

The Docker compatibility runner now executes every remaining suite after an image build or test failure, then returns a non-zero status so CI reports the failure without hiding results from later suites.

Type of change

  • Bug fix (fix:)
  • New feature (feat:)
  • Breaking change (feat!: or fix!:)
  • Docs / chore

AWS Compatibility

A failed compatibility suite no longer stops the other independent suites from running. The workflow still fails at the end when any suite failed, so CI keeps its failure signal while producing a complete result set.

Checklist

  • ./mvnw test passes locally
  • New or updated integration test added
  • Commit messages follow Conventional Commits

@github-actions

Copy link
Copy Markdown

🎉 Thanks for your first pull request to Floci!

Your CI checks need a maintainer to approve them before they run. That is GitHub's standard gate on first-time contributors, not a problem with your PR — so if the checks look like they are doing nothing, that is why. Once a maintainer approves, CI and the compatibility suite start automatically. Nothing is needed from you in the meantime.

While you wait, a couple of things that make review faster:

  • Link the issue this fixes with Closes #N in the description
  • Commits follow Conventional Commits (feat(s3): ..., fix(dynamodb): ...)
  • Behaviour changes come with a test — see CONTRIBUTING.md

Come join us in Slack — it is the fastest way to reach maintainers if you get stuck, or want feedback on an approach before investing more time in it.

@nkootstra
nkootstra marked this pull request as ready for review September 10, 2026 06:06
@greptile-apps

greptile-apps Bot commented Sep 10, 2026

Copy link
Copy Markdown

Greptile Summary

This PR updates the Docker compatibility runner to aggregate build and test failures while continuing through every independent suite.

  • Records failures and returns a non-zero status after all suites finish.
  • Skips execution only for suites whose image build failed.
  • Adds regression coverage for run failures, build failures, and successful runs.
  • Runs the regression script automatically in the compatibility workflow.

Confidence Score: 5/5

The PR appears safe to merge, with the prior automation concern fully addressed by the new compatibility workflow job.

The runner preserves the CI failure signal while continuing independent suites, and the added workflow now automatically exercises the regression script for both runner and test-script changes.

Important Files Changed

Filename Overview
docker/run-docker-tests.sh Aggregates Docker build and suite failures, continues remaining suites, and reports the final status.
docker/test-run-docker-tests.sh Tests run-failure aggregation, build-failure handling, continuation behavior, and successful completion.
.github/workflows/compatibility.yml Adds automated execution of the Docker runner regression test and relevant path triggers.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Start compatibility runner] --> B[Build next suite image]
    B -->|Build fails| C[Record failure]
    B -->|Build succeeds| D[Run suite container]
    D -->|Run fails| C
    D -->|Run succeeds| E{More suites?}
    C --> E
    E -->|Yes| B
    E -->|No| F{Any failure recorded?}
    F -->|Yes| G[Exit non-zero]
    F -->|No| H[Exit zero]
Loading

Reviews (2): Last reviewed commit: "test: run Docker failure aggregation in ..." | Re-trigger Greptile

Comment thread docker/test-run-docker-tests.sh
@hectorvent hectorvent added bug Something isn't working github-actions labels Sep 10, 2026
@nkootstra nkootstra changed the title fix: continue Docker compatibility tests after failures fix(github-actions): continue Docker compatibility tests after failures Sep 10, 2026

@pgermosen pgermosen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Traced this against the actual script on main rather than just the description, and it's fixing two real bugs, not one. The old docker build call had no failure handling under this script's set -e, so a build failure would silently kill the whole run before later suites ever executed. Separately, the docker run failure was only ever echoed, never tracked, so the script always exited 0 regardless, meaning CI never actually failed on a real test failure. Both are fixed here.

The test harness is genuinely convincing too, not just exercising the code. I checked that the hardcoded suite list in the test matches the real SUITES array exactly, so it'd actually catch drift instead of silently passing on a stale assumption.

@pgermosen
pgermosen merged commit 051a2f0 into floci-io:main Sep 10, 2026
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working github-actions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants