ci: upload unit test coverage to Codecov [RHIDP-14400] by rm3l · Pull Request #435 · redhat-developer/rhdh-must-gather

rm3l · 2026-10-06T11:48:28Z

Description

Add Codecov integration for unit test coverage reporting, mirroring the setup in the rhdh-operator repo.

Coverage checks are informational only and will not block merges.

Which issue(s) does this PR fix or relate to

Fixes RHIDP-14400

PR acceptance criteria

Tests
Documentation

How to test changes / Special notes to the reviewer

preview from this branch: https://app.codecov.io/gh/redhat-developer/rhdh-must-gather/tree/rm3l%2Fredhat-developer-hub-must-gather%3ARHIDP-14400--collect-must-gather-coverage-upload-to-codecov-io/

The nightly workflow can be tested after merge.

Add codecov.yml configuration and upload steps to the PR test workflow. Coverage checks are informational only and will not block merges. Changes: - Makefile: add -coverprofile=cover.out to the test target - codecov.yml: project/patch status as informational, unittests flag with carryforward, ignore non-source directories - codecov-checks.yaml: new workflow to validate codecov.yml against the Codecov API - test.yaml: upload cover.out after make test using codecov/codecov-action v6 - .gitignore: add cover.out Assisted-by: Claude

rhdh-qodo-merge · 2026-10-06T11:48:52Z

PR Summary by Qodo

Upload PR and nightly unit-test coverage to Codecov

✨ Enhancement ⚙️ Configuration changes 🕐 20-40 Minutes

AI Description

• Generate Go coverage profiles during unit tests and upload them from PR and nightly workflows.
• Configure informational Codecov reporting and validate configuration changes against the Codecov
 API.
• Add nightly lint and unit tests across release branches alongside existing E2E tests.

Diagram

graph TD
  PR["PR tests"] --> Test["make test"] --> Profile["cover.out"] --> Upload["Codecov action"] --> Codecov(["Codecov"])
  Nightly["Nightly tests"] --> Test
  Checks["Config checks"] --> Config["codecov.yml"] --> Codecov

High-Level Assessment

Generating coverage in the shared Makefile target keeps local, PR, and nightly test runs consistent. Separate upload and configuration-validation workflows fit the existing CI structure; coverage reporting remains informational.

Files changed (6) +147 / -2

Other (6) +147 / -2

codecov-checks.yamlValidate Codecov configuration on relevant changes +63/-0

Validate Codecov configuration on relevant changes
• Adds a workflow that submits codecov.yml to Codecov's validation API on relevant pushes and PRs. It reports invalid configuration and fails when validation cannot complete.
.github/workflows/codecov-checks.yaml

nightly.yamlAdd branch-matrix unit tests and coverage upload +43/-1

Add branch-matrix unit tests and coverage upload
• Adds lint, unit tests, and Codecov coverage upload for main and two release branches. Renames the workflow Nightly Tests to reflect its expanded scope.
.github/workflows/nightly.yaml

test.yamlUpload PR unit-test coverage +10/-0

Upload PR unit-test coverage

• Uploads the generated coverage profile after eligible PR unit-test runs, identifying the PR and head commit. Upload errors do not fail CI.

.github/workflows/test.yaml

.gitignoreIgnore generated Go coverage profile +1/-0

Ignore generated Go coverage profile
• Excludes cover.out from version control after local or CI test runs.
.gitignore

MakefileGenerate coverage during unit tests +1/-1

Generate coverage during unit tests
• Makes the test target write a Go coverage profile to cover.out for local and CI runs.
Makefile

codecov.ymlConfigure informational coverage reporting +29/-0

Configure informational coverage reporting

• Sets informational project and patch coverage statuses, a carryforward unit-test flag, and a PR comment layout. Excludes test, tooling, and entry-point paths from reporting.

codecov.yml

rhdh-qodo-merge · 2026-10-06T11:51:24Z

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

1. ~~Release nightlies can omit coverage~~ ✗ Dismissed

Description

The new unit-tests job checks out each release branch before running make test, so it uses that
branch’s Makefile rather than the Makefile changed in this PR. If either release branch still has
the previous test command without -coverprofile=cover.out, its tests produce no file for the
subsequent non-blocking Codecov upload.

Code

.github/workflows/nightly.yaml[R25-27]

+        with:
+          ref: ${{ matrix.branch }}
+          fetch-depth: 0

Relevance

●●● Strong
Checking out release branches bypasses this PR’s Makefile, potentially preventing the promised
coverage artifact for nightly uploads.
PR-#349

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The matrix checks out main and two release refs, then runs their make test and uploads
cover.out. The current Makefile produces that file only because of the option introduced by this
PR; the release refs are not available in this checkout, so the finding is conditional on their
Makefiles retaining the previous command.

.github/workflows/nightly.yaml[17-27]
.github/workflows/nightly.yaml[42-52]
Makefile[76-78]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Release nightly jobs use each release branch's Makefile, which may not contain this PR's new coverage option. In that case, the upload has no `cover.out` file.
## Fix Focus Areas
- .github/workflows/nightly.yaml[23-27]
- .github/workflows/nightly.yaml[42-52]
- Makefile[76-78]
## Recommended Fix
Make the nightly test step generate `cover.out` independently of release-branch Makefile contents, or backport the coverage-producing test target to every matrix branch before relying on it.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

2. ~~Release changes skip coverage validation~~ ✓ Resolved

Description

codecov-checks.yaml restricts both push and pull-request triggers to main. A change to the
Codecov configuration on a supported release branch therefore receives no validation run, even
though the repository runs tests and builds for release branches.

Code

.github/workflows/codecov-checks.yaml[R10-12]
+  pull_request:
+    branches:
+      - main

Relevance

●●● Strong
Release-branch workflow coverage is an established repository requirement; similar CI changes added
missing release triggers.
PR-#261
PR-#349

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
Both triggers list only main; the existing PR test workflow lists main and a release-branch
pattern, establishing that release branches are supported review targets.
.github/workflows/codecov-checks.yaml[3-13]
.github/workflows/test.yaml[3-8]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new Codecov configuration check does not run for release-branch pushes or pull requests.
## Fix Focus Areas
- .github/workflows/codecov-checks.yaml[3-13]
- .github/workflows/test.yaml[3-8]
## Recommended Fix
Include the supported release-branch pattern in both trigger branch lists while retaining the existing push path filter.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

3. ~~Nightly coverage lacks carryforward~~ ✓ Resolved

Description

codecov.yml enables carryforward only for the unittests flag, despite the nightly upload also
using a nightly flag. When a commit has no nightly upload, the configuration does not request
carryforward for that flag, contrary to the stated nightly-coverage setup.

Code

codecov.yml[R18-23]

+flags:
+  unittests:
+    paths:
+      - cmd/
+      - internal/
+    carryforward: true

Relevance

●●● Strong
The workflow uses a nightly flag, but codecov.yml does not configure carryforward for that flag.
ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The nightly workflow uploads with unittests,nightly, while the complete flags section defines
only unittests and attaches carryforward: true to that entry.

.github/workflows/nightly.yaml[45-50]
codecov.yml[18-24]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Nightly uploads use a flag that has no carryforward configuration in `codecov.yml`.
## Fix Focus Areas
- codecov.yml[18-23]
- .github/workflows/nightly.yaml[45-50]
## Recommended Fix
Add a `nightly` entry under `flags` with the intended paths and `carryforward: true`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

View medium (1)

4. ~~Lint failures hide nightly test results~~ ✓ Resolved

Description

The nightly unit-tests job runs the lint action before make test in the same job. When lint
fails on any matrix branch, that branch never runs its unit tests or reaches its coverage upload, so
the nightly run cannot reveal whether tests also regressed.

Code

.github/workflows/nightly.yaml[R37-43]

+      - name: Run lint
+        uses: golangci/golangci-lint-action@ba0d7d2ec06a0ea1cb5fa41b2e4a3ab91d21278a # v9
+        with:
+          version: v2.14.0
+
+      - name: Run tests
+        run: make test

Relevance

●● Moderate
The ordering can hide tests after lint failures, but no close precedent establishes the team’s
preferred workflow structure.
ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The newly added steps place lint before tests and upload, with no condition allowing the later steps
to run after lint fails.

.github/workflows/nightly.yaml[37-50]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A lint failure stops the new nightly job before unit tests and coverage upload run.
## Fix Focus Areas
- .github/workflows/nightly.yaml[37-50]
## Recommended Fix
Put lint and unit tests in separate jobs or otherwise ensure both checks execute while preserving a failing result for either check.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Context sources

✅ Compliance rules (platform): 4 rules

✅ Cross-repo context — repo relationships

Review mode: Auto: ⚖️ Balanced: Multiple CI workflows and coverage configuration create meaningful but manageable risk.

Tip of the day

💡 Did you know, you can add REVIEW.md to your repo root and Qodo follows it on every PR

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Add lint and unit-tests jobs to the nightly workflow. They run as separate jobs so a lint failure does not block test execution or coverage upload. Unit tests upload coverage to Codecov with the nightly flag across all release branches. The release-1.10 branch is bash-based: it is excluded from the lint job (no golangci-lint config) and its Codecov upload step is skipped (its Makefile does not produce cover.out). Renamed the workflow from "Nightly E2E Tests" to "Nightly Tests" to reflect the broader scope. Assisted-by: Claude

github-actions · 2026-10-06T12:03:11Z

PR images are available (for 1 week):

quay.io/rhdh-community/rhdh-must-gather:pr-435
quay.io/rhdh-community/rhdh-must-gather:pr-435-45e5efa7d

The nightly workflow uploads coverage with the 'nightly' flag, but codecov.yml only defined 'unittests'. Add a 'nightly' flag entry with the same source paths and carryforward enabled so Codecov tracks nightly coverage correctly. Assisted-by: Claude

The codecov-checks workflow only triggered on main, so changes to codecov.yml on release branches or PRs targeting them would not be validated. Add the release branch pattern to both push and pull_request triggers, matching test.yaml. Assisted-by: Claude

rm3l · 2026-10-06T12:13:32Z

/cherrypick release-2.1

openshift-cherrypick-robot · 2026-10-06T12:13:35Z

@rm3l: once the present PR merges, I will cherry-pick it on top of release-2.1 in a new PR and assign it to you.

Details

In response to this:

/cherrypick release-2.1

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

github-actions · 2026-10-06T12:19:12Z

PR images are available (for 1 week):

quay.io/rhdh-community/rhdh-must-gather:pr-435
quay.io/rhdh-community/rhdh-must-gather:pr-435-758817ebe

openshift-ci Bot requested review from Fortune-Ndlovu and zdrapela October 6, 2026 11:48

rhdh-qodo-merge Bot reviewed Oct 6, 2026

View reviewed changes

Comment thread .github/workflows/nightly.yaml

Comment thread .github/workflows/nightly.yaml Outdated

Comment thread codecov.yml

Comment thread .github/workflows/codecov-checks.yaml

rm3l added 2 commits October 6, 2026 14:03

rm3l force-pushed the RHIDP-14400--collect-must-gather-coverage-upload-to-codecov-io branch from 45e5efa to 758817e Compare October 6, 2026 12:05

rm3l mentioned this pull request Oct 6, 2026

test: improve Go test idioms across the codebase [RHIDP-14400] #421

Open

1 of 2 tasks

Navigation Menu

Repository navigation

ci: upload unit test coverage to Codecov [RHIDP-14400] - #435

ci: upload unit test coverage to Codecov [RHIDP-14400]#435
rm3l wants to merge 4 commits into
redhat-developer:mainfrom
rm3l:RHIDP-14400--collect-must-gather-coverage-upload-to-codecov-io

rm3l commented Oct 6, 2026 •

edited

Loading

Uh oh!

rhdh-qodo-merge Bot commented Oct 6, 2026

Uh oh!

rhdh-qodo-merge Bot commented Oct 6, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Oct 6, 2026

Uh oh!

rm3l commented Oct 6, 2026

Uh oh!

openshift-cherrypick-robot commented Oct 6, 2026

Uh oh!

github-actions Bot commented Oct 6, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

rm3l commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Which issue(s) does this PR fix or relate to

PR acceptance criteria

How to test changes / Special notes to the reviewer

Uh oh!

rhdh-qodo-merge Bot commented Oct 6, 2026

PR Summary by Qodo

Uh oh!

rhdh-qodo-merge Bot commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review by Qodo

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Oct 6, 2026

Uh oh!

rm3l commented Oct 6, 2026

Uh oh!

openshift-cherrypick-robot commented Oct 6, 2026

Uh oh!

github-actions Bot commented Oct 6, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

rm3l commented Oct 6, 2026 •

edited

Loading

rhdh-qodo-merge Bot commented Oct 6, 2026 •

edited

Loading