Skip to content

feat(action): Add reusable GitHub action#8

Merged
andreiborza merged 3 commits into
mainfrom
ab/github-action
Dec 4, 2025
Merged

feat(action): Add reusable GitHub action#8
andreiborza merged 3 commits into
mainfrom
ab/github-action

Conversation

@andreiborza

@andreiborza andreiborza commented Dec 4, 2025

Copy link
Copy Markdown
Member

This PR adds a GitHub action that upstream repos at Sentry can use to run it on their CI and get alerted about failures via issues created in their repo.

Note: this is all vibe-coded and might require a couple of back and forths for testing, which is notoriously difficult with actions.

Note 2: A couple of things could be improved, for example having a way to run npm run cli setup specific to the language passed so that the action doesn't require setting up runtime and fetching SDKs for all supported languages but that's something we can do in followup PRs and is mainly a question of improving the cli.

Comment thread action.yml

// Try to read test results from CTRF report
let resultsContent = 'Test execution failed. Check the [workflow run](' + runUrl + ') for details.';
const ctrfPath = '${{ github.action_path }}/shared/orchestration/test-results/ctrf-report.json';

This comment was marked as outdated.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not correct.

The cli script in package.json has cd shared/orchestration && npm run cli --

So when we run npm run cli run js it:

  1. Changes directory to shared/orchestration
  2. Runs the CLI from there
  3. The ./test-results path is relative to shared/orchestration, so the output goes to shared/orchestration/test-results/

@sentrivana sentrivana left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TY! Looks good overall afaict, left a couple comments

Comment thread action.yml Outdated
Comment thread action.yml Outdated
Comment thread README.md Outdated
Co-authored-by: Ivana Kellyer <ivana.kellyer@sentry.io>
Comment thread action.yml

// Try to read test results from CTRF report
let resultsContent = 'Test execution failed. Check the [workflow run](' + runUrl + ') for details.';
const ctrfPath = '${{ github.action_path }}/shared/orchestration/test-results/ctrf-report.json';

This comment was marked as outdated.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this should be the correct path.

Comment thread action.yml
Comment on lines +120 to +125
const issues = await github.rest.issues.listForRepo({
owner: context.repo.owner,
repo: context.repo.repo,
labels: issueLabel,
state: 'open'
});

This comment was marked as outdated.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The action is intended to be used with a single language, should be fine.

Comment thread action.yml

// Try to read test results from CTRF report
let resultsContent = 'Test execution failed. Check the [workflow run](' + runUrl + ') for details.';
const ctrfPath = '${{ github.action_path }}/shared/orchestration/test-results/ctrf-report.json';

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: CTRF test report is written to test-results/ctrf-report.json but action.yml expects it at shared/orchestration/test-results/ctrf-report.json, causing a file not found error.
Severity: CRITICAL | Confidence: High

🔍 Detailed Analysis

The GitHub action has a path mismatch for the CTRF test report file. The Run tests step writes the CTRF report to ${{ github.action_path }}/test-results/ctrf-report.json because the --output-dir flag is not explicitly provided, and the CLI defaults to ./test-results. However, action.yml line 85 expects the file at ${{ github.action_path }}/shared/orchestration/test-results/ctrf-report.json. This mismatch causes fs.existsSync(ctrfPath) to always return false, leading to the fallback message being used and detailed test results never being included in GitHub issues, contradicting documented behavior.

💡 Suggested Fix

Either add --output-dir ./shared/orchestration/test-results to the CLI command on action.yml line 61, or change the expected path on action.yml line 85 to ./test-results/ctrf-report.json.

🤖 Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.

Location: action.yml#L85

Potential issue: The GitHub action has a path mismatch for the CTRF test report file.
The `Run tests` step writes the CTRF report to `${{ github.action_path
}}/test-results/ctrf-report.json` because the `--output-dir` flag is not explicitly
provided, and the CLI defaults to `./test-results`. However, `action.yml` line 85
expects the file at `${{ github.action_path
}}/shared/orchestration/test-results/ctrf-report.json`. This mismatch causes
`fs.existsSync(ctrfPath)` to always return `false`, leading to the fallback message
being used and detailed test results never being included in GitHub issues,
contradicting documented behavior.

Did we get this right? 👍 / 👎 to inform future reviews.
Reference ID: 5491106

@sentrivana sentrivana left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ty!

@andreiborza andreiborza merged commit a045597 into main Dec 4, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants