Skip to content

Test diffs are too hard to visualize on CI #1580

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
steve-chavez opened this issue May 1, 2025 · 0 comments
Open

Test diffs are too hard to visualize on CI #1580

steve-chavez opened this issue May 1, 2025 · 0 comments

Comments

@steve-chavez
Copy link
Member

Problem

When CI fails due to an error in the tests, they're hard to visualize:

  • The raw logs have to be accessed.
  • Then one has to CTRL +F "regression.diffs".

The feedback must be immediate or otherwise fixing tests is a chore, team members are not fixing them quickly. See an example here #994 (comment).

Solution

Provide a clean way to see the regression.diffs on CI, without having to access raw logs.

Notes

The CI logs contain a full dump of the bash scripts used, the successful output of tests, all of which is completely unnecessary. It should only contain errors.

@steve-chavez steve-chavez added bug Something isn't working and removed bug Something isn't working labels May 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant