Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: test report summary output #2900

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

crazy-max
Copy link
Member

Aggregate data from JSON reports of Go tests using gotestsum and adds a step for unit and integration tests that will render the output as markdown and create a GitHub summary in our workflow using teststat:

image

Also some useful stats about slow tests:

image

Was also thinking using tparse to have a better overview: https://github.com/crazy-max/buildkit/actions/runs/2453700879#summary-6772166685

image

But output is quite huge.

@thaJeztah Maybe gotestsum could also generate a markdown summary output so we don't need extra tools. WDYT?

@crazy-max crazy-max requested a review from tonistiigi June 14, 2022 08:20
@crazy-max crazy-max force-pushed the ci-test-summary branch 3 times, most recently from a517d86 to dc1335b Compare June 14, 2022 23:45
@crazy-max crazy-max marked this pull request as draft July 30, 2022 22:08
@crazy-max
Copy link
Member Author

As discussed with @tonistiigi, resulting summary might not be useful in the current state. What would be interesting is to capture popular failure points over time to catch performance regressions. Without dedicated nodes it looks difficult to catch atm.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant