Migrate weekly performance tests to Vitest - #358
Merged
Conversation
Replace Mocha delayed registration with serialized Vitest collection and explicit worker lifecycle cleanup while preserving the weekly report and authentication contracts. Validated transformer build, performance test type-check, lint, and focused cleanup/registration tests. The credentialed suite has not been executed locally; live Bentley Hub runtime behavior remains to be validated by the weekly ADO pipeline. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
|
Can you have one perf pipeline run against this branch? To verify things work and that we don't have to do any updates in our internal repo's yaml |
hl662
reviewed
Jul 29, 2026
hl662
left a comment
hl662
left a comment
Contributor
There was a problem hiding this comment.
Fluff out either the README.md for the performance tests, or have a separate ARCHITECTURE.md for the performance tests package (i think the latter). Seems like we'll have to explain a high level overview of how our perf tests are structured and setup at the end of our stacked PRs
Keep credential loading in the Hub-backed regression suite so local helper unit tests run without CI or a .env file. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Use IModelHost validity for partial-startup cleanup and keep unlimited timeouts local to the credentialed weekly regression file. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Rely on Vitest's per-test reporting instead of emitting an unstructured completion message. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Explain the weekly regression lifecycle, helper unit tests, registration matrix, cleanup, reporting contract, and extension boundaries. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
hl662
approved these changes
Jul 29, 2026
hl662
left a comment
hl662
left a comment
Contributor
There was a problem hiding this comment.
nice, let's get this in and move on to next parts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
@types/mocha, Chai 4, andts-nodeNew test helpers
These helpers are not required by Vitest. They make behavior that was embedded in the previous Mocha suite explicit and independently testable:
Cleanup.tsguarantees that every registered teardown task is attempted, including database closure, report export,IModelHostshutdown, and authorization sign-out. If the operation and cleanup both fail, it preserves both errors instead of allowing teardown to hide the primary failure.RegressionTestRegistration.tsconstructs the supported test-case/transformer-module combinations synchronously before Vitest registers tests. It skips modules that do not implement the operation required by a test case, preserving the old conditional Mocha registration behavior without asynchronous collection callbacks.Cleanup.test.tsandRegressionTestRegistration.test.tsprovide focused, non-credentialed coverage for those behaviors.This PR targets
mainand is intended as the new bottom layer beneath the existing quick-performance PRs #353, #354, #356, and #357. It does not modify those branches or their stack metadata.Validation
pnpm --dir packages/transformer buildpnpm --dir packages/performance-tests buildpnpm --dir packages/performance-tests lintCI=true pnpm --dir packages/performance-tests exec vitest run test/unit/Cleanup.test.ts test/unit/RegressionTestRegistration.test.ts(4 tests passed)@types/mocha, Chai 4, orts-nodeThe credentialed Hub-backed weekly runtime suite was intentionally not run locally. Hosted weekly Azure validation remains deferred, consistent with the source migration handoff.
generated with GPT-5.6 Sol