Repository navigation
Refactoring unit and integration tests - #1917
Conversation
There was a problem hiding this comment.
Pull request overview
This PR refactors and reorganizes the unit and integration test suite for Quint. The main changes update test expectations to match recent backend changes (replacing TypeScript with Rust, adding stack traces to runtime errors, and introducing diagnostics), reorganize integration tests into a clearer folder structure, and update CI workflows for consistency.
Changes:
- Refactored integration tests into organized subdirectories under
integration-tests/(lang, runtime/typescript, runtime/rust, verification, distribution) - Updated test expectations to match current Rust backend output (stack traces, diagnostics, error formatting)
- Fixed output routing for debug diagnostics in Rust REPL to use provided writer instead of console.log
- Renamed and reorganized CI workflow files for better clarity and consistency
Reviewed changes
Copilot reviewed 30 out of 33 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
quint/testFixture/runtimeErrorStackTrace.qnt |
New fixture for testing runtime error stack traces |
quint/test/runtime/rust/repl.test.ts |
Updated import paths and added backend: 'rust' flag |
quint/test/runtime/deprecated/repl.test.ts |
Updated import paths for deprecated TypeScript backend tests |
quint/test/runtime/deprecated/trace.test.ts |
Updated import paths |
quint/test/runtime/deprecated/compile.test.ts |
Updated import paths |
quint/src/rust/replServerWrapper.ts |
Added out parameter to route diagnostic output properly |
quint/src/repl.ts |
Threaded out writer through to ReplServerWrapper constructor |
quint/package.json |
Updated npm scripts to reflect new integration test structure |
quint/rust-backend-local-tests.md |
Deleted (moved to integration-tests) |
quint/rust-backend-integration-tests/quint-test.md |
Deleted (moved to integration-tests) |
quint/io-cli-tests.md |
Deleted (moved to integration-tests/lang/io.md) |
quint/cli-tests.md |
Deleted (moved to integration-tests/lang/cli.md) |
quint/integration-tests/verification.md |
Minor formatting fix |
quint/integration-tests/runtime/typescript/test.md |
New organized TypeScript backend test file |
quint/integration-tests/runtime/typescript/run.md |
New organized TypeScript backend run tests |
quint/integration-tests/runtime/typescript/repl.md |
New organized TypeScript backend REPL tests |
quint/integration-tests/runtime/rust/test.md |
New organized Rust backend test file |
quint/integration-tests/runtime/rust/run.md |
New organized Rust backend run tests with updated expectations |
quint/integration-tests/runtime/rust/repl.md |
New organized Rust backend REPL tests with updated expectations |
quint/integration-tests/lang/io.md |
Reorganized language IO tests |
quint/integration-tests/lang/cli.md |
Reorganized language CLI tests |
quint/integration-tests/distribution/rust-release.md |
New distribution tests for Rust release downloads |
quint/integration-tests/distribution/apalache.md |
New distribution tests for Apalache downloads |
CONTRIBUTING.md |
Updated documentation for new test organization |
.github/workflows/*.yml |
Renamed and reorganized CI workflow files |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
bugarela
left a comment
There was a problem hiding this comment.
Looking good. I have some minor things, and a lack of understanding on the unit test changes
@bugarela , I have to revert back the repl tests for the deprecated backend to main and apply some small changes to the test expectations (to include state change displays on verbosity=2) which were not covered by the expected output because that part of the code was writing directly to the console log instead of the out const and were thus not displayed in the test output (which uses it's own output). I guess that file was the source of the confusion. |
bugarela
left a comment
There was a problem hiding this comment.
LGTM! Thanks for taking this on and fixing the bugs you found in the way!
Hello,
this is a PR to update and organize the unit tests, the integration tests and the CI workflows. A big chunk of the PR is composed of updates to the expectations of the unit and integration tests to match to the latest updates (replacing the TS backend with Rust, adding stack traces to runtime errors, introducing diagnostics...).
I also changes the behavior of the integration tests, so now most of the tests (beside the distribution tests) build a rust evaluator from
./evaluatorinstead of downloading the released evaluator and grouped them in a dedicated folder with a better separation of concerns.I also renamed some of the CI workflows for better consistency with the rest of the CI checks.
I would really appreciate if you can double check with me if I am using the right
--backendflag in the updated tests, I might have missed some tests.(including screenshots is helpful)
CHANGELOG.mdfor any new functionality