Skip to content

Refactoring unit and integration tests - #1917

Merged
beu5a merged 33 commits into
mainfrom
yassine/ci-tests-refactoring
Feb 26, 2026
Merged

beu5a merged 33 commits into
mainfrom
yassine/ci-tests-refactoring

Conversation

@beu5a

@beu5a beu5a commented Feb 24, 2026 •

Copy link
Copy Markdown
Collaborator

Hello,
this is a PR to update and organize the unit tests, the integration tests and the CI workflows. A big chunk of the PR is composed of updates to the expectations of the unit and integration tests to match to the latest updates (replacing the TS backend with Rust, adding stack traces to runtime errors, introducing diagnostics...).

I also changes the behavior of the integration tests, so now most of the tests (beside the distribution tests) build a rust evaluator from ./evaluator instead of downloading the released evaluator and grouped them in a dedicated folder with a better separation of concerns.
I also renamed some of the CI workflows for better consistency with the rest of the CI checks.

I would really appreciate if you can double check with me if I am using the right --backend flag in the updated tests, I might have missed some tests.

  • I have read and I understand the Note on AI-assisted contributions
  • Changes manually tested locally and confirmed to work as described
    (including screenshots is helpful)
  • Tests added for any new code
  • Documentation added for any new functionality
  • Entries added to the respective CHANGELOG.md for any new functionality

beu5a and others added 29 commits February 24, 2026 09:55
@beu5a
beu5a marked this pull request as ready for review February 25, 2026 15:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors and reorganizes the unit and integration test suite for Quint. The main changes update test expectations to match recent backend changes (replacing TypeScript with Rust, adding stack traces to runtime errors, and introducing diagnostics), reorganize integration tests into a clearer folder structure, and update CI workflows for consistency.

Changes:

  • Refactored integration tests into organized subdirectories under integration-tests/ (lang, runtime/typescript, runtime/rust, verification, distribution)
  • Updated test expectations to match current Rust backend output (stack traces, diagnostics, error formatting)
  • Fixed output routing for debug diagnostics in Rust REPL to use provided writer instead of console.log
  • Renamed and reorganized CI workflow files for better clarity and consistency

Reviewed changes

Copilot reviewed 30 out of 33 changed files in this pull request and generated no comments.

Show a summary per file
File Description
quint/testFixture/runtimeErrorStackTrace.qnt New fixture for testing runtime error stack traces
quint/test/runtime/rust/repl.test.ts Updated import paths and added backend: 'rust' flag
quint/test/runtime/deprecated/repl.test.ts Updated import paths for deprecated TypeScript backend tests
quint/test/runtime/deprecated/trace.test.ts Updated import paths
quint/test/runtime/deprecated/compile.test.ts Updated import paths
quint/src/rust/replServerWrapper.ts Added out parameter to route diagnostic output properly
quint/src/repl.ts Threaded out writer through to ReplServerWrapper constructor
quint/package.json Updated npm scripts to reflect new integration test structure
quint/rust-backend-local-tests.md Deleted (moved to integration-tests)
quint/rust-backend-integration-tests/quint-test.md Deleted (moved to integration-tests)
quint/io-cli-tests.md Deleted (moved to integration-tests/lang/io.md)
quint/cli-tests.md Deleted (moved to integration-tests/lang/cli.md)
quint/integration-tests/verification.md Minor formatting fix
quint/integration-tests/runtime/typescript/test.md New organized TypeScript backend test file
quint/integration-tests/runtime/typescript/run.md New organized TypeScript backend run tests
quint/integration-tests/runtime/typescript/repl.md New organized TypeScript backend REPL tests
quint/integration-tests/runtime/rust/test.md New organized Rust backend test file
quint/integration-tests/runtime/rust/run.md New organized Rust backend run tests with updated expectations
quint/integration-tests/runtime/rust/repl.md New organized Rust backend REPL tests with updated expectations
quint/integration-tests/lang/io.md Reorganized language IO tests
quint/integration-tests/lang/cli.md Reorganized language CLI tests
quint/integration-tests/distribution/rust-release.md New distribution tests for Rust release downloads
quint/integration-tests/distribution/apalache.md New distribution tests for Apalache downloads
CONTRIBUTING.md Updated documentation for new test organization
.github/workflows/*.yml Renamed and reorganized CI workflow files

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@bugarela bugarela left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good. I have some minor things, and a lack of understanding on the unit test changes

Comment thread quint/integration-tests/lang/io.md
Comment thread quint/integration-tests/runtime/rust/test.md Outdated
Comment thread quint/test/runtime/rust/repl.test.ts
Comment thread quint/test/runtime/rust/repl.test.ts
Comment thread quint/test/runtime/deprecated/repl.test.ts Outdated
Comment thread quint/test/runtime/deprecated/repl.test.ts
Comment thread quint/test/runtime/rust/repl.test.ts
@beu5a

beu5a commented Feb 25, 2026 •

Copy link
Copy Markdown
Collaborator Author

Looking good. I have some minor things, and a lack of understanding on the unit test changes

@bugarela , I have to revert back the repl tests for the deprecated backend to main and apply some small changes to the test expectations (to include state change displays on verbosity=2) which were not covered by the expected output because that part of the code was writing directly to the console log instead of the out const and were thus not displayed in the test output (which uses it's own output). I guess that file was the source of the confusion.

@bugarela bugarela left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for taking this on and fixing the bugs you found in the way!

@beu5a beu5a mentioned this pull request Feb 26, 2026
2 of 5 tasks

@erickpintor erickpintor left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@beu5a
beu5a enabled auto-merge February 26, 2026 14:20
@beu5a
beu5a disabled auto-merge February 26, 2026 14:40
@beu5a
beu5a enabled auto-merge February 26, 2026 14:40
@beu5a
beu5a merged commit 512d354 into main Feb 26, 2026
50 of 51 checks passed
@beu5a
beu5a deleted the yassine/ci-tests-refactoring branch February 26, 2026 15:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants