Skip to content

Skipped test should be forbidden #750

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
fcollonval opened this issue Mar 21, 2025 · 0 comments
Open

Skipped test should be forbidden #750

fcollonval opened this issue Mar 21, 2025 · 0 comments
Labels
bug Something isn't working status:Needs Triage

Comments

@fcollonval
Copy link
Member

fcollonval commented Mar 21, 2025

Description

In #745, the configuration to forbid only and pending tests on CI was added. This successfully prevent the use of .only on CI. But actually does not forbid .skip as seen in the CI log of #745: https://github.com/jupyterlab/lumino/actions/runs/13718590093/job/38368769283

Relevant CI log for posterity > @lumino/widgets:"test:chromium"

@lumino/[email protected] test
npm run build:test && web-test-runner tests/lib/bundle.test.js --node-resolve --playwright --browsers chromium

@lumino/[email protected] build:test
npm run clean:test && tsc --build tests && cd tests && rollup -c

@lumino/[email protected] clean:test
rimraf tests/lib tests/tsconfig.tsbuildinfo

./lib/index.spec.js → ./lib/bundle.test.js...
(!) Circular dependencies
../../../node_modules/chai/lib/chai.js -> ../../../node_modules/chai/lib/chai/utils/index.js -> ../../../node_modules/chai/lib/chai/utils/addProperty.js -> ../../../node_modules/chai/lib/chai.js
../../../node_modules/chai/lib/chai.js -> ../../../node_modules/chai/lib/chai/utils/index.js -> ../../../node_modules/chai/lib/chai/utils/addMethod.js -> ../../../node_modules/chai/lib/chai.js
../../../node_modules/chai/lib/chai.js -> ../../../node_modules/chai/lib/chai/utils/index.js -> ../../../node_modules/chai/lib/chai/utils/overwriteProperty.js -> ../../../node_modules/chai/lib/chai.js
...and 3 more
created ./lib/bundle.test.js in 1.9s

Chromium: |██████████████████████████████| 0/1 test files | 0 passed, 0 failed

Running tests...

Running 1 test files...

Chromium: |██████████████████████████████| 1/1 test files | 938 passed, 0 failed, 1 skipped

Reproduce

  1. Flag a test with it.skip
  2. Set environment variable CI=1
  3. Execute the tests, it does not fail

Expected behavior

When on CI, skipped tests should result in failing tests.

Important

The test do failed when run in debug mode within the browser.

@fcollonval fcollonval added bug Something isn't working status:Needs Triage labels Mar 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working status:Needs Triage
Projects
None yet
Development

No branches or pull requests

1 participant