Repository navigation
feat(ci): add test coverage check - #75
Closed
barrettruth wants to merge 28 commits into
Closed
barrettruth wants to merge 28 commits into
barrettruth wants to merge 28 commits into
Conversation
# Conflicts: # test/linter/cpplint_spec.lua
replaces monolithic test job with parallel jobs grouped by package manager. each job installs only what it needs and runs its specific tests. - uses nvim-neorocks/nvim-busted-action for neovim + vusted setup - updates all actions to latest versions (checkout@v4, setup-python@v5, etc) - adds coverage job to warn when tools are missing tests - removes setup.sh and env.sh (no longer needed)
- Strip _extmark_id in linter helper (internal nvim field) - Update tests to use separate code field instead of message suffix - Update end_col values to match guard.nvim behavior - Update rustfmt expected output for new import formatting style - Update buf formatter expected output (compact style) - Add missing W292 diagnostic to flake8 test - Remove buf linter test (requires buf.yaml config)
Move tests from test/{formatter,linter}/ to test/{pip,npm,go,rust,lua,binary,clang,haskell}/
to allow CI to use glob patterns (busted test/pip/*_spec.lua) instead of
listing each test file explicitly.
This means adding new tools no longer requires workflow changes.
Run each clang test file in its own busted process to prevent test state from leaking between clang-format and clang-tidy tests.
Revert checkmake linter to original fn-based approach. Update test to match actual diagnostic output order.
Original test was named swiftformat.lua (missing _spec suffix) so busted never ran it. Restored with correct naming and added swiftformat installation to CI.
This reverts commit 51d94b1.
This reverts commit 3b5c3af.
# Conflicts: # test/lua/cpplint_spec.lua # test/lua/zsh_spec.lua
Old test helpers used guard.nvim's async pipeline with a hardcoded vim.wait(3000), causing flaky CI failures. New approach runs tools directly via vim.system():wait() — zero timing issues, tests real tool output, no dependency on guard.nvim's format/filetype modules. This commit strips all existing tests and CI jobs down to a single MVP (black formatter + flake8 linter) to validate the approach. Remaining tools will be re-added in a follow-up PR. BREAKING CHANGE: test helpers renamed from fmt_helper/lint_helper to a unified test/helper.lua with run_fmt(), run_lint(), get_linter()
Re-adds all previously covered tools using the new synchronous test infrastructure from the prior commit. All 8 CI job categories restored (pip, npm, go, rust, lua, binary, clang, haskell). Linter assertions now use structural checks (has diagnostics, correct source/bufnr/types) instead of exact field matching, preventing breakage when tool versions change output formatting.
POSIX sh script that extracts tool names from formatter.lua and linter/init.lua, compares against existing test specs, and fails hard when any tool lacks a test.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a POSIX sh script (
scripts/check-coverage) that parsesformatter.lua and linter/init.lua for tool definitions, compares
against existing
*_spec.luatest files, and exits non-zero when anytool is missing a test.
CI runs this as the first job (
coverage). This PR will fail CI —32/84 tools currently have tests. The job is intentionally hard-failing
so that missing coverage is visible on every push and PR.
The
check-coverageMakefile target is also available for local use.