Skip to content

feat(ci): add test coverage check - #75

Closed
barrettruth wants to merge 28 commits into
nvimdev:mainfrom
barrettruth:feat/coverage-check
Closed

barrettruth wants to merge 28 commits into
nvimdev:mainfrom
barrettruth:feat/coverage-check

Conversation

@barrettruth

Copy link
Copy Markdown
Contributor

Adds a POSIX sh script (scripts/check-coverage) that parses
formatter.lua and linter/init.lua for tool definitions, compares
against existing *_spec.lua test files, and exits non-zero when any
tool is missing a test.

CI runs this as the first job (coverage). This PR will fail CI —
32/84 tools currently have tests. The job is intentionally hard-failing
so that missing coverage is visible on every push and PR.

The check-coverage Makefile target is also available for local use.

# Conflicts:
#	test/linter/cpplint_spec.lua
replaces monolithic test job with parallel jobs grouped by package manager.
each job installs only what it needs and runs its specific tests.

- uses nvim-neorocks/nvim-busted-action for neovim + vusted setup
- updates all actions to latest versions (checkout@v4, setup-python@v5, etc)
- adds coverage job to warn when tools are missing tests
- removes setup.sh and env.sh (no longer needed)
- Strip _extmark_id in linter helper (internal nvim field)
- Update tests to use separate code field instead of message suffix
- Update end_col values to match guard.nvim behavior
- Update rustfmt expected output for new import formatting style
- Update buf formatter expected output (compact style)
- Add missing W292 diagnostic to flake8 test
- Remove buf linter test (requires buf.yaml config)
Move tests from test/{formatter,linter}/ to test/{pip,npm,go,rust,lua,binary,clang,haskell}/
to allow CI to use glob patterns (busted test/pip/*_spec.lua) instead of
listing each test file explicitly.

This means adding new tools no longer requires workflow changes.
Run each clang test file in its own busted process to prevent
test state from leaking between clang-format and clang-tidy tests.
Revert checkmake linter to original fn-based approach.
Update test to match actual diagnostic output order.
Original test was named swiftformat.lua (missing _spec suffix)
so busted never ran it. Restored with correct naming and
added swiftformat installation to CI.
# Conflicts:
#	test/lua/cpplint_spec.lua
#	test/lua/zsh_spec.lua
Old test helpers used guard.nvim's async pipeline with a hardcoded
vim.wait(3000), causing flaky CI failures. New approach runs tools
directly via vim.system():wait() — zero timing issues, tests real
tool output, no dependency on guard.nvim's format/filetype modules.

This commit strips all existing tests and CI jobs down to a single
MVP (black formatter + flake8 linter) to validate the approach.
Remaining tools will be re-added in a follow-up PR.

BREAKING CHANGE: test helpers renamed from fmt_helper/lint_helper
to a unified test/helper.lua with run_fmt(), run_lint(), get_linter()
Re-adds all previously covered tools using the new synchronous test
infrastructure from the prior commit. All 8 CI job categories restored
(pip, npm, go, rust, lua, binary, clang, haskell).

Linter assertions now use structural checks (has diagnostics, correct
source/bufnr/types) instead of exact field matching, preventing
breakage when tool versions change output formatting.
POSIX sh script that extracts tool names from formatter.lua and
linter/init.lua, compares against existing test specs, and fails hard
when any tool lacks a test.
@barrettruth barrettruth closed this Feb 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant