Repository navigation
test: verify Busser works with Test Kitchen in CI - #84
Merged
Merged
Conversation
config:recommended turns the dependency dashboard on, so Renovate keeps an issue open in every repository restating what its pull requests already say. Across seven repositories that is seven standing issues nobody reads. `:disableDependencyDashboard` turns it off. Renovate still opens the update pull requests exactly as before; it just stops narrating them in an issue. Signed-off-by: Tim Smith <tsmith84@proton.me>
These plugins exist to be run by Test Kitchen, and nothing in CI ever ran them that way. The cucumber features drive `busser` directly, which covers the plugin but skips the whole verifier path -- how Test Kitchen installs busser, installs the plugin, transfers the suite and invokes it. This adds a `kitchen verify` job that runs the real thing: * **`driver: exec` with `transport: exec`**, both shipped with Test Kitchen, so the "machine under test" is the CI runner. No Docker, no VM, no credentials, a couple of seconds per run. * **The real `busser` verifier**, pointed at the running Ruby rather than Chef's omnibus and told not to sudo for a path inside the workspace. * **`kitchen-preinstall.sh`** puts this working tree into the Busser root first, so the job tests the branch rather than the last release from RubyGems. `test/integration/default/` holds a small suite in the plugin's own language. ## Two things worth knowing about the setup **It deliberately does not use bundler.** No `bundler-cache`, no `bundle exec`. The verifier shells out to `gem list` to decide what to install, and bundler in the environment makes that report the bundle's gems rather than the Busser root's -- so it skips installing busser and the run dies on a path that was never created. Test Kitchen is not run from inside a project's bundle in real use either. **The pre-install has to install busser itself and run the postinstall.** The verifier's check is `grep "^busser"` with no anchor at the end, so a plugin named `busser-*` satisfies it and busser would never be installed. And skipping `busser plugin install` also skips the plugin's postinstall, which is where a plugin installs the framework it drives. Verified both directions locally: a passing suite exits 0, and a deliberately failing test makes `kitchen verify` exit 20. The suite here uses busser-bash as the exercise -- a real plugin, installed by the verifier from RubyGems, driven by the busser built from this working tree. The internal dummy runner would prove the chain runs but can never fail, so it would not notice a regression. The test asserts on the environment Busser is supposed to hand a suite, which is where the isolation bugs live. Signed-off-by: Tim Smith <tsmith84@proton.me>
The runners require "bundler/setup", which walks up from the suite looking for
a Gemfile. With the Busser root under ${{ github.workspace }} it found this
project's Gemfile and tried to materialize that bundle inside the isolated gem
home, which cannot work:
```text
Could not find cookstyle-9.0.0, aruba-2.4.1, cucumber-11.1.1, ...
in locally installed gems (Bundler::GemNotFound)
```
A real Busser root is /opt/busser -- never inside the project -- so this was an
artefact of the CI layout rather than anything the plugins do. Moving it to
/tmp/busser-kitchen matches reality and fixes it.
It only showed up on CI because a root outside any Gemfile tree, which is what I
had locally, makes bundler/setup a no-op.
Signed-off-by: Tim Smith <tsmith84@proton.me>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
These plugins exist to be run by Test Kitchen, and nothing in CI ever ran them
that way. The cucumber features drive
busserdirectly, which covers the pluginbut skips the whole verifier path -- how Test Kitchen installs busser, installs
the plugin, transfers the suite and invokes it.
This adds a
kitchen verifyjob that runs the real thing:driver: execwithtransport: exec, both shipped with Test Kitchen, sothe "machine under test" is the CI runner. No Docker, no VM, no credentials,
a couple of seconds per run.
busserverifier, pointed at the running Ruby rather thanChef's omnibus and told not to sudo for a path inside the workspace.
kitchen-preinstall.shputs this working tree into the Busser root first,so the job tests the branch rather than the last release from RubyGems.
test/integration/default/holds a small suite in the plugin's own language.Two things worth knowing about the setup
It deliberately does not use bundler. No
bundler-cache, nobundle exec.The verifier shells out to
gem listto decide what to install, and bundler inthe environment makes that report the bundle's gems rather than the Busser
root's -- so it skips installing busser and the run dies on a path that was
never created. Test Kitchen is not run from inside a project's bundle in real
use either.
The pre-install has to install busser itself and run the postinstall. The
verifier's check is
grep "^busser"with no anchor at the end, so a pluginnamed
busser-*satisfies it and busser would never be installed. And skippingbusser plugin installalso skips the plugin's postinstall, which is where aplugin installs the framework it drives.
Verified both directions locally: a passing suite exits 0, and a deliberately
failing test makes
kitchen verifyexit 20.The suite here uses busser-bash as the exercise -- a real plugin, installed by
the verifier from RubyGems, driven by the busser built from this working tree.
The internal dummy runner would prove the chain runs but can never fail, so it
would not notice a regression. The test asserts on the environment Busser is
supposed to hand a suite, which is where the isolation bugs live.