Skip to content

test: verify the plugin works with Test Kitchen in CI - #27

Merged
tas50 merged 2 commits into
mainfrom
kitchen-integration
Aug 29, 2026
Merged

tas50 merged 2 commits into
mainfrom
kitchen-integration

Conversation

@tas50

@tas50 tas50 commented Aug 28, 2026

Copy link
Copy Markdown
Member

These plugins exist to be run by Test Kitchen, and nothing in CI ever ran them
that way. The cucumber features drive busser directly, which covers the plugin
but skips the whole verifier path -- how Test Kitchen installs busser, installs
the plugin, transfers the suite and invokes it.

This adds a kitchen verify job that runs the real thing:

  • driver: exec with transport: exec, both shipped with Test Kitchen, so
    the "machine under test" is the CI runner. No Docker, no VM, no credentials,
    a couple of seconds per run.
  • The real busser verifier, pointed at the running Ruby rather than
    Chef's omnibus and told not to sudo for a path inside the workspace.
  • kitchen-preinstall.sh puts this working tree into the Busser root first,
    so the job tests the branch rather than the last release from RubyGems.

test/integration/default/ holds a small suite in the plugin's own language.

Two things worth knowing about the setup

It deliberately does not use bundler. No bundler-cache, no bundle exec.
The verifier shells out to gem list to decide what to install, and bundler in
the environment makes that report the bundle's gems rather than the Busser
root's -- so it skips installing busser and the run dies on a path that was
never created. Test Kitchen is not run from inside a project's bundle in real
use either.

The pre-install has to install busser itself and run the postinstall. The
verifier's check is grep "^busser" with no anchor at the end, so a plugin
named busser-* satisfies it and busser would never be installed. And skipping
busser plugin install also skips the plugin's postinstall, which is where a
plugin installs the framework it drives.

Verified both directions locally: a passing suite exits 0, and a deliberately
failing test makes kitchen verify exit 20.

The suite includes a helper.sh, which is not named *_test.sh or *_spec.sh
and so must never run. If the glob ever widens, it fails the suite loudly.

tas50 added 2 commits August 28, 2026 16:59
These plugins exist to be run by Test Kitchen, and nothing in CI ever ran them
that way. The cucumber features drive `busser` directly, which covers the plugin
but skips the whole verifier path -- how Test Kitchen installs busser, installs
the plugin, transfers the suite and invokes it.

This adds a `kitchen verify` job that runs the real thing:

* **`driver: exec` with `transport: exec`**, both shipped with Test Kitchen, so
  the "machine under test" is the CI runner. No Docker, no VM, no credentials,
  a couple of seconds per run.
* **The real `busser` verifier**, pointed at the running Ruby rather than
  Chef's omnibus and told not to sudo for a path inside the workspace.
* **`kitchen-preinstall.sh`** puts this working tree into the Busser root first,
  so the job tests the branch rather than the last release from RubyGems.

`test/integration/default/` holds a small suite in the plugin's own language.

## Two things worth knowing about the setup

**It deliberately does not use bundler.** No `bundler-cache`, no `bundle exec`.
The verifier shells out to `gem list` to decide what to install, and bundler in
the environment makes that report the bundle's gems rather than the Busser
root's -- so it skips installing busser and the run dies on a path that was
never created. Test Kitchen is not run from inside a project's bundle in real
use either.

**The pre-install has to install busser itself and run the postinstall.** The
verifier's check is `grep "^busser"` with no anchor at the end, so a plugin
named `busser-*` satisfies it and busser would never be installed. And skipping
`busser plugin install` also skips the plugin's postinstall, which is where a
plugin installs the framework it drives.

Verified both directions locally: a passing suite exits 0, and a deliberately
failing test makes `kitchen verify` exit 20.

The suite includes a `helper.sh`, which is not named `*_test.sh` or `*_spec.sh`
and so must never run. If the glob ever widens, it fails the suite loudly.

Signed-off-by: Tim Smith <tsmith84@proton.me>
The runners require "bundler/setup", which walks up from the suite looking for
a Gemfile. With the Busser root under ${{ github.workspace }} it found this
project's Gemfile and tried to materialize that bundle inside the isolated gem
home, which cannot work:

```text
Could not find cookstyle-9.0.0, aruba-2.4.1, cucumber-11.1.1, ...
  in locally installed gems (Bundler::GemNotFound)
```

A real Busser root is /opt/busser -- never inside the project -- so this was an
artefact of the CI layout rather than anything the plugins do. Moving it to
/tmp/busser-kitchen matches reality and fixes it.

It only showed up on CI because a root outside any Gemfile tree, which is what I
had locally, makes bundler/setup a no-op.

Signed-off-by: Tim Smith <tsmith84@proton.me>
@tas50
tas50 merged commit c70cd11 into main Aug 29, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant