| description | Run your tests directly with BoxLang in the CLI. | ||
|---|---|---|---|
| icon | boxing-glove | ||
| metaLinks |
|
TestBox ships with a BoxLang CLI runner for Linux/Mac and Windows. Run your entire test suite — or individual bundles — directly from the terminal with rich, colored output, real-time streaming, dry runs, and detailed performance analysis.
{% hint style="danger" %} The BoxLang CLI runner is a BoxLang-only feature. It runs tests in the BoxLang OS runtime, not inside a web server. {% endhint %}
BoxLang allows you to build web applications, CLI, serverless, Android, and more. You can use this runner to test each environment. However, please note that if you will be testing web apps from the CLI, you will need to install the web support module into the OS runtime.
If you want to test your web application from the CLI without a running web server, install the bx-web-support module into the BoxLang OS runtime. This adds web BIFs, components, and a mock HTTP server so you can do full life-cycle testing purely from the CLI.
install-bx-module bx-web-support{% hint style="success" %} If you are building exclusively a web application, we suggest you use the CommandBox runner which calls your runner via HTTP, or the TestBox RUN IDE for a browser-based experience. {% endhint %}
The BoxLang runner automatically attempts to load your Application.bx mappings from the project root before running any specs. This means custom path mappings, datasources, and settings defined in Application.bx are available to your tests without any extra configuration — bringing the CLI experience much closer to a full web server environment.
The runner scripts live in testbox/ inside your TestBox installation:
run— Linux / macOSrun.bat— Windows
The runner must be executed from the root of your BoxLang project.
./testbox/run
./testbox/run my.bundle
./testbox/run --directory=tests.specs
./testbox/run --bundles=my.bundle./testbox/run.bat
./testbox/run.bat my.bundle
./testbox/run.bat --directory=tests.specs
./testbox/run.bat --bundles=my.bundle| Option | Description | Default |
|---|---|---|
--bundles |
Dot-notation list of test bundles to run. Mutually exclusive with --directory. |
* |
--bundles-pattern |
Glob pattern for discovering bundles. | *Spec*.cfc|*Test*.cfc|*Spec*.bx|*Test*.bx |
--directory |
Dot-notation directory to scan for tests. Mutually exclusive with --bundles. |
tests.specs |
--recurse |
Recurse into sub-directories. | true |
--eager-failure |
Stop the run on the first failure. | false |
--verbose |
Show all spec results as they run. | false |
The --stream flag activates real-time output, pushing each spec result to the terminal as it completes rather than buffering until the suite finishes.
./testbox/run --stream
./testbox/run --directory=tests.specs --stream
{% hint style="info" %} Streaming is especially useful in CI environments where you need live progress rather than waiting for the full suite to finish before seeing any output. {% endhint %}
The --dry-run flag discovers and reports every spec that would be executed — without actually running any spec closures. Use it to audit coverage, validate label filters, and build test inventories before committing to a full run.
# Formatted text output
./testbox/run --dry-run
# Raw JSON output — ideal for CI tools and external scripts
./testbox/run --dry-run=json
./testbox/run --dry-run=json --bundles=tests.specs.MySpec | jq .The output lists every suite and spec that would be executed, along with their labels and any skip reasons.
{% hint style="success" %}
Dry run respects all the same filtering options as a normal run: --labels, --bundles, --directory, --filter-suites, --filter-specs, etc.
{% endhint %}
For programmatic dry runs from BoxLang code, see Dry Run & Spec Discovery.
Focus your terminal output exclusively on failures and errors, hiding all passing and skipped specs:
./testbox/run --show-failed-onlyFine-grained control over which result types appear in the output:
./testbox/run --show-passed=false # hide passing specs
./testbox/run --show-skipped=false # hide skipped specsControl how much stack trace detail is shown for failures and exceptions:
./testbox/run --stacktrace=short # default — condensed first frame only
./testbox/run --stacktrace=full # complete Java/BoxLang stack traceAbort the run after a set number of failures — useful for keeping CI output manageable:
./testbox/run --max-failures=10Identify slow specs and surface them at the end of the run:
# Flag any spec taking longer than 500 ms as slow
./testbox/run --slow-threshold-ms=500
# Print a summary of the 5 slowest specs after the run
./testbox/run --top-slowest=5./testbox/run --show-failed-only --stacktrace=short --max-failures=5 --top-slowest=3| Option | Description | Default |
|---|---|---|
--reporter |
The reporter to use. Use agent for compact JSON built for AI agents, see Reporters. |
text |
--reportpath |
Directory for report files. | /tests/results |
--properties-summary |
Generate a .properties summary file. |
true |
--properties-filename |
Name for the properties file. | TEST.properties |
--write-report |
Write the report to a file. | true |
--write-json-report |
Write a JSON report alongside the main report. | false |
--write-visualizer |
Write the test visualizer to the report directory. | false |
The BoxLang runner starting and stopping your web server
These options shine for browser tests, but --retries and --failed work for every bundle, .bx or .cfc.
| Option | Description | Default |
|---|---|---|
--retries |
How many extra times to run a failing or erroring spec. A spec retries argument and a bundle retries annotation take precedence. Skipped specs are never retried. |
0 |
--failed |
Run only the bundles and specs that failed or errored in the last run, read from {reportpath}/.testbox-failed.json. Every run writes that file. When it is missing or lists nothing, a message is printed and nothing runs. |
false |
--web-server |
A shell command that starts a web server before the tests run, stopped with its child processes after the tests. Runs with sh -c (cmd /c on Windows) from the current directory. |
|
--web-server-url |
The URL polled until the web server answers with a status below 500. Also the default baseURL of browser bundles. |
http://localhost:8080 |
--web-server-timeout |
How long to wait for the web server to answer, in seconds. When it does not answer in time, the runner stops it, prints the reason and exits with code 1. |
60 |
# Start the app, run the browser specs, retry flaky ones once
./testbox/run --directory=tests.browser --web-server="boxlang-miniserver --port 8080" --retries=1
# Rerun only what failed last time
./testbox/run --failed{% hint style="info" %}
Quote a --web-server command that contains spaces. When a launcher splits it at its spaces anyway, the runner joins the words after --web-server= back together until the next runner option, so a command whose own flags share a runner option name (such as --directory) belongs in a script. See Running Browser Tests for the details and a CI example.
{% endhint %}
Pass arbitrary options directly through to the selected runner:
--runner-{option}=value| Option | Description |
|---|---|
--labels |
Only run specs/suites with these labels. |
--excludes |
Skip specs/suites with these labels. |
--filter-bundles |
Limit execution to these bundle names. |
--filter-suites |
Limit execution to these suite names. Direct name matching is supported at any nesting depth. |
--filter-specs |
Limit execution to these spec names. |
{% hint style="info" %}
Suite filtering (--filter-suites) performs direct name matching at any nesting depth — if a suite's name exactly matches the filter value it will always be included regardless of how deeply it is nested.
{% endhint %}