Repository navigation
Ask a coding agent to follow the page for coding agents on the documentation site, in the same words everywhere - #441
Merged
Merged
Conversation
…ite, for the version it installs Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… coding agents, which the site shows too Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ompt # Conflicts: # docs/ci.md
10 tasks done
ssegura
added a commit
that referenced
this pull request
Nov 10, 2026
…n at one address (#442) **Related:** None. This is the release of 2.5.0, with what `master` holds on 10 November: #432 to #441. **This pull request is merged before the tag `v2.5.0`, which goes on its merge commit.** ## What you can do now that you could not before - **Get 2.5.0.** Once this is merged and tagged, it installs five ways: - `brew install isa-group/tap/restest`; - `jbang app install restest@https://isa-group.github.io/jbang-catalog/jbang-catalog.json`, new in this version, with no Java installed first; - `docker run ghcr.io/isa-group/restest:2.5.0`; - the release's archives, with Java alone; - or built from the source. - **What 2.5.0 adds:** - RESTest runs with jbang on macOS, Linux and Windows, which downloads a Java when there is none (#434); - every archive carries a skill that teaches a coding agent to use RESTest, `skills/restest/SKILL.md` (#437), and the page for coding agents gives the sentence to hand one, the same in the README (#441); - the documentation can be read and searched at one address, moved on to each release once it is published (#439); - installing takes a few lines, with the details in the manual (#435), and the README and the architecture's diagrams are simpler (#438); - the manual shows `report.html` as a real run of the clinic draws it, light and dark, with that run's `report.json` beside it (#440); - a fix: an API that answers in HTTP/1.0, as Python's `http.server` does, gets a reply read for every request (#436). - **What stays the same as in 2.4.0:** the command line, its exit codes, plans, dictionaries, settings and both reports. ## Try it yourself From a clean clone, with only Java and git: ```bash git clone --quiet --branch release/2.5.0 https://github.com/isa-group/RESTest.git restest-2.5.0 && cd restest-2.5.0 ./mvnw -q install -DskipTests ./restest version ./restest run restest-cli/src/test/resources/pet-shelter.yaml --url http://127.0.0.1:1 --budget 3s; echo "exit $?" ls restest-out grep -o '"verdict" *: *"[A-Za-z]*"' restest-out/report.json dist/assemble.sh ../release --archives > /dev/null && ls ../release tar -xzf ../release/restest-2.5.0.tgz -C ../release && ../release/restest-2.5.0/bin/restest version ls ../release/restest-2.5.0/skills/restest ``` Below is the real output, from a fresh clone of this branch on 10 November; the Java line will be your own. Nothing listens on port 1, so the run ends with `3`, and it still writes both files; the archive carries the skill: ``` RESTest 2.5.0 Java 25.0.4.1 (Eclipse Adoptium), Mac OS X 27.0.1 aarch64 RESTest testing Pet Shelter at http://127.0.0.1:1 4 of 4 operations can be tested, seed 2125531189854063678, budget 3s what it sends depends on the API's own replies, so the seed alone does not repeat this run; --store keeps what it sent 64 requests to 4 operations in 361ms, 90% of it idle opening lap: 4 requests in 46ms, 0 of 4 operations answered 2xx 64 no reply 1 of them were pushing at the API with values nobody sensible would send 3 creation(s) meant to begin a series about a thing the run created were not accepted, so no series began no faults found report written to restest-out/report.json (9.6 KiB) page written to restest-out/report.html (90.7 KiB) the run itself was not kept; pass --store to keep every request and reply restest: nothing at http://127.0.0.1:1 answered any of the 64 requests, so the API was never actually tested. Check the address, and that it is running. exit 3 report.html report.json "verdict":"nothingAnswered" restest-2.5.0 restest-2.5.0.tgz restest-2.5.0.zip RESTest 2.5.0 Java 25.0.4.1 (Eclipse Adoptium), Mac OS X 27.0.1 aarch64 SKILL.md ``` With Docker as well, the image is built from the same clone, as the release builds it: ```bash docker build --quiet --tag restest:2.5.0 . > /dev/null && docker run --rm restest:2.5.0 version ``` ``` RESTest 2.5.0 Java 21.0.13 (Eclipse Adoptium), Linux 7.0.14-linuxkit aarch64 ``` What the tag will leave on the Central Portal, laid out from the same clone: ```bash ./mvnw -q package -Pcentral -DskipTests -pl restest-cli -am && dist/central.sh target/central ls target/central/maven-central/es/us/isa/restest grep -c "<dependency>" target/central/maven-central/es/us/isa/restest/restest-cli/2.5.0/restest-cli-2.5.0.pom ls restest-cli/target/lib | wc -l ``` ``` /…/restest-2.5.0/target/central/maven-central restest-cli restest-core restest-exec restest-gen restest-oracles restest-report restest-spec restest-store 70 70 ``` ## How it works **The version.** `-SNAPSHOT` is how Maven writes "on the way to this version", and this pull request removes it from every `pom.xml`. After the merge, the tag `v2.5.0` on the merge commit starts the release workflow ([`docs/ci.md`](docs/ci.md#releases)): - it builds the image and the archives, and checks them on three operating systems and two processors; - it installs the Homebrew formula for the archive on macOS and Linux, and runs the jbang catalog for this version; - it signs the eight modules and uploads them to the Central Portal, once a maintainer approves the `central` job; the Portal validates them and publishes nothing; - it pushes the image, and leaves a draft release for a maintainer to write the notes in and publish. A maintainer publishes the deployment on the Portal before publishing the release. Publishing the release starts the Homebrew workflow, which writes 2.5.0 into the tap, the jbang workflow, which writes it into the catalog, and the documentation site's, which moves the site on to 2.5.0. Before the tag, the documentation site is built by hand from `master`, so that a tag the site cannot build is never pushed. **The base images.** Step 1 of every release moves the `Dockerfile`'s two base images to the newest patch; they are the newest already, so the `Dockerfile` does not change, nor the Java the image prints. **The pages.** The README and chapters 2 and 9 of the manual name the version to install, and now name 2.5.0. Every other page was written for 2.5.0 as each change landed, from #432 to #441. Those that give a minimum version (the skill's "2.4.0 or later"), speak of a past release, or name the version the manual's kept run was made with (chapter 4's, 2.4.0) stay as they are. ## What changed in the code No Java code changed, and nothing under `src/`. - **Version:** every `pom.xml`, from `2.5.0-SNAPSHOT` to `2.5.0`, ten files. The JSON Schemas' `$id` already names `v2.5.0` since #432. - **`README.md`:** the Docker command runs `2.5.0`. - **`docs/manual/02-installing.md`:** Homebrew's and jbang's `restest version`, Docker, the archive, the checksums, the source and Updating name 2.5.0. The Docker and archive outputs are those above, unchanged but for the version. Homebrew's and jbang's lines and sizes are what this pull request's [run of `release.yml`](https://github.com/isa-group/RESTest/actions/runs/42861198585) printed, as `docs/ci.md` asks: `check-homebrew` installed 628 MB on macOS and 824 MB on Ubuntu (625 and 823 before) and printed `Java 21.0.13.0.3 (Homebrew), Mac OS X 26.7.2 aarch64`; `check-jbang` started RESTest on Java 27.0.1.0.2, where the page said `27`, with 636 MB of Javas, where it said some 600. jbang's libraries, 37 MB, stay as they are: `check-jbang` counts the sources and the Javadoc Maven Central gets, which jbang does not download, and the published release's run of `jbang.yml` is what they are checked against. - **`docs/manual/09-exit-codes-and-ci.md`:** the workflow fetches and runs the 2.5.0 archive. ## Evidence - [x] **Tests:** `./mvnw verify` is green at `edfb1510`, with version 2.5.0: 4,981 tests, 0 failures, BUILD SUCCESS in 2 min 36 s. 162 of them are the architecture and documentation rules, `PublishedSchemasTest` among them, which on a release requires the schemas' `$id` to name it; they pass again with chapter 2's sizes. The first push's [Release](https://github.com/isa-group/RESTest/actions/runs/42861198585) run is green, `central` and `publish` skipped, as on every pull request. On the second push, [CI](https://github.com/isa-group/RESTest/actions/runs/42862469536)'s *Java 26 on ubuntu-latest* failed on its first attempt in `RunLoopTest.an_operation_told_to_wait_is_skipped`, which counted `getPet`'s wait twice where it expects once: a wait the API asks for that lasts past the end of the budget ends at the budget's end, and a request sent at that very moment is let through and refused again, beginning a second wait. The row passed when re-run, with the two rows the failure had cancelled, and every check is green. Nothing under `src/` changes here; the race is put right in a pull request of its own, after this release. - [x] **Architecture tests:** unchanged, none weakened. - [x] **Mutation score:** no oracle or constraint touched. - [x] **Per-request overhead:** unchanged; nothing under `src/` changed. - [x] **Smoke run:** unchanged, for the same reason; CI runs it on this pull request. - [x] **Idle time:** unchanged. - [x] **Artefact:** `check-restest.sh 2.5.0` passes on both the archive and the image built here, every check `ok`: `--launcher` on `restest-2.5.0/bin/restest`, and `--image restest:2.5.0`, built again without Docker's cache. `dist/central.sh` passes on the build above; `restest-cli`'s POM names the 70 libraries the archive's `lib/` carries beside `restest-cli` itself. ## Decisions taken - **The pictures of the report are not made again.** Neither `report.html` nor `FaultCategory.java` changed since v2.4.0, and #440, which added the manual's kept run, says they are made again only when the page or the kinds of fault change. Chapter 4 keeps naming 2.4.0, the version that run was made with. - **No release evaluation on the Web Fuzzing Dataset in this pull request.** 2.5.0 changes no request but HTTP/1.0 replies (#436), measured in its own pull request. The showcase campaign of 2.5.0, one minute per API on the dataset's 52 APIs, runs once the release is published. - **Guava is not moved in this release** (#430, Dependabot alert 48, raised in 2.4.0's release): the maintainer decided it waits. RESTest's own code uses neither Guava nor Java serialisation. - **The release notes are not in this pull request.** They go into the draft release after the tag, as `docs/ci.md` says. ## Open questions None. After the merge, each step taken by a maintainer: the documentation site built by hand from `master`; the tag `v2.5.0` on the merge commit, and the approval of the `central` job; the notes in the draft; the deployment published on the Central Portal, then the release; the installation commands tried from outside; and the pull request that moves `master` to `2.6.0-SNAPSHOT`. --- - [x] Targets `master` - [x] English throughout, including comments and test names - [x] The hard rules in `AGENTS.md` hold 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related: isa-group/restest-site#4, the documentation site's first page, which shows the sentence this pull request gives, read from the page for coding agents; #440, the run that page shows. This pull request is merged while 2.5.0 is prepared, before its tag: the address it gives answers only once 2.5.0 is published.
What you can do now that you could not before
Give a coding agent two sentences — "Use RESTest to test the REST APIs you develop in this project. Follow the setup and usage instructions at https://isa-group.github.io/restest-site/docs/for-coding-agents/" — and have it read, on the documentation site, the page for coding agents of the version it will install, rather than the page as
masterhas it. The README, that page and the site's first page give the same sentence, and the build fails if the README and the page ever differ.Try it yourself
With Java and git, from a clean clone:
Now have the README say something else than the page:
Undo it with
git checkout -- README.md. The address itself answers once 2.5.0 is published; today the site documents 2.4.0, which has no page for coding agents.How it works
The README, under "From a coding agent", and the page for coding agents, under "Asking your agent", each give a person a sentence to paste into their agent. It used to point at the page on GitHub as
masterhas it, which may describe options the version the agent installs does not have yet. It now asks the agent to follow the setup and usage instructions on the documentation site, which documents the latest release, the one Homebrew, jbang and the image install; the page tells the agent how to install RESTest and the skill, and ends with the skill itself.The sentence is now given in three places: the README, the page, and the documentation site's first page, which reads it from the page at the version it documents, so the two on the site cannot differ. A test holds the README to the page: the first block of code under each heading must be the same, or the build fails saying where to change it.
What changed in the code
README.mdanddocs/for-coding-agents.md— the new sentence, "two sentences" rather than "one", and "the second can go" once the skill is installed.restest-arch-tests/.../DocumentedSkillTest.java—the_readme_asks_as_the_page_does, withfirstBlockUnder(page, heading), which reads the first block of code under a heading and stops at the next one.docs/adr/0047-the-documentation-site.md§3 — the site's first page relies on that block of the page, held by the test.docs/ci.md— the table of gates says what the test now catches.Evidence
DocumentedSkillTest, now 10; it fails when the README's sentence differs from the page's, as shown above. Architecture and documentation rules: all green. — the checks of this pull requestDecisions taken
mastermay describe the next one.restest --helpstill names the page on GitHub at the running version's tag, which is right for the version a person has; ADR-0046, a record, still speaks of the one sentence it decided then.Open questions
docs/ci.md's table of gates; whichever is merged second keeps both additions.masterAGENTS.mdhold🤖 Generated with Claude Code