Skip to content

Keep both reports when a value quoted in a fault is cut inside an emoji - #427

Merged
ssegura merged 1 commit into
masterfrom
fix/half-a-character-in-a-report
Oct 29, 2026
Merged

ssegura merged 1 commit into
masterfrom
fix/half-a-character-in-a-report

Conversation

@ssegura

@ssegura ssegura commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Related: None. Found in a code-quality review of the pull requests merged since #403.

What you can do now that you could not before

Get your report when RESTest tells you about a value with an emoji in it. A fault quotes the value RESTest sent, cut at 40 characters; when the cut fell inside an emoji, half of it was left in the text, and writing report.json failed on it. Both report.json and report.html were lost, and the run ended with 4, as if RESTest itself had broken. Now the quote is cut between whole characters, and the reports are written whatever text they hold.

Try it yourself

With only Java and git. The "API" is a stand-in of a few lines on the small HTTP server that comes with Java, answering 200 to everything. The document gives one query parameter a single accepted value, 38 letters and an emoji, so the first 40 pieces of the quote of anything RESTest sends for it, a quotation mark and 38 letters before the emoji, end in the middle of the emoji (see "How it works"). The run is stopped after 60 requests.

git fetch origin && git switch fix/half-a-character-in-a-report
./mvnw -q install -DskipTests
mkdir -p try
cat > try/Stand.java <<'EOF'
import com.sun.net.httpserver.HttpServer;
import java.net.InetSocketAddress;

/** A stand-in API that answers every request with the status code it is given, and no body. */
public class Stand {
    public static void main(String[] args) throws Exception {
        int status = Integer.parseInt(args[0]);
        HttpServer server = HttpServer.create(new InetSocketAddress("127.0.0.1", 8799), 0);
        server.createContext("/", exchange -> {
            exchange.getRequestBody().readAllBytes();
            exchange.sendResponseHeaders(status, -1);
            exchange.close();
        });
        server.start();
    }
}
EOF
cat > try/moods.yaml <<'EOF'
openapi: 3.0.3
info:
  title: Moods
  version: "1.0"
paths:
  /moods:
    get:
      operationId: findMoods
      parameters:
        - name: mood
          in: query
          required: true
          schema:
            type: string
            enum: ["xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx😀"]
      responses:
        "200":
          description: the moods
        "400":
          description: a mood it does not know
EOF
java try/Stand.java 200 &
./restest run try/moods.yaml --url http://127.0.0.1:8799 --budget 20s --seed 1 \
    --set schedule.stopAfterRequests=true --set schedule.requestsBeforeStopping=60
echo "exit $?"
kill %1

Real output, cut where marked. The stand-in accepts every value RESTest sends on purpose to be refused, so the run finds 15 errors and ends with 1; what matters is the quote, which keeps its emoji, and the two lines saying the reports were written:

RESTest testing Moods at http://127.0.0.1:8799

1 of 1 operations can be tested, seed 1, budget 20s
  what it sends depends on the API's own replies, so the seed alone does not repeat this run; --store keeps what it sent
  2 setting(s) are not what RESTest does by default; --print-settings lists them with where each came from
[...]
error    Invalid data accepted  (F206)
         findMoods - GET http://127.0.0.1:8799/moods?mood=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx%F0%9F%98%80X  ->  200
         the API accepted a request it should have refused: sent "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx😀... for query parameter 'mood', which is not on its list of accepted values
         curl -i -X GET 'http://127.0.0.1:8799/moods?mood=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx%F0%9F%98%80X' -H 'Accept: */*' -H 'User-Agent: RESTest/2.0'

error    Invalid data accepted  (F206)
         findMoods - GET http://127.0.0.1:8799/moods?mood=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX%F0%9F%98%80  ->  200
         the API accepted a request it should have refused: sent "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX😀... for query parameter 'mood', which is not on its list of accepted values
         curl -i -X GET 'http://127.0.0.1:8799/moods?mood=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX%F0%9F%98%80' -H 'Accept: */*' -H 'User-Agent: RESTest/2.0'
[...]
60 requests to 1 operations in 338ms, 88% of it idle
  opening lap: 1 request in 44ms, 1 of 1 operations answered 2xx
  60 2xx
  5 of them were pushing at the API with values nobody sensible would send
  15 of them changed one thing in a request the API had accepted
15 errors, on 1 operation:
  15 x Invalid data accepted  (F206)
        make the API refuse what its document forbids; where the document forbids more than the API should, loosen the document instead
report written to restest-out/report.json (18.7 KiB)
page written to restest-out/report.html (99.8 KiB)
the run itself was not kept; pass --store to keep every request and reply
exit 1

The same commands on master (git switch --detach origin/master && ./mvnw -q install -DskipTests, then the same run). The screen shows a ? where the half emoji is, the reports are not written, and the run ends with 4:

error    Invalid data accepted  (F206)
         findMoods - GET http://127.0.0.1:8799/moods?mood=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX%F0%9F%98%80  ->  200
         the API accepted a request it should have refused: sent "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX?... for query parameter 'mood', which is not on its list of accepted values
         curl -i -X GET 'http://127.0.0.1:8799/moods?mood=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX%F0%9F%98%80' -H 'Accept: */*' -H 'User-Agent: RESTest/2.0'
[...]
60 requests to 1 operations in 341ms, 88% of it idle
  opening lap: 1 request in 44ms, 1 of 1 operations answered 2xx
  60 2xx
  5 of them were pushing at the API with values nobody sensible would send
  15 of them changed one thing in a request the API had accepted
15 errors, on 1 operation:
  15 x Invalid data accepted  (F206)
        make the API refuse what its document forbids; where the document forbids more than the API should, loosen the document instead
restest: 1 listener(s) failed and 0 event(s) never arrived, so what is printed above may be incomplete and the files may not have been written
exit 4

How it works

Java keeps text in 16-bit pieces. Most characters take one piece, but an emoji, like many characters outside the most common ones, takes two, and only the pair means anything. The quote counted pieces, so a cut after 40 of them could keep the first half of an emoji and drop the second. A half character cannot be written in UTF-8, the encoding every report file is in, and the writer the report files went through refused the whole file rather than that one character.

The quote now counts whole characters, so it never cuts inside one; that is how RESTest already cut everywhere else it shortens text. And the report files are now written so that half a character, should one come from anywhere else (an API's reply can carry one, written as an escape in its JSON), becomes a ? instead of costing the report. The screen already did that, which is why the run on master printed its faults and then lost its files.

What changed in the code

  • restest-gen: Mutations.quoted cuts at 40 code points (codePointCount, offsetByCodePoints) rather than 40 chars.
  • restest-report: WholeFile.write writes through an OutputStreamWriter whose UTF-8 encoder replaces malformed input, instead of Files.newBufferedWriter, whose encoder reports it. WholeFile is the only place production code writes a text file, so this covers report.json and report.html.
  • Tests: MutationsTest.quoted_whole_characters (a value whose quote ends inside an emoji: no half character in any description, and the whole emoji kept); HtmlReportTest.half_a_character_is_written_as_a_question_mark (a finding holding half an emoji: both files written, with ? in its place).

Evidence

  • Tests: 2 new test methods. Each was seen to fail on master: the first with half an emoji left in the description, the second with UncheckedIOException: the report could not be written to …/report.json, caused by MalformedInputException. ./mvnw verify is green on all ten modules with this change and Report undeclared status codes and content types on operations whose replies RESTest read whole #428 together, 4,945 tests; CI runs this branch alone.
  • Architecture tests: none added or weakened.
  • Mutation score: not applicable; neither oracles nor constraints were touched.
  • Per-request overhead: not measured. The quote is built only when a change is made on purpose, and counting code points in 40 characters costs nothing worth measuring; the files are written once, when the run ends.
  • Smoke run, two containerised APIs: not run here (it needs Docker); CI runs it. Neither API's document holds a value like this one.
  • Idle time: unchanged; 88% in the 60-request runs above, both builds.
  • Artefact: the two runs above.

Decisions taken

  • Whole code points, not whole graphemes. A flag or an emoji joined from several (a family, a skin tone) can still be cut between the code points it is made of. That looks odd in a quote but is valid text, and every file holds it. Cutting between graphemes would need a BreakIterator for a 40-character quote.
  • A ?, rather than an escape that keeps the half character. JSON can write half an emoji as \ud83d, which would keep the value exactly in report.json. But the JSON library writes such text through to the file unescaped, so keeping it would mean escaping every emoji in every report, and the page and any other text RESTest writes would still need the replacement. Replacing at the one place files are written covers every source of half characters, not only this one.
  • Both fixes in one pull request. Either alone closes the case seen: the first stops RESTest from making half characters, the second stops one from costing a report.

Open questions

None.


  • Targets master
  • English throughout, including comments and test names
  • The hard rules in AGENTS.md hold
  • Nothing from "Out of scope" in docs/DESIGN.md without explicit approval

🤖 Generated with Claude Code

A change to an accepted request quotes the value it sent, cut at 40
characters. The cut counted UTF-16 units, so it could fall between the
two halves of an emoji and leave half a character in the description.
WholeFile then wrote report.json with a strict UTF-8 encoder, which
refused it: both report.json and report.html were lost, and the run
ended with 4.

The quote is now cut at 40 code points, as every other cut in the code
base is, and WholeFile writes with an encoder that puts a question mark
where half a character stands, whatever brought it there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ssegura added a commit that referenced this pull request Oct 29, 2026
…replies RESTest read whole (#428)

**Related:** None. Found in a code-quality review of the pull requests
merged since #403; #406 introduced the rule this corrects.

## What you can do now that you could not before

Have a status code or a content type the document does not declare
reported (`F920`, `F921`) on an operation with a shape RESTest does not
fully understand — a `discriminator`, a schema the document points at
and never defines — in a parameter, a request body or a reply. Such an
operation used to count as "read in part", and both checks stood aside
from every reply to it, though every status code and content type it
declares had been read. Across the Web Fuzzing Dataset's 52 documents
and the 46 of RESTest's corpus, that kept 378 of 3,627 operations away
from both checks; 4 are now, the ones with a reply RESTest really could
not read. On ohsome-api it was 126 of 134 operations, on jasper 59 of
61.

## Try it yourself

With only Java and git. The "API" is a stand-in of a few lines on the
small HTTP server that comes with Java, answering `404` to everything.
The document builds a query parameter of one operation, and the body of
the only reply of another, from a schema it never defines; neither
declares `404`. The run is stopped after 60 requests.

```bash
git fetch origin && git switch fix/read-in-part-only-for-an-unread-reply
./mvnw -q install -DskipTests
mkdir -p try
cat > try/Stand.java <<'EOF'
import com.sun.net.httpserver.HttpServer;
import java.net.InetSocketAddress;

/** A stand-in API that answers every request with the status code it is given, and no body. */
public class Stand {
    public static void main(String[] args) throws Exception {
        int status = Integer.parseInt(args[0]);
        HttpServer server = HttpServer.create(new InetSocketAddress("127.0.0.1", 8799), 0);
        server.createContext("/", exchange -> {
            exchange.getRequestBody().readAllBytes();
            exchange.sendResponseHeaders(status, -1);
            exchange.close();
        });
        server.start();
    }
}
EOF
cat > try/moths.yaml <<'EOF'
openapi: 3.0.3
info:
  title: Night animals
  version: "1.0"
paths:
  /moths:
    get:
      operationId: listMoths
      parameters:
        - name: kind
          in: query
          schema:
            $ref: '#/components/schemas/NoSuchShape'
      responses:
        "200":
          description: every moth
  /larks:
    get:
      operationId: listLarks
      responses:
        "200":
          description: every lark
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/NoSuchShape'
components:
  schemas: {}
EOF
java try/Stand.java 404 &
./restest run try/moths.yaml --url http://127.0.0.1:8799 --budget 20s --seed 1 \
    --set schedule.stopAfterRequests=true --set schedule.requestsBeforeStopping=60
echo "exit $?"
kill %1
```

Real output, cut where marked. Both operations are reported for the
`404` they do not declare, as warnings, so the run ends with `0`:

```
RESTest testing Night animals at http://127.0.0.1:8799

2 of 2 operations can be tested, seed 1, budget 20s
  what it sends depends on the API's own replies, so the seed alone does not repeat this run; --store keeps what it sent
  2 setting(s) are not what RESTest does by default; --print-settings lists them with where each came from
4 part(s) of the document could not be read:
  info: attribute paths.'/moths'(get).parameters.[kind].schemas.#/components/schemas/NoSuchShape is missing
  info: attribute paths.'/larks'(get).responses.200.content.'application/json'.schema.#/components/schemas/NoSuchShape is missing
  paths./moths.get: this operation's data includes a shape RESTest does not fully understand yet
  paths./larks.get: this operation's data includes a shape RESTest does not fully understand yet

warning  Undeclared status code  (F920)
         listMoths - GET http://127.0.0.1:8799/moths  ->  404
         the API answered 404, which the document does not declare for this operation; it declares 200
         curl -i -X GET 'http://127.0.0.1:8799/moths' -H 'Accept: */*' -H 'User-Agent: RESTest/2.0'

warning  Undeclared status code  (F920)
         listLarks - GET http://127.0.0.1:8799/larks  ->  404
         the API answered 404, which the document does not declare for this operation; it declares 200
         curl -i -X GET 'http://127.0.0.1:8799/larks' -H 'Accept: application/json' -H 'User-Agent: RESTest/2.0'
[...]
60 requests to 2 operations in 306ms, 89% of it idle
  opening lap: 2 requests in 44ms, 0 of 2 operations answered 2xx
  60 4xx
  2 of them were pushing at the API with values nobody sensible would send, which accounts for some of the 60 refusals above
60 warnings, on 2 operations, which do not fail the run (checks.failOnWarnings):
  60 x Undeclared status code  (F920)
        declare the status code for the operation in the document, or stop the API answering it
report written to restest-out/report.json (23.7 KiB)
page written to restest-out/report.html (104.8 KiB)
the run itself was not kept; pass --store to keep every request and reply
exit 0
```

The same commands on `master` (`git switch --detach origin/master &&
./mvnw -q install -DskipTests`, then the same run) find nothing, and
count every reply as one the check could not judge:

```
60 requests to 2 operations in 300ms, 88% of it idle
  opening lap: 2 requests in 37ms, 0 of 2 operations answered 2xx
  60 4xx
  2 of them were pushing at the API with values nobody sensible would send, which accounts for some of the 60 refusals above
no faults found
some replies could not be checked; report.json names their operations:
  the status code of 60 replies, on 2 operations: RESTest could not read all the document says about the operation
report written to restest-out/report.json (10.3 KiB)
page written to restest-out/report.html (91.3 KiB)
the run itself was not kept; pass --store to keep every request and reply
exit 0
```

A reply RESTest really could not read still keeps both checks away from
its operation; `AReplyUnreadIsNotUndeclaredTest` runs that case end to
end, and the screen now names the reply itself:
`paths./bats.get.responses.404: the response '404' is a reference that
does not resolve…`.

## How it works

`F920` and `F921` judge a reply by what the document leaves out: a
status code it never declares for the operation, a content type it never
declares for that status code. So they must stand aside when RESTest
could not read one of the operation's replies, which might be the very
one that declares what came back. Every part of a document RESTest
cannot use is recorded with a word for what it cost, and the checks
stood aside from every operation with the word `degraded`, "tested with
less than the document holds". But most of those have every reply in
place: a shape RESTest does not fully understand, in a parameter, a
request body or the body of a reply, costs the requests RESTest builds,
or the check of a body against its shape, never the list of status codes
and content types.

When RESTest reads the document, a reply it cannot read at all — one
given as a reference to a reply the document never defines, or declared
under a status code that is not one — is now recorded at the reply
itself, `paths./pets.get.responses.404`, rather than at the operation.
The checks stand aside only from an operation with such a record.
Everything else is recorded as before.

## What changed in the code

- `restest-spec`: `OperationConverter.convertResponses` locates its two
issues about a reply that could not be read at `…responses.<status>`. A
header that could not be read, and the issue about a shape not
understood, stay at the operation, with a comment saying why.
- `restest-core`:
- `ApiModel.readInPart` counts a `DEGRADED` issue only when its location
is a reply itself: what follows the last `.responses.` is a status code,
with no dot in it.
- `SpecificationIssue`'s `location` and `degraded` say so for whoever
else writes issues.
- `WhyNotJudged.READ_IN_PART`'s words, which the screen,
`notChecked[].why` and the page show, now read "RESTest could not read a
reply the document declares for the operation".
- `restest-oracles`: `HeldToTheDocument`'s Javadoc only; neither check
changed.
- Docs: `faults.md` (`F920`, `F921`), `report.md` (the `readInPart`
reason), `known-limitations.md`. ADR-0012: an amendment of 29 October,
with the alternatives weighed. ADR-0016: a correction pointing at it,
after the one of 17 October.
- Tests:
-
`AReplyUnreadIsNotUndeclaredTest.a_shape_it_cannot_read_hides_no_reply`,
new: the document above, end to end.
- `ApiModelTest.an_operation_read_in_part`, extended: a request-side
shape and a header inside a reply do not count.
- `MalformedDocumentTest`, two tests extended: a reply under an unusable
status key is located at it and counts; a header given as a reference to
nothing does not.
- The oracles' fixtures, and the screen line in
`a_renamed_operation_is_held_to_its_document`, use the reply's location.

## Evidence

- [x] **Tests:** 1 new test method, 3 extended. The new end-to-end test
fails on `master` (no `F920` at all), and so does the extension of
`ApiModelTest` (the request-side shape counts). `./mvnw verify` is green
on all ten modules with this change and #427 together, 4,945 tests; CI
runs this branch alone.
- [x] **Architecture tests:** none added or weakened.
- [x] **Mutation score, `restest-oracles`:** not run; the only change
there is a Javadoc. CI's mutation row holds the threshold.
- [x] **Per-request overhead:** not measured. `readInPart` is asked once
per reply the two checks would otherwise report, and the new test runs
only for a `DEGRADED` issue on that operation: one search of a short
string.
- [x] **The reader, over 98 documents:** every document of the Web
Fuzzing Dataset (52) and of RESTest's corpus (46), 3,627 operations,
read by this branch, counting the operations read in part by the old
rule (any `DEGRADED` issue) and the new. Old: 378, in 12 documents, 126
of them on ohsome-api, 114 on the corpus's ohsome aggregation document,
59 on jasper, 45 on proxyprint. New: 4, all on catwatch, whose document
declares a reply under the status code `0`.
- [x] **Smoke run, two containerised APIs:** not run here (it needs
Docker); CI runs it. The activity catalogue's document, one of the two,
is among the 98 above and has no operation read in part by either rule.
- [x] **Idle time:** 88–89% in the 60-request runs above, both builds;
nothing in the request loop changed.
- [x] **Artefact:** the two runs above. No campaign was run: how many
more `F920` and `F921` a run of ohsome-api or jasper reports now is for
the next release evaluation.

## Decisions taken

- **Where the issue is, rather than a new word for what it cost.**
ADR-0012's amendment weighs a sixth effect (it would change the word
`report.json` has written since 2.2.0 for the very issues the two checks
were built around) and a field on `SpecificationIssue` (a new
constructor for a record `restest-core` publishes, which the amendment
of 17 October declined). The location already says where an issue is.
- **Only a whole reply counts, not whatever is inside one.** A header
that could not be read, or a shape inside a reply's body, hides no
status code and no content type. The rule is exact — a location ending
at a status code — so that locating such an issue more precisely later
does not widen it again.
- **What the user sees changes in three ways**, all of them corrections:
- `documentIssues[].location` of a reply that could not be read now ends
in `.responses.<status>`; `operation` is unchanged.
- Operations that were left alone now get `F920` and `F921`; with
`checks.failOnWarnings` on, such a run can end with `1` where it ended
with `0`.
  - The words for `readInPart` change.

No switch was added: the rule is the existing checks applied as
documented, and `checks.undeclaredStatusCode` and
`checks.undeclaredContentType` still turn them off.
- **The name `readInPart` is kept**, in the model and as the reason's
word in `report.json`, with its meaning narrowed in its Javadoc and in
the docs.
- **From the review of this change**, acted on: the reason's words and
two Javadocs still gave the old meaning; the reply-location branch
behind catwatch's four operations had no test; ADR-0016 stated the old
rule; the check counted anything inside a reply, wider than the
decision.

## Open questions

None.

---

- [x] Targets `master`
- [x] English throughout, including comments and test names
- [x] The hard rules in `AGENTS.md` hold
- [x] Nothing from "Out of scope" in `docs/DESIGN.md` without explicit
approval

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@ssegura
ssegura merged commit 8766111 into master Oct 29, 2026
11 checks passed
@ssegura
ssegura deleted the fix/half-a-character-in-a-report branch October 29, 2026 18:09
ssegura added a commit that referenced this pull request Oct 30, 2026
…asks, keep to a rate, and Maven Central (#431)

**Related:** None. This is the release of 2.4.0, with what `master`
holds on 30 October: #422 to #429.

## What you can do now that you could not before

- **Get 2.4.0.** Once this is merged and tagged, it installs four ways,
as 2.3.0 does:
  - `brew install isa-group/tap/restest`;
  - `docker run ghcr.io/isa-group/restest:2.4.0`;
  - the release's archives, with Java alone;
  - or built from the source.
- **What 2.4.0 adds:**
- a run whose opening lap the API answers with `401` throughout stops
there and ends with `5` (#422);
- an operation answered `429`, or `503` with `Retry-After`, is left
alone for as long as the API asks (#424);
- `--set engine.mostRequestsPerMinute=<n>` keeps a run to a rate, evenly
spaced (#425);
- the eight modules go to Maven Central as
`es.us.isa.restest:restest-*`, from this release on (#429);
- fixes: both reports kept when a quoted value is cut inside an emoji
(#427), `F920` and `F921` reported on operations RESTest read whole
(#428), and every Jackson library on a version with no known flaw
(#423).
- **What stays the same as in 2.3.0:** the command line, plans and
dictionaries. A run can end with one exit code more, `5`, and
`report.json` only gains members.

## Try it yourself

From a clean clone, with only Java and git:

```bash
git clone --quiet --branch release/2.4.0 https://github.com/isa-group/RESTest.git restest-2.4.0 && cd restest-2.4.0
./mvnw -q install -DskipTests
./restest version
./restest run restest-cli/src/test/resources/pet-shelter.yaml --url http://127.0.0.1:1 --budget 3s; echo "exit $?"
ls restest-out
grep -o '"verdict" *: *"[A-Za-z]*"' restest-out/report.json
dist/assemble.sh ../release --archives > /dev/null && ls ../release
tar -xzf ../release/restest-2.4.0.tgz -C ../release && ../release/restest-2.4.0/bin/restest version
```

Below is the real output, from a fresh clone of this branch on 30
October; the Java line will be your own. Nothing listens on port 1, so
the run ends with `3`, and it still writes both files:

```
RESTest 2.4.0
Java 25.0.4.1 (Eclipse Adoptium), Mac OS X 27.0.1 aarch64
RESTest testing Pet Shelter at http://127.0.0.1:1

4 of 4 operations can be tested, seed -4339058821541680037, budget 3s
  what it sends depends on the API's own replies, so the seed alone does not repeat this run; --store keeps what it sent

64 requests to 4 operations in 363ms, 90% of it idle
  opening lap: 4 requests in 46ms, 0 of 4 operations answered 2xx
  64 no reply
  1 of them were pushing at the API with values nobody sensible would send
  3 creation(s) meant to begin a series about a thing the run created were not accepted, so no series began
no faults found
report written to restest-out/report.json (9.6 KiB)
page written to restest-out/report.html (90.7 KiB)
the run itself was not kept; pass --store to keep every request and reply
restest: nothing at http://127.0.0.1:1 answered any of the 64 requests, so the API was never actually tested. Check the address, and that it is running.
exit 3
report.html
report.json
"verdict":"nothingAnswered"
restest-2.4.0
restest-2.4.0.tgz
restest-2.4.0.zip
RESTest 2.4.0
Java 25.0.4.1 (Eclipse Adoptium), Mac OS X 27.0.1 aarch64
```

With Docker as well, the image is built from the same clone, as the
release builds it:

```bash
docker build --quiet --tag restest:2.4.0 . > /dev/null && docker run --rm restest:2.4.0 version
```

```
RESTest 2.4.0
Java 21.0.13 (Eclipse Adoptium), Linux 7.0.14-linuxkit aarch64
```

What the tag will leave on the Central Portal, laid out from the same
clone, now with no `-SNAPSHOT` in any name:

```bash
./mvnw -q package -Pcentral -DskipTests -pl restest-cli -am && dist/central.sh target/central
ls target/central/maven-central/es/us/isa/restest
ls target/central/maven-central/es/us/isa/restest/restest-core/2.4.0
grep -c "<dependency>" target/central/maven-central/es/us/isa/restest/restest-cli/2.4.0/restest-cli-2.4.0.pom
ls restest-cli/target/lib | wc -l
```

```
/…/restest-2.4.0/target/central/maven-central
restest-cli
restest-core
restest-exec
restest-gen
restest-oracles
restest-report
restest-spec
restest-store
restest-core-2.4.0-javadoc.jar
restest-core-2.4.0-javadoc.jar.md5
restest-core-2.4.0-javadoc.jar.sha1
restest-core-2.4.0-sources.jar
restest-core-2.4.0-sources.jar.md5
restest-core-2.4.0-sources.jar.sha1
restest-core-2.4.0.jar
restest-core-2.4.0.jar.md5
restest-core-2.4.0.jar.sha1
restest-core-2.4.0.pom
restest-core-2.4.0.pom.md5
restest-core-2.4.0.pom.sha1
70
      70
```

## How it works

**The version.** `-SNAPSHOT` is how Maven writes "on the way to this
version", and this pull request removes it from every `pom.xml`. After
the merge, the tag `v2.4.0` on the merge commit starts the release
workflow ([`docs/ci.md`](docs/ci.md#releases)):
- it builds the image and the archives, and checks them on three
operating systems and two processors;
- it installs the Homebrew formula for the archive on macOS and Linux;
- it signs the eight modules and uploads them to the Central Portal,
once a maintainer approves the `central` job; the Portal validates them
and publishes nothing;
- it pushes the image, and leaves a draft release for a maintainer to
write the notes in and publish.

A maintainer publishes the deployment on the Portal before publishing
the release, and publishing the release starts the Homebrew workflow,
which writes 2.4.0 into the tap.

**The base images.** Step 1 of every release moves the `Dockerfile`'s
two base images to the newest patch published today. This time the Java
in the image moves too, from 21.0.12.1 to 21.0.13, as the output above
shows, and the pages that print it say so.

**The pages.** The README and chapters 2 and 9 of the manual name the
version to install, and now name 2.4.0. Every other page was written for
2.4.0 as each change landed, from #422 to #429.

## What changed in the code

No Java code changed, and nothing under `src/`.

- **Version:** every `pom.xml`, from `2.4.0-SNAPSHOT` to `2.4.0`, ten
files. The JSON Schemas' `$id` already names `v2.4.0` since #421.
- **`Dockerfile`:** the digests of `eclipse-temurin:25-jdk` and
`eclipse-temurin:21-jre` as they are today.
- **`README.md`:** the installation section fetches, clones and runs
`2.4.0`. Its `restest version` output is the image's above.
- **`docs/manual/02-installing.md`:** the same for Docker, the archive,
the checksums, the source and Updating. The Docker and archive outputs
are those above. The two lines under Homebrew's `restest version` are
what this pull request's run of `release.yml` printed in
`check-homebrew` on macOS, as `docs/ci.md` asks:

  ```
  RESTest 2.4.0
  Java 21.0.12.1 (Homebrew), Mac OS X 26.6.2 aarch64
  ```

They are the lines chapter 2 already had once `2.3.0` became `2.4.0`,
since the formula still installs Homebrew's `openjdk@21` 21.0.12.1 on
macOS; only the version changed.
- **`docs/manual/09-exit-codes-and-ci.md`:** the workflow fetches and
runs the 2.4.0 archive. Its sentence on releases before 2.3.0, which
write no page, stays as it is.
- **`docs/DESIGN.md`:** the stack table named `jackson-core` 2.22.1,
which #423 moved to 2.22.3 in the root POM; it now says 2.22.3.

## Evidence

- [x] **Tests:** `./mvnw verify` is green at `d19edf0f`, with version
2.4.0: 4,945 tests, 0 failures, BUILD SUCCESS in 2 min 33 s. 141 of them
are the architecture and documentation rules, `PublishedSchemasTest`
among them, which on a release requires the schemas' `$id` to name it.
CI on `5baafe06`:
[CI](https://github.com/isa-group/RESTest/actions/runs/41073186888) and
[Release](https://github.com/isa-group/RESTest/actions/runs/41073186769),
19 checks green, and `central` and `publish` skipped, as on every pull
request. In that Release run, `check-homebrew` installed the formula for
this branch's archive on macOS and on Ubuntu: 625 MB and 823 MB, as the
pages say. *Java 25 on windows-latest* failed on its first attempt in
`RunLoopTest.a_slow_401_to_the_lap_stops_the_run_when_it_arrives`, which
took 4.16 s where it allows less than 4 s; the same row passed when
re-run, and that test passed on every row of `master`'s last runs.
Nothing under `src/` changed here.
- [x] **Architecture tests:** unchanged, none weakened.
- [x] **Mutation score:** no oracle or constraint touched.
- [x] **Per-request overhead:** unchanged; nothing under `src/` changed.
- [x] **Smoke run:** unchanged, for the same reason; CI runs it on this
pull request.
- [x] **Idle time:** unchanged.
- [x] **Artefact:** `check-restest.sh 2.4.0` passes on both the archive
and the image built here, every check `ok`.
  - **The archive**, `--launcher` on `restest-2.4.0/bin/restest`:
    - version;
    - `JAVA_TOOL_OPTIONS`;
    - an unknown option answering `2`;
    - a run with no answer ending with `3` and its files;
    - the schemas and `THIRD-PARTY.txt` in the folder;
    - `127` with no Java and with a `JAVA_HOME` that holds none;
    - a symbolic link.
- **The image**, `--image restest:2.4.0`: the same, and `docker stop`
ending a run with `143` and a report that says it was cut short.
- **What Maven Central gets:** `dist/central.sh` passes on the build
above, with every check it makes, fetching each module offline with
Maven among them; `restest-cli`'s POM names the 70 libraries
`restest-cli/target/lib` holds, which the archive's `lib/` carries
beside `restest-cli` itself.

## Decisions taken

- **No release evaluation on the Web Fuzzing Dataset, as for 2.3.0.**
The three safeguards change a run only against an API that answers `401`
to the whole opening lap (#422), that asks to wait (#424), or when a
rate is named (#425, off otherwise); #428 was measured across the
dataset's 52 documents in its own pull request; #427 and #423 change no
request.
- **The base images are today's patch,** as step 1 of a release asks
while Dependabot is off. The image's Java moves from 21.0.12.1 to
21.0.13, so the `restest version` lines of the README and chapter 2 that
come from the image move with it.
- **Chapter 2's Homebrew lines come from this pull request's CI,** not
from a Mac here. 2.4.0 cannot be installed from the tap before it is
published, and `docs/ci.md` names `check-homebrew`'s output as their
source. They needed no commit of their own: what the job printed is what
the page already said.
- **`docs/DESIGN.md`'s `jackson-core` version is corrected here,** being
one line in the release's own stack table; the rest of the table agrees
with the root POM, the wrapper and the `Dockerfile`.
- **The release notes are not in this pull request.** They go into the
draft release after the tag, as `docs/ci.md` says.

## Open questions

- **A new Dependabot alert,
[48](https://github.com/isa-group/RESTest/security/dependabot/48), high,
opened today:** Guava up to 33.7.1 allocates memory without bound when
it deserialises some of its collections, fixed in 33.7.2. The release
carries `guava-33.5.0-jre.jar`, which the OpenAPI parser and the HTTP
client bring and the root POM pins. RESTest's own code uses neither
Guava nor Java serialisation. Whether 2.4.0 waits for a pull request
that moves Guava, as #423 did for Jackson, is the maintainer's call;
this one does not touch it.
- **After the merge, each step taken by a maintainer:**
- the tag `v2.4.0` on the merge commit, and the approval of the
`central` job when the run asks for it;
  - the notes in the draft;
- the deployment `RESTest 2.4.0 run <run>` published on the Central
Portal, and then the release, which also starts the Homebrew workflow
for 2.4.0;
- the README's installation commands tried from outside, `brew install
isa-group/tap/restest` among them, and
`https://repo1.maven.org/maven2/es/us/isa/restest/restest-cli/2.4.0/`
answering;
- the pull request that moves `master` to `2.5.0-SNAPSHOT`, and the
schemas' `$id` to `v2.5.0`.

---

- [x] Targets `master`
- [x] English throughout, including comments and test names
- [x] The hard rules in `AGENTS.md` hold
- [x] Nothing from "Out of scope" in `docs/DESIGN.md` without explicit
approval

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant