Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion docs/adr/0009-non-blocking-engine.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# ADR-0009: The engine never blocks on computation; idle time replaces a throughput floor

**Status:** Accepted
**Status:** Accepted, amended on 29 October 2026
**Date:** 2026-09-11

## Context
Expand Down Expand Up @@ -55,3 +55,12 @@ says nothing about real APIs and does not pretend to.
- **A fixed requests-per-minute floor.** Rejected: assumes knowledge of the API's latency.
- **Measuring wall-clock throughput only.** Confounds our speed with the API's.
- **Doing all expensive work up front, before testing.** Exactly what failed in 2026.

## Amendment (29 October 2026): idle time the API asked for

An API can ask a run to send nothing for a while — a 429, or a 503 with `Retry-After` — and a run now
obeys it ([ADR-0042](0042-wait-when-told-to-wait.md)). Time spent that way is idle time: nothing is
in flight. It is not the tool being slow, which is what idle time exists to catch, so a run says how
much of its idle time came while it waited as the API asked, in `report.json`
(`toldToWait.idle`) and in brackets after the idle share on the first line of its summary. The
tool's own idleness is the rest.
13 changes: 12 additions & 1 deletion docs/adr/0016-how-a-fault-is-presented.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# ADR-0016: A fault is identified by its catalogue number, described in our own words, and classified twice

**Status:** Accepted, amended at M1.10 and on 10, 12, 15, 17 and 22 October 2026
**Status:** Accepted, amended at M1.10 and on 10, 12, 15, 17, 22 and 29 October 2026
**Date:** 2026-09-15

## Context
Expand Down Expand Up @@ -467,3 +467,14 @@ It is what decides how a run ends, and every row of `report.json` that counts fa
([ADR-0037](0037-errors-and-warnings.md)). A reply that is both an error and a warning — a request
broken on purpose, accepted with a status code the document does not declare — is one of each, as it
was two faults before.

## Amendment (29 October 2026): a reply that asks the run to wait, now that it waits

The amendment of 15 October said a 503 that carries `Retry-After` is reported like any other, and
that until RESTest waited when told to, a 503 its own pace provoked would be reported too. It now
waits ([ADR-0042](0042-wait-when-told-to-wait.md)): the operation that drew a 429, or a 503 with
`Retry-After`, is left alone for as long as the API asks. Nothing changes in how such a reply is
reported. A 503 is still `F121`, and a 429 the document does not declare still `F920`: the API did
answer it, and the document did leave it out. There are few such replies once the run waits — one
for each wait, and those already on their way when it began — rather than one for every request the
run's pace provoked.
18 changes: 17 additions & 1 deletion docs/adr/0025-settings.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# ADR-0025: The numbers somebody decided live in one settings object, layered from four sources, and never in the plan

**Status:** Accepted, amended at M11.1, M9.1, M10.1, M10.3, M8.5, M11.2, M12.1b, M9.5, M9.6, M9.7, M10.4-M10.7 and M9.8-M9.9, at 2.1, and on 1, 12, 13, 14, 15, 17, 22, 25 and 28 October 2026
**Status:** Accepted, amended at M11.1, M9.1, M10.1, M10.3, M8.5, M11.2, M12.1b, M9.5, M9.6, M9.7, M10.4-M10.7 and M9.8-M9.9, at 2.1, and on 1, 12, 13, 14, 15, 17, 22, 25, 28 and 29 October 2026
**Date:** 2026-09-22

## Context
Expand Down Expand Up @@ -797,3 +797,19 @@ also decides when a run ends rather than what it sends
switches that are not levers, and the page of switches has a file that turns it off, since it is on
by default. The rule has no number to put beside it: the stretch it judges is the opening lap, and
401 is a fact of HTTP rather than a choice. One hundred and two settings in nine groups.

## Amendment (29 October 2026): a switch and two numbers for an API that asks the run to wait

`schedule.waitWhenToldTo`, on by default, with `schedule.firstGuessedWait` (`1s`) and
`schedule.longestGuessedWait` (`1m`) beside it ([ADR-0042](0042-wait-when-told-to-wait.md)). An
operation the API answers 429, or 503 with `Retry-After`, is left alone for as long as the API says,
or, when it does not say, for the first of the two numbers, doubled each time it is told again up to
the second; the whole API once a second operation is told while another waits. They are in
`schedule.*`, beside the stop of 28 October, because the loop that decides what to send decides the
wait; `engine.*` holds what the HTTP engine itself reads. The switch is listed among the switches that
are not levers: it is a courtesy to the API rather than a way of finding more, so it plays no part in
an ablation, and it changes what is sent only against an API that asks to wait. Since which turns it
skips depends on when the API asked, the file for getting the seed back turns it off, and a run with
it on says under its seed that the seed alone does not repeat it. A first wait of no time is refused,
since an API that does not say how long would be asked again at once, and so is a ceiling below the
first wait. One hundred and five settings in nine groups.
Loading
Loading