diff --git a/docs/adr/0012-canonical-model.md b/docs/adr/0012-canonical-model.md index 8aefc8a8..21c08828 100644 --- a/docs/adr/0012-canonical-model.md +++ b/docs/adr/0012-canonical-model.md @@ -1,6 +1,6 @@ # ADR-0012: The canonical model is immutable records and sealed types, and says so when it cannot represent something -**Status:** Accepted, amended at M1.8, M2.1a and M2.1b, and on 17 and 26 October 2026 +**Status:** Accepted, amended at M1.8, M2.1a and M2.1b, and on 17, 26 and 29 October 2026 **Date:** 2026-09-12 ## Context @@ -145,6 +145,51 @@ shape, and it is lossless in both directions: a 3.0 document's `minimum: 5` plus --- +## Amendment (29 October 2026): only a reply that could not be read makes an operation read in part + +**Date:** 2026-10-29 + +**`ApiModel.readInPart` counts a `DEGRADED` issue only when it is about one of the operation's +replies, whole, and the reader says which ones are by where it puts them: a reply it cannot read is +located at the reply itself, `paths./pets.get.responses.404`, rather than at the operation.** The +checks of a status code and of a content type the document does not declare leave alone an +operation read in part, since a reply the document declares and RESTest could not read would +otherwise look undeclared. Until now every `DEGRADED` issue counted, and most hide no reply: a parameter, a request +body or the body of a reply built from a shape RESTest does not fully understand — a +`discriminator`, a shape the document never defines — is read with less than the document holds, +but every status code and content type the operation declares is still read. Such an operation lost +both checks with nothing missing from its replies. Of the 3,627 operations in the 98 documents read +for the amendment of 26 October, 378, in 12 documents, counted as read in part; 4 do now, all in +catwatch, which declares a reply under the status code `0`. On ohsome-api 126 of 134 operations were +left alone, and on jasper 59 of 61. + +**Where the issue is, rather than what it cost**, because the two kinds cost the same: the operation +is tested with less than the document holds, which is what `DEGRADED` says. Three other answers +were weighed: + +- **A sixth effect, for a reply that could not be read.** It would say two things in one field, the + cost and the place, and it would change the word `report.json` has written since 2.2.0 for the + very issues the two checks were built around. +- **A field on the issue naming the part of the operation it is about.** The location already says + where an issue is, which is what it is for, and the field would change the constructor of a record + `restest-core` publishes, as the amendment of 17 October declined to. +- **Counting whatever is inside a reply too**: a shape not understood in its body, a header given as + a reference that leads nowhere. Neither hides a status code or a content type, which is all the + two checks look for; the check of a body against its shape has a reason of its own for a shape it + cannot use, `shapeUnusable`. + +**A location counts only when it is a reply itself**: what follows its last `.responses.` is a +status code, which holds no dot. An issue placed later inside a reply — a header's, at +`…responses.200.headers.X-Rate` — then does not count, and `SpecificationIssue` says where a reply +that could not be read goes, for whoever else writes issues. + +**What changes in a run**: those operations are held to the status codes and content types they +declare, and a reply they do not declare is reported as on any other operation; with +`checks.failOnWarnings` on, such a run can now end with `1`. The screen, `documentIssues` in +`report.json` and the page give a reply that could not be read its own location, and the words for +`readInPart` on the screen, in `notChecked[].why` and on the page say that a reply the document +declares could not be read. + ## Amendment (26 October 2026): what no request can carry is left out, and said **Date:** 2026-10-26 diff --git a/docs/adr/0016-how-a-fault-is-presented.md b/docs/adr/0016-how-a-fault-is-presented.md index 3743dc81..13cd0fb7 100644 --- a/docs/adr/0016-how-a-fault-is-presented.md +++ b/docs/adr/0016-how-a-fault-is-presented.md @@ -442,6 +442,14 @@ scout-api no renamed operation is also one RESTest read in part, so the replay's two rules judge those eight operations as they are defined to; that what could not be read is said of the right operation is shown by RESTest's tests, not by this run. +**A correction of 29 October: only a reply RESTest could not read leaves an operation alone.** That +`F920` and `F921` leave alone any operation a `DEGRADED` issue names no longer holds either. A shape +RESTest does not fully understand, in a parameter, a request body or the body of a reply it did +read, hides no status code and no content type the operation declares, and such an operation is +judged; only a reply the document declares and RESTest could not read at all still keeps it from +both rules ([ADR-0012](0012-canonical-model.md)'s amendment of 29 October, with what it changes +across 98 documents). The reason a reply gets for it, `readInPart`, now says so in its words too. + ## Amendment (22 October 2026): what a person reads first **Date:** 2026-10-22 diff --git a/docs/adr/README.md b/docs/adr/README.md index d253437d..be4a5726 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -64,7 +64,7 @@ so records written from now on cite none. | [0009](0009-non-blocking-engine.md) | The engine never blocks; idle time replaces a throughput floor | Accepted, amended twice on 29 October 2026 | | [0010](0010-idl-strategy.md) | IDL: relicensed assets, ANTLR4 parser, solver behind an interface | Accepted, not yet built (14 October 2026) | | [0011](0011-evaluation-harness.md) | Evaluation harness in `evaluation/`, outside the Maven build — amended at M1.9: in a repository of its own, so `evaluation/` was never created | Accepted, amended at M1.9 and on 9 October 2026 | -| [0012](0012-canonical-model.md) | The canonical model: immutable records, sealed types, recorded gaps | Accepted, amended at M1.8, M2.1a and M2.1b, and on 17 October 2026 | +| [0012](0012-canonical-model.md) | The canonical model: immutable records, sealed types, recorded gaps | Accepted, amended at M1.8, M2.1a and M2.1b, and on 17, 26 and 29 October 2026 | | [0013](0013-input-generation.md) | Where input values come from, and how a campaign is put together | Accepted, amended at M1.11, M2.7a, M2.4, M2.10a, M9.1, M9.3, M10.1 and M10.3, and on 10, 12, 13 and 25 October 2026 | | [0014](0014-response-conformance.md) | A reply is judged against the specification document itself | Accepted, amended at M1.8 and on 10, 17, 19 and 22 October 2026 | | [0015](0015-command-line-contract.md) | One command, a budget spent in full, and what an exit code means | Accepted, amended at M1.7, M1.8, M2.7a, M2.10a, M11.1, M9.1, M11.3, M12.1a, M12.1c and M12.1b, in #314 and #344, and on 1, 11, 12, 19, 22, 25 and 28 October 2026 | diff --git a/docs/faults.md b/docs/faults.md index 016b03a7..b7513933 100644 --- a/docs/faults.md +++ b/docs/faults.md @@ -319,16 +319,18 @@ and `default` included. counts each such reply as one this check could not judge (`noReplyDeclared`, under `notChecked` in `report.json`), unless the operation is one of those below, read in part, which comes first (`readInPart`). - - On an operation RESTest could not read in full, a status code it could not find among what it - read: a reply the document declares, and RESTest could not read — one that points at a shape the - document never defines, say — would otherwise look undeclared. The run lists what it could not - read before it starts, and counts each such reply as one this check could not judge - (`readInPart`). A status code it did read is held to as on any other operation. A reply the - library RESTest reads documents with drops on its own, such as one written as a word rather than - an object, is not known to be missing, and does look undeclared ([known - limitations](known-limitations.md#what-a-run-says)). An operation RESTest only had to call by - its method and path, because the document gives its name to an operation RESTest reads first, - is not one of these: the new name costs it nothing else. + - On an operation one of whose replies RESTest could not read, a status code it could not find + among what it read: a reply the document declares, and RESTest could not read — one given as a + reference to a reply the document never defines, say, or under a status code that is not one — + would otherwise look undeclared. The run lists what it could not read before it starts, and + counts each such reply as one this check could not judge (`readInPart`). A status code it did + read is held to as on any other operation. A reply the library RESTest reads documents with + drops on its own, such as one written as a word rather than an object, is not known to be + missing, and does look undeclared ([known limitations](known-limitations.md#what-a-run-says)). + Nothing else RESTest says it could not read hides a reply, so nothing else makes an operation one + of these: not a shape it does not fully understand, in a parameter, a request body or the body of + a reply it did read, not a header of a reply it could not read, and not a name of RESTest's own, + given because the document gives the operation's name to an operation RESTest reads first. - A reply to a request whose body RESTest sent, on purpose, under a media type the operation does not accept: a document is not expected to describe how an API refuses a body it never promised to take. @@ -369,8 +371,8 @@ warning Undeclared content type (F921) - A document that declares `*/*`, which promises no type in particular — and is what a Swagger 2.0 document that says nothing about types is read as, so such a reply counts as checked. - A server error, and a reply to a body sent under a media type the operation does not accept, as - for `F920`; and, on an operation RESTest could not read in full, a type it could not find among - what it read, counted as for `F920` (`readInPart`). + for `F920`; and, on an operation one of whose replies RESTest could not read, a type it could + not find among what it read, counted as for `F920` (`readInPart`). - **What the API wrote is quoted cut short.** The fault quotes the content type as the API sent it, cut at 100 characters, with anything that would move a terminal's cursor or change its colours, and invisible characters that would reorder or hide what is shown around them, replaced by `?`. diff --git a/docs/known-limitations.md b/docs/known-limitations.md index f61dc9f2..d0ef97f2 100644 --- a/docs/known-limitations.md +++ b/docs/known-limitations.md @@ -138,9 +138,10 @@ of its own. Whether you may test an API at all is not something any tool can che - **A reply the document declares in a form RESTest cannot read is reported as undeclared.** The library RESTest reads documents with drops a reply written as something other than an object — `"404": "not found"` — and says so before the run, without naming the operation. That operation's - 404s are then `F920`, though the document meant to declare them. On an operation RESTest knows it - read only in part, such as one with a reply pointing at a shape the document never defines, a - status code it could not find among what it read is left alone, and counted under `notChecked`. + 404s are then `F920`, though the document meant to declare them. On an operation one of whose + replies RESTest knows it could not read, such as one given as a reference to a reply the document + never defines, a status code it could not find among what it read is left alone, and counted + under `notChecked`. - **A body the document describes only in words is not checked against a type.** A reply with a body whose status code the document declares with no content type at all is left alone, whatever type it comes as, and counted under `notChecked`. A Swagger 2.0 document that names no type at all is diff --git a/docs/report.md b/docs/report.md index faeb4de0..ced35325 100644 --- a/docs/report.md +++ b/docs/report.md @@ -526,7 +526,7 @@ those, `F920` and `F921`. | `shapeUnusable` | shape | The document gives a shape RESTest cannot use: one pointing at another file, at something the document does not define, or round in a circle, one with a pattern the checker cannot read, or a document that could not be kept to check against | | `keyHidden` | shape | A key handed over with `--auth` came back in the body and was hidden there, and the body could then no longer be read as JSON, so what is wrong with it may be the hiding's doing | | `noReplyDeclared` | status code | The document declares no reply at all for the operation, so it promised none | -| `readInPart` | status code, content type | RESTest could not read all the document says about the operation — a reply pointing at a shape it never defines, say — and what came back is not among what it could read, so it may be declared in the part it could not. What it could read is held to as on any other operation | +| `readInPart` | status code, content type | RESTest could not read a reply the document declares for the operation — one given as a reference to a reply it never defines, say — and what came back is not among what it could read, so it may be declared in the reply it could not. What it could read is held to as on any other operation | | `noContentTypeDeclared` | content type | The document declares the status code with no content type, describing that reply only in words | ### `dictionaries` diff --git a/restest-cli/src/test/java/io/restest/cli/AReplyUnreadIsNotUndeclaredTest.java b/restest-cli/src/test/java/io/restest/cli/AReplyUnreadIsNotUndeclaredTest.java index bf585361..532548e9 100644 --- a/restest-cli/src/test/java/io/restest/cli/AReplyUnreadIsNotUndeclaredTest.java +++ b/restest-cli/src/test/java/io/restest/cli/AReplyUnreadIsNotUndeclaredTest.java @@ -58,6 +58,11 @@ * whatever it is sent: nothing is broken on purpose in a request to it, since the identifier is the * only thing such a request could have changed. The other answers 404, which it does not declare, * and is reported: having nothing left out that a request could carry, it is held to its document. + * + *

The fourth document builds a parameter of one operation, and the body of a reply of another, + * from a shape it never defines. RESTest says so before the run, and both operations are still held + * to the replies they declare, since every one of those was read: the 404 neither declares is + * reported. */ class AReplyUnreadIsNotUndeclaredTest { @@ -149,6 +154,38 @@ class AReplyUnreadIsNotUndeclaredTest { description: the profile """; + /** A shape the document never defines, in a request of one operation and a reply of another. */ + private static final String A_SHAPE_NEVER_DEFINED = """ + openapi: 3.0.3 + info: + title: Night animals + version: "1.0" + paths: + /moths: + get: + operationId: listMoths + parameters: + - name: kind + in: query + schema: + $ref: '#/components/schemas/NoSuchShape' + responses: + "200": + description: every moth + /larks: + get: + operationId: listLarks + responses: + "200": + description: every lark + content: + application/json: + schema: + $ref: '#/components/schemas/NoSuchShape' + components: + schemas: {} + """; + private static WireMockServer api; @BeforeAll @@ -203,7 +240,8 @@ void a_renamed_operation_is_held_to_its_document(@TempDir Path directory) throws .describedAs("what RESTest could not read, and the new names, are said before the " + "run") .contains( - "paths./bats.get: the response '404' is a reference that does not resolve", + "paths./bats.get.responses.404: the response '404' is a reference that " + + "does not resolve", "paths./bats.get: another operation is already called 'listOwls'; " + "renamed 'GET /bats' to stay unique", "renamed 'GET /moths' to stay unique"); @@ -211,6 +249,30 @@ void a_renamed_operation_is_held_to_its_document(@TempDir Path directory) throws assertThat(operationsWith(920, report)).containsExactlyInAnyOrder("listOwls", "GET /moths"); } + @Test + @DisplayName("a shape RESTest cannot read, in a request or inside a reply it read, hides no " + + "reply the document declares, so a 404 it does not declare is reported") + void a_shape_it_cannot_read_hides_no_reply(@TempDir Path directory) throws IOException { + StringWriter screen = new StringWriter(); + + int answer = ARun.run(A_SHAPE_NEVER_DEFINED, api.baseUrl(), directory, screen); + + JsonValue.JsonObject report = report(directory.resolve("out")); + assertThat(answer) + .describedAs("an undeclared status code is a warning, which ends no run with 1 " + + "unless asked; the run said:%n%s", screen) + .isZero(); + assertThat(screen.toString()) + .describedAs("what RESTest could not read is said before the run") + .contains("paths./moths.get: this operation's data includes a shape RESTest does " + + "not fully understand yet", + "paths./larks.get: this operation's data includes a shape RESTest does " + + "not fully understand yet"); + assertThat(codesIn(report)).containsExactly(920); + assertThat(operationsWith(920, report)) + .containsExactlyInAnyOrder("listMoths", "listLarks"); + } + @Test @DisplayName("an identifier a path has no place for is left out, and said, so that no request " + "sent as it was accepted is reported as broken data accepted, and the operation is " diff --git a/restest-core/src/main/java/io/restest/core/model/ApiModel.java b/restest-core/src/main/java/io/restest/core/model/ApiModel.java index 42bf0f5d..e6d082c6 100644 --- a/restest-core/src/main/java/io/restest/core/model/ApiModel.java +++ b/restest-core/src/main/java/io/restest/core/model/ApiModel.java @@ -69,6 +69,12 @@ public record ApiModel( Map securitySchemes, Optional security) { + /** + * What comes before the status code in the location of one of an operation's replies, written + * the way a {@link SpecificationIssue} writes one: {@code paths./pets.get.responses.404}. + */ + private static final String A_REPLY = ".responses."; + public ApiModel { Objects.requireNonNull(title, "title"); Objects.requireNonNull(version, "version"); @@ -170,21 +176,34 @@ public List unreadableOperations() { } /** - * Whether the document says something about this operation that could not be read, though the - * operation itself could: a reply pointing at a shape the document never defines, say. + * Whether a reply the document declares for this operation could not be read, though the + * operation itself could: a reply given as a reference to one the document never defines, say. * - *

Such an operation is tested with what could be read. Anything that would judge it by what - * the document leaves out has to know, since what could not be read may be what it is looking - * for. An operation that only had to be given another name, because the document gives its name - * to another operation too, was read whole, and is not one of these. + *

Such an operation is tested with what could be read. Anything that would judge a reply by + * what the document leaves out - a status code, a content type - has to know, since the reply + * that could not be read may be the very one that declares it. Nothing else set aside about an + * operation hides a reply it declares: not a parameter or a request body of a shape RESTest does + * not fully understand, not such a shape inside a reply that was read, and not a name of + * RESTest's own, given because the document gives the operation's name to another one too. * * @param operation the operation, by its identifier - * @return whether reading it set something aside + * @return whether reading it set aside a reply it declares */ public boolean readInPart(OperationId operation) { Objects.requireNonNull(operation, "operation"); return issues.stream().anyMatch(issue -> issue.effect() == SpecificationIssue.Effect.DEGRADED - && issue.operation().filter(operation::equals).isPresent()); + && issue.operation().filter(operation::equals).isPresent() + && atAReply(issue.location())); + } + + /** + * Whether a location is one of an operation's replies itself, and not a place inside one such + * as {@code paths./pets.get.responses.404.headers.X-Rate}: a status code, which holds no dot, + * ends it. + */ + private static boolean atAReply(String location) { + int at = location.lastIndexOf(A_REPLY); + return at >= 0 && location.indexOf('.', at + A_REPLY.length()) < 0; } /** Every operation using the given method, in document order. */ diff --git a/restest-core/src/main/java/io/restest/core/model/SpecificationIssue.java b/restest-core/src/main/java/io/restest/core/model/SpecificationIssue.java index afd3b9af..c57a35f8 100644 --- a/restest-core/src/main/java/io/restest/core/model/SpecificationIssue.java +++ b/restest-core/src/main/java/io/restest/core/model/SpecificationIssue.java @@ -37,7 +37,9 @@ * about each one, right next to it. * * @param location where in the document the problem is, written the way the document is navigated: - * {@code paths./pets.get.parameters[2]}. Precise enough to open the file and look + * {@code paths./pets.get.parameters[2]}. Precise enough to open the file and look. A reply that + * could not be read at all is located at the reply itself, {@code paths./pets.get.responses.404}, + * and nothing else is: that is how {@link ApiModel#readInPart} tells it apart * @param message what could not be used and why, in a sentence a reader of the report can act on * @param operation the operation it concerns, absent when the problem is the document as a whole * @param effect what it cost us @@ -97,7 +99,11 @@ public static SpecificationIssue skipped(String location, OperationId operation, return about(location, operation, message, Effect.OPERATION_SKIPPED); } - /** An operation being tested with less than the document says. */ + /** + * An operation being tested with less than the document says. When what was set aside is one of + * its replies, whole, the location is that reply's, {@code paths./pets.get.responses.404}; for + * anything else, a part of a reply included, it is somewhere else. + */ public static SpecificationIssue degraded(String location, OperationId operation, String message) { return about(location, operation, message, Effect.DEGRADED); diff --git a/restest-core/src/main/java/io/restest/core/oracle/WhyNotJudged.java b/restest-core/src/main/java/io/restest/core/oracle/WhyNotJudged.java index 72ebacd1..eca9638c 100644 --- a/restest-core/src/main/java/io/restest/core/oracle/WhyNotJudged.java +++ b/restest-core/src/main/java/io/restest/core/oracle/WhyNotJudged.java @@ -61,11 +61,12 @@ public enum WhyNotJudged { NO_REPLY_DECLARED("noReplyDeclared", "the document declares no reply for the operation"), /** - * RESTest could not read everything the document says about the operation - a reply pointing - * at a shape it never defines, say - so what looks undeclared may be declared in the part that - * could not be read. + * RESTest could not read a reply the document declares for the operation - one given as a + * reference to a reply the document never defines, say - so what looks undeclared may be + * declared in the reply that could not be read. */ - READ_IN_PART("readInPart", "RESTest could not read all the document says about the operation"), + READ_IN_PART("readInPart", + "RESTest could not read a reply the document declares for the operation"), /** The document declares the status code with no content type, describing it only in words. */ NO_CONTENT_TYPE_DECLARED("noContentTypeDeclared", diff --git a/restest-core/src/test/java/io/restest/core/model/ApiModelTest.java b/restest-core/src/test/java/io/restest/core/model/ApiModelTest.java index 5d530ff7..29163dd7 100644 --- a/restest-core/src/test/java/io/restest/core/model/ApiModelTest.java +++ b/restest-core/src/test/java/io/restest/core/model/ApiModelTest.java @@ -267,19 +267,25 @@ void an_issue_says_what_it_cost() { } @Test - @DisplayName("an operation is read in part only when something about it was set aside and the " + @DisplayName("an operation is read in part only when a reply it declares was set aside and the " + "operation kept, not when it was skipped whole, only given another name because the " + "document gives its name to another operation too, only rid of something no request " - + "could carry, or the issue was the document's") + + "could carry, set aside something other than a reply, or the issue was the document's") void an_operation_read_in_part() { OperationId listPets = OperationId.of("GET /pets"); OperationId deletePet = OperationId.of("deletePet"); OperationId listCats = OperationId.of("GET /cats"); OperationId listOwners = OperationId.of("GET /owners"); + OperationId addPet = OperationId.of("addPet"); ApiModel api = ApiModel.of("Pets", "1.0", List.of()).withIssues(List.of( SpecificationIssue.renamed("paths./pets.get", listPets, "renamed to stay unique"), - SpecificationIssue.degraded("paths./pets.get", listPets, + SpecificationIssue.degraded("paths./pets.get.responses.404", listPets, "the response '404' is a reference that does not resolve"), + SpecificationIssue.degraded("paths./pets.post", addPet, + "this operation's data includes a shape RESTest does not fully understand " + + "yet"), + SpecificationIssue.degraded("paths./pets.post.responses.201.headers.Location", + addPet, "the header 'Location' is a reference that does not resolve"), SpecificationIssue.skipped("paths./pets/{id}.delete", deletePet, "the schema does not resolve"), SpecificationIssue.renamed("paths./cats.get", listCats, "renamed to stay unique"), @@ -295,6 +301,11 @@ void an_operation_read_in_part() { assertThat(api.readInPart(listOwners)) .describedAs("what no request could carry was never going to be judged") .isFalse(); + assertThat(api.readInPart(addPet)) + .describedAs("a shape it could not read, in a request or inside a reply it did " + + "read, and a part of a reply it could not read, hide no reply it " + + "declares") + .isFalse(); assertThat(api.readInPart(OperationId.of("getPet"))).isFalse(); } diff --git a/restest-oracles/src/main/java/io/restest/oracles/HeldToTheDocument.java b/restest-oracles/src/main/java/io/restest/oracles/HeldToTheDocument.java index aa1dd713..bcb0af66 100644 --- a/restest-oracles/src/main/java/io/restest/oracles/HeldToTheDocument.java +++ b/restest-oracles/src/main/java/io/restest/oracles/HeldToTheDocument.java @@ -41,11 +41,11 @@ * * *

None of these is a reply a rule could not judge: there was nothing for it to judge. An - * operation RESTest could read only in part - a reply pointing at a shape that does not exist, say - - * is different, and each rule deals with it itself: what the part it read declares can still be - * held to, while what looks undeclared may be declared in the part it could not read. An operation - * RESTest only had to give another name, because the document gives its name to another operation - * too, was read whole and is held to it like any other. + * operation one of whose replies RESTest could not read - one given as a reference to a reply the + * document never defines, say - is different, and each rule deals with it itself: what the replies + * it read declare can still be held to, while what looks undeclared may be declared in the reply it + * could not read. An operation RESTest only had to give another name, because the document gives + * its name to another operation too, was read whole and is held to it like any other. */ final class HeldToTheDocument { diff --git a/restest-oracles/src/test/java/io/restest/oracles/UndeclaredContentTypeOracleTest.java b/restest-oracles/src/test/java/io/restest/oracles/UndeclaredContentTypeOracleTest.java index 1553e8c4..db68f70a 100644 --- a/restest-oracles/src/test/java/io/restest/oracles/UndeclaredContentTypeOracleTest.java +++ b/restest-oracles/src/test/java/io/restest/oracles/UndeclaredContentTypeOracleTest.java @@ -83,10 +83,10 @@ class UndeclaredContentTypeOracleTest { private static final ApiModel API = ApiModel.of("Pets", "1.0.0", List.of(ADD_PET, LIST_OWNERS, LIST_VETS, LIST_VISITS, LIST_CLINICS, LIST_KENNELS)) - .withIssues(List.of(SpecificationIssue.degraded("paths./visits.get", LIST_VISITS.id(), - "the response '400' could not be represented"), - SpecificationIssue.degraded("paths./kennels.get", LIST_KENNELS.id(), - "the response '200' could not be represented"), + .withIssues(List.of(SpecificationIssue.degraded("paths./visits.get.responses.400", + LIST_VISITS.id(), "the response '400' could not be represented"), + SpecificationIssue.degraded("paths./kennels.get.responses.200", + LIST_KENNELS.id(), "the response '200' could not be represented"), SpecificationIssue.renamed("paths./clinics.get", LIST_CLINICS.id(), "renamed to stay unique"))); diff --git a/restest-oracles/src/test/java/io/restest/oracles/UndeclaredStatusOracleTest.java b/restest-oracles/src/test/java/io/restest/oracles/UndeclaredStatusOracleTest.java index 841ffa0e..54113d55 100644 --- a/restest-oracles/src/test/java/io/restest/oracles/UndeclaredStatusOracleTest.java +++ b/restest-oracles/src/test/java/io/restest/oracles/UndeclaredStatusOracleTest.java @@ -89,12 +89,12 @@ class UndeclaredStatusOracleTest { private static final ApiModel API = ApiModel.of("Pets", "1.0.0", List.of(LIST_PETS, ADD_PET, ADD_OWNER, LIST_VISITS, LIST_OWNERS, HEALTH, LIST_VETS, LIST_CLINICS, LIST_KENNELS)) - .withIssues(List.of(SpecificationIssue.degraded("paths./vets.get", LIST_VETS.id(), - "the response '404' is a reference that does not resolve to anything the " - + "document declares"), - SpecificationIssue.degraded("paths./kennels.get", LIST_KENNELS.id(), - "the response '200' is a reference that does not resolve to anything " - + "the document declares"), + .withIssues(List.of(SpecificationIssue.degraded("paths./vets.get.responses.404", + LIST_VETS.id(), "the response '404' is a reference that does not resolve to " + + "anything the document declares"), + SpecificationIssue.degraded("paths./kennels.get.responses.200", + LIST_KENNELS.id(), "the response '200' is a reference that does not " + + "resolve to anything the document declares"), SpecificationIssue.renamed("paths./clinics.get", LIST_CLINICS.id(), "renamed to stay unique"))); diff --git a/restest-spec/src/main/java/io/restest/spec/OperationConverter.java b/restest-spec/src/main/java/io/restest/spec/OperationConverter.java index c3638e5b..dd89a773 100644 --- a/restest-spec/src/main/java/io/restest/spec/OperationConverter.java +++ b/restest-spec/src/main/java/io/restest/spec/OperationConverter.java @@ -513,17 +513,23 @@ private static List convertResponses(ApiResponses responses, Comp } List converted = new ArrayList<>(); responses.forEach((status, response) -> { + // A reply that cannot be read is said of the reply itself, not of the operation: that + // is how the model tells it from anything else set aside, which leaves every reply the + // operation declares where the checks of a status code and a content type can see it. + String reply = location + ".responses." + status; boolean wasReference = response != null && response.get$ref() != null; ApiResponse resolved = resolveResponse(response, components); if (resolved == null) { if (wasReference) { - issues.add(SpecificationIssue.degraded(location, operationId, "the response '" + issues.add(SpecificationIssue.degraded(reply, operationId, "the response '" + status + "' is a reference that does not resolve to anything the " + "document declares")); } return; } Map content = convertContent(resolved.getContent(), components); + // A header that cannot be read leaves the reply's status code and content types where + // the checks can see them, so it is not said of the reply as a whole. Map headers = convertHeaders(resolved.getHeaders(), components, operationId, location, issues); try { @@ -532,7 +538,7 @@ private static List convertResponses(ApiResponses responses, Comp } catch (IllegalArgumentException e) { // A malformed response - most often an unusable status key - costs only itself: the // operation is still testable against whatever it does answer correctly. - issues.add(SpecificationIssue.degraded(location, operationId, + issues.add(SpecificationIssue.degraded(reply, operationId, "the response '" + status + "' could not be represented: " + message(e))); } }); diff --git a/restest-spec/src/test/java/io/restest/spec/MalformedDocumentTest.java b/restest-spec/src/test/java/io/restest/spec/MalformedDocumentTest.java index 67468bdd..d1455704 100644 --- a/restest-spec/src/test/java/io/restest/spec/MalformedDocumentTest.java +++ b/restest-spec/src/test/java/io/restest/spec/MalformedDocumentTest.java @@ -270,6 +270,12 @@ void a_junk_response_key_degrades_only_that_response(@TempDir Path dir) throws E assertThat(api.operations().get(0).responses()).hasSize(1); assertThat(api.issues()).hasSize(1); assertThat(api.issues().get(0).effect()).isEqualTo(SpecificationIssue.Effect.DEGRADED); + assertThat(api.issues().get(0).location()) + .describedAs("said of the reply, not of the operation") + .isEqualTo("paths./widgets.get.responses.200 OK"); + assertThat(api.readInPart(OperationId.of("GET /widgets"))) + .describedAs("the reply it could not read may declare what looks undeclared") + .isTrue(); } @Test @@ -352,6 +358,9 @@ void an_unresolved_header_reference_is_reported(@TempDir Path dir) throws Except assertThat(api.operations().get(0).responses().get(0).headers()).isEmpty(); assertThat(api.issues()).hasSize(1); assertThat(api.issues().get(0).effect()).isEqualTo(SpecificationIssue.Effect.DEGRADED); + assertThat(api.readInPart(OperationId.of("GET /widgets"))) + .describedAs("a header it could not read hides no status code and no content type") + .isFalse(); } @Test