Merge pull request #471 from ipfs/gateway-content-location

gateways: document Content-Location

Author: hacdias

src/http-gateways/path-gateway.md

@@ -4,7 +4,7 @@ description: >
   The comprehensive low-level HTTP Gateway enables the integration of IPFS
   resources into the HTTP stack through /ipfs and /ipns namespaces, supporting
   both deserialized and verifiable response types.
-date: 2023-03-30
+date: 2024-04-17
 maturity: reliable
 editors:
   - name: Marcin Rataj
@@ -242,13 +242,35 @@ These are the equivalents:
 When both `Accept` HTTP header  and `format` query parameter are present,
 `Accept` SHOULD take precedence.
 
+A Client SHOULD include the `format` query parameter in the request URL, in
+addition to the `Accept` header. This provides the best interoperability and
+ensures consistent HTTP cache behavior across various gateway implementations.
+
+A Gateway SHOULD include the
+[`Content-Location`](#content-location-response-header) header in the response when:
+- the request contains an `Accept` header specifying a well-known response
+  format, but the URL does not include the `format` query parameter
+- the `format` parameter is present, but does not match the format from `Accept`
+
 ### `dag-scope` (request query parameter)
 
-Only used on CAR requests, same as :ref[dag-scope] from :cite[trustless-gateway].
+Optional, can be used to limit the scope of verifiable DAG requests such as CAR, same as :ref[dag-scope] from :cite[trustless-gateway].
 
 ### `entity-bytes` (request query parameter)
 
-Only used on CAR requests, same as :ref[entity-bytes] from :cite[trustless-gateway].
+Optional, can be used to limit the scope of verifiable DAG requests such as CAR, same as :ref[entity-bytes] from :cite[trustless-gateway].
+
+### `car-version` (request query parameter)
+
+Optional, specific to CAR requests, same as :ref[car-version] from :cite[trustless-gateway].
+
+### `car-order` (request query parameter)
+
+Optional, specific to CAR requests, same as :ref[car-order] from :cite[trustless-gateway].
+
+### `car-dups` (request query parameter)
+
+Optional, specific to CAR requests, same as :ref[car-dups] from :cite[trustless-gateway].
 
 # HTTP Response
 
@@ -486,12 +508,35 @@ To illustrate, `?filename=testтест.pdf` should produce:
   not attempt to render raw bytes. CID and `.bin` file extension should be used
   if  a custom `filename` was not provided with the request.
 
+### `Content-Location` (response header)
+
+Returned when a non-default content format has been negotiated with the
+[`Accept` header](#accept-request-header) but `format` was missing from the URL.
+
+The value of this field SHOULD include
+the URL of the resource with the `format` query parameter included, so that
+generic HTTP caches can store deserialized, CAR, and block responses separately.
+
+:::note
+
+For example, a request to `/ipfs/{cid}` with `Accept: application/vnd.ipld.raw`
+SHOULD return a `Content-Location: /ipfs/{cid}?format=raw` header in order for
+block response to be cached separately from deserialized one.
+
+:::
+
 ### `Content-Length` (response header)
 
 Represents the length of returned HTTP payload.
 
+:::warning
+
+<!-- TODO https://github.com/ipfs/specs/issues/461 -->
+
 NOTE: the value may differ from the real size of requested data if compression or chunked `Transfer-Encoding` are used.
-<!-- TODO (https://github.com/ipfs/in-web-browsers/issues/194) IPFS clients looking for UnixFS file size should use value from `X-Ipfs-DataSize` instead. -->
+See [ipfs/specs#461](https://github.com/ipfs/specs/issues/461).
+
+:::
 
 ### `Content-Range` (response header)
 
@@ -513,8 +558,6 @@ deterministic.
 Returned only when response status code is [`301` Moved Permanently](#301-moved-permanently).
 The value informs the HTTP client about new URL for requested resource.
 
-This header is more widely used in [SUBDOMAIN_GATEWAY.md](./SUBDOMAIN_GATEWAY.md#location-response-header).
-
 #### Use in directory URL normalization
 
 Gateway MUST return a redirect when a valid UnixFS directory was requested
@@ -530,6 +573,10 @@ It also ensures the same behavior on path gateways (`https://example.com/ipfs/ci
 and origin-isolated HTTP contexts `https://cid.ipfs.dweb.link`
 or non-HTTP URLs like `ipfs://cid`, where empty path component is implicit `/`.
 
+#### Use in interop with Subdomain Gateway
+
+See [`Location` section](https://specs.ipfs.tech/http-gateways/subdomain-gateway/#location-response-header) of :cite[subdomain-gateway].
+
 ### `X-Ipfs-Path` (response header)
 
 Used for HTTP caching and indicating the IPFS address of the data.
@@ -567,15 +614,6 @@ NOTE: while the first CID will change every time any article is changed,
 the last root (responsible for specific article or a subdirectory) may not
 change at all, allowing for smarter caching beyond what standard Etag offers.
 
-<!-- TODO: https://github.com/ipfs/in-web-browsers/issues/194
-- `X-Ipfs-DagSize`
-    - Indicates the total size of the DAG (raw data + IPLD metadata) representing the requested resource.
-        - For UnixFS this is equivalent  to  `CumulativeSize` from   `ipfs files stat`
-- `X-Ipfs-DataSize`
-    - Indicates the original byte size of the raw data (not impacted by HTTP transfer encoding or compression), without IPFS/IPLD metadata.
-        - For UnixFS this is equivalent to `Size` from `ipfs files stat` or `ipfs dag stat`
--->
-
 ### `X-Content-Type-Options` (response header)
 
 Optional, present in certain response types:

src/http-gateways/trustless-gateway.md

@@ -4,7 +4,7 @@ description: >
   The minimal subset of HTTP Gateway response types facilitates data retrieval
   via CID and ensures integrity verification, all while eliminating the need to
   trust the gateway itself.
-date: 2023-06-20
+date: 2024-04-17
 maturity: reliable
 editors:
   - name: Marcin Rataj
@@ -90,7 +90,18 @@ mode (no deserialized responses) and `Accept` header is missing.
 
 ## Request Query Parameters
 
-### :dfn[dag-scope] (request query parameter)
+### :dfn[`format`] (request query parameter)
+
+Same as [`format`](https://specs.ipfs.tech/http-gateways/path-gateway/#format-request-query-parameter) in :cite[path-gateway], but with limited number of supported response types:
+- `format=raw` → `application/vnd.ipld.raw`
+- `format=car` → `application/vnd.ipld.car`
+- `format=ipns-record` → `application/vnd.ipfs.ipns-record`
+
+A Client SHOULD include the `format` query parameter in the request URL, in
+addition to the `Accept` header. This provides the best interoperability and
+ensures consistent HTTP cache behavior across various gateway implementations.
+
+### :dfn[`dag-scope`] (request query parameter)
 
 Optional, `dag-scope=(block|entity|all)` with default value `all`, only available for CAR requests.
 
@@ -111,7 +122,7 @@ path segments.
 
 When present, returned `Etag` must include unique prefix based on the passed scope type.
 
-### :dfn[entity-bytes] (request query parameter)
+### :dfn[`entity-bytes`] (request query parameter)
 
 The optional `entity-bytes=from:to` parameter is available only for CAR
 requests.
@@ -183,6 +194,30 @@ returned:
     returned to the client, the HTTP status code has already been sent to the
     client.
 
+### :dfn[`car-version`] (request query parameter)
+
+Optional, only used on CAR requests.
+
+Serves same purpose as [CAR `version` content type parameter](#car-version-content-type-parameter).
+
+In case both are present in the request, the value from the [`Accept`](#accept-request-header) HTTP Header has priority and a matching [`Content-Location`](#content-location-response-header) SHOULD be returned with the response.
+
+### :dfn[`car-order`] (request query parameter)
+
+Optional, only used on CAR requests.
+
+Serves same purpose as [CAR `order` content type parameter](#car-order-content-type-parameter).
+
+In case both are present in the request, the value from the [`Accept`](#accept-request-header) HTTP Header has priority and a matching [`Content-Location`](#content-location-response-header) SHOULD be returned with the response.
+
+### :dfn[`car-dups`] (request query parameter)
+
+Optional, only used on CAR requests.
+
+Serves same purpose as [CAR `dups` content type parameter](#car-dups-content-type-parameter).
+
+In case both are present in the request, the value from the [`Accept`](#accept-request-header) HTTP Header has priority and a matching [`Content-Location`](#content-location-response-header) SHOULD be returned with the response.
+
 # HTTP Response
 
 Below MUST be implemented **in addition** to "HTTP Response" of :cite[path-gateway].
@@ -203,6 +238,12 @@ If a CAR stream was requested:
 
 MUST be returned and set to `attachment` to ensure requested bytes are not rendered by a web browser.
 
+### `Content-Location` (response header)
+
+Same as in :cite[path-gateway], SHOULD be returned when Trustless Gateway
+supports more than a single response format and the `format` query parameter is
+missing or does not match well-known format from `Accept` header.
+
 # Block Responses (application/vnd.ipld.raw)
 
 An opaque bytes matching the requested block CID
@@ -217,7 +258,7 @@ A CAR stream for the requested
 content type (with optional `order` and `dups` params), path and optional
 `dag-scope` and `entity-bytes` URL parameters.
 
-## CAR version
+## CAR `version` (content type parameter)
 
 Value returned in
 [`CarV1Header.version`](https://ipld.io/specs/transport/car/carv1/#header)