Merge pull request #417 from ipfs/ipip/schema-peer

IPIP-417: Delegated Peer Routing HTTP API
ipfs · Sep 25, 2023 · 970b81d · 970b81d
2 parents 9d9e50a + 95999ef
commit 970b81d
Show file tree

Hide file tree

Showing 2 changed files with 189 additions and 55 deletions.
diff --git a/src/ipips/ipip-0417.md b/src/ipips/ipip-0417.md
@@ -0,0 +1,68 @@
+---
+title: "IPIP-0417: Delegated Peer Routing HTTP API"
+date: 2023-05-29
+ipip: proposal
+editors:
+  - name: Henrique Dias
+    github: hacdias
+    url: https://hacdias.com/
+    affiliation:
+      name: Protocol Labs
+      url: https://protocol.ai/
+relatedIssues:
+  - https://github.com/ipfs/specs/pull/410
+  - https://github.com/ipfs/kubo/pull/9877
+order: 417
+tags: ['ipips']
+---
+
+## Summary
+
+This IPIP specifies `/routing/v1/peers/{peer-id}` HTTP API to offload peer routing onto another server.
+
+## Motivation
+
+The motivation of this IPIP extends the one of :cite[ipip-0337] and :cite[ipip-0379],
+which introduced delegated content routing and delegated naming, respectively. Now,
+we expand upon those basis to introduce peer routing, reducing the barrier for interaction
+across different systems.
+
+## Detailed design
+
+Add `/routing/v1/peers/{peer-id}` to the existing :cite[http-routing-v1] specification,
+as well as the new Peer schema, that replaces the existing "known transfer protocols".
+
+## Design rationale
+
+In line with the remaining Routing V1 API, this IPIP introduces a new HTTP GET
+endpoint that is used to retrieve peer records for a certain peer.
+
+### User benefit
+
+The user benefit brought by this PR is similar to the one in :cite[ipip-0379],
+but instead of offloading the naming process, we offload the peer discovery and
+routing.
+
+### Compatibility
+
+The section "Known Transfer Protocols" has been removed and replaced by a "Known Schemas"
+section. Before, we used to have protocol specific schemas, such as `bitswap` and `graphsync-filecoinv1`.
+
+The usage of these schemas is no longer encouraged. Instead, clients and
+servers SHOULD be updated to use the new, more generic, `peer` schema, which
+avoids returning the same peer multiple times, making results more efficient
+when a peer supports more than one protocol.
+
+See more in the "Compatibility" section of :cite[ipip-0337].
+
+### Security
+
+See the "Security" section of :cite[ipip-0337].
+
+### Alternatives
+
+See the "Alternatives" section of :cite[ipip-0337].
+
+### Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
diff --git a/src/routing/http-routing-v1.md b/src/routing/http-routing-v1.md
@@ -4,16 +4,31 @@ description: >
   Delegated routing is a mechanism for IPFS implementations to use for offloading
   content routing and naming to another process/server. This specification describes
   an HTTP API for delegated content routing.
-date: 2023-03-22
+date: 2023-08-31
 maturity: reliable
 editors:
   - name: Gus Eggert
     github: guseggert
+    affiliation:
+      name: Protocol Labs
+      url: https://protocol.ai/
   - name: Masih H. Derkani
     github: masih
+    affiliation:
+      name: Protocol Labs
+      url: https://protocol.ai/
   - name: Henrique Dias
     url: https://hacdias.com/
     github: hacdias
+    affiliation:
+      name: Protocol Labs
+      url: https://protocol.ai/
+  - name: Marcin Rataj
+    github: lidel
+    url: https://lidel.org/
+    affiliation:
+      name: Protocol Labs
+      url: https://protocol.ai/
 xref:
   - ipns-record
 order: 0
@@ -32,75 +47,89 @@ As such, human-readable encodings of types are preferred. This specification may
 
 - CIDs are always string-encoded using a [multibase]-encoded [CIDv1].
 - Multiaddrs are string-encoded according to the [human-readable multiaddr specification][multiaddr].
-- Peer IDs are string-encoded according [PeerID string representation specification][peer-id-representation].
+- Peer IDs are string-encoded according [PeerID string representation specification][peer-id-representation]: either a Multihash in Base58btc, or a CIDv1 with libp2p-key (`0x72`) codec.
 - Multibase bytes are string-encoded according to [the Multibase spec][multibase], and SHOULD use base64.
 - Timestamps are Unix millisecond epoch timestamps.
 
 Until required for business logic, servers should treat these types as opaque strings, and should preserve unknown JSON fields.
 
-### Versioning
+## Versioning
 
 This API uses a standard version prefix in the path, such as `/v1/...`. If a backwards-incompatible change must be made, then the version number should be increased.
 
-### Provider Records
+## Content Routing API
 
-A provider record contains information about a content provider, including the transfer protocol and any protocol-specific information useful for fetching the content from the provider.
+### `GET /routing/v1/providers/{cid}`
 
-The information required to write a record to a router (*"write" provider records*) may be different than the information contained when reading provider records (*"read" provider records*).
+#### Path Parameters
 
-For example, indexers may require a signature in `bitswap` write records for authentication of the peer contained in the record, but the read records may not include this authentication information.
+- `cid` is the [CID](https://github.com/multiformats/cid) to fetch provider records for.
 
-Both read and write provider records have a minimal required schema as follows:
+#### Response Status Codes
+
+- `200` (OK): the response body contains 0 or more records.
+- `404` (Not Found): must be returned if no matching records are found.
+- `422` (Unprocessable Entity): request does not conform to schema or semantic constraints.
+
+#### Response Body
 
 ```json
 {
-    "Protocol": "<transfer_protocol_name>",
-    "Schema": "<transfer_protocol_schema>",
+  "Providers": [
+    {
+      "Schema": "<schema>",
+      "ID": "bafz...",
+      "Addrs": ["/ip4/..."],
+      ...
+    },
     ...
+  ]
 }
 ```
 
-Where:
+The `application/json` responses SHOULD be limited to 100 providers.
 
-- `Protocol` is the multicodec name of the transfer protocol or an opaque string (for experimenting with novel protocols without a multicodec)
-- `Schema` denotes the schema to use for encoding/decoding the record
-  - This is separate from the `Protocol` to allow this HTTP API to evolve independently of the transfer protocol
-  - Implementations should switch on this when parsing records, not on `Protocol`
-- `...` denotes opaque JSON, which may contain information specific to the transfer protocol
+The client SHOULD be able to make a request with `Accept: application/x-ndjson` and get a [stream](#streaming) with more results.
 
-Specifications for some transfer protocols are provided in the "Transfer Protocols" section.
+Each object in the `Providers` list is a record conforming to a schema, usually the [Peer Schema](#peer-schema).
 
-## Content Providers API
+## Peer Routing API
 
-### `GET /routing/v1/providers/{cid}`
+### `GET /routing/v1/peers/{peer-id}`
 
 #### Path Parameters
 
-- `cid` is the [CID](https://github.com/multiformats/cid) to fetch provider records for.
+- `peer-id` is the [Peer ID](https://github.com/libp2p/specs/blob/master/peer-ids/peer-ids.md) to fetch peer records for,
+represented as a CIDv1 encoded with `libp2p-key` codec.
 
 #### Response Status Codes
 
-- `200` (OK): the response body contains 0 or more records.
+- `200` (OK): the response body contains the peer record.
 - `404` (Not Found): must be returned if no matching records are found.
 - `422` (Unprocessable Entity): request does not conform to schema or semantic constraints.
 
 #### Response Body
 
 ```json
 {
-  "Providers": [
+  "Peers": [
     {
-      "Protocol": "<protocol_name>",
       "Schema": "<schema>",
+      "Protocols": ["<protocol-a>", "<protocol-b>", ...],
+      "ID": "bafz...",
+      "Addrs": ["/ip4/..."],
       ...
-    }
+    },
+    ...
   ]
 }
 ```
 
-Response limit: 100 providers
+The `application/json` responses SHOULD be limited to 100 peers.
+
+The client SHOULD be able to make a request with `Accept: application/x-ndjson` and get a [stream](#streaming) with more results.
 
-Each object in the `Providers` list is a *read provider record*.
+Each object in the `Peers` list is a record conforming to the [Peer Schema](#peer-schema).
 
 ## IPNS API
 
@@ -123,7 +152,7 @@ Each object in the `Providers` list is a *read provider record*.
 
 #### Response Body
 
-The response body contains a  :ref[IPNS Record] serialized using the verifiable [`application/vnd.ipfs.ipns-record`](https://www.iana.org/assignments/media-types/application/vnd.ipfs.ipns-record) protobuf format.
+The response body contains a :ref[IPNS Record] serialized using the verifiable [`application/vnd.ipfs.ipns-record`](https://www.iana.org/assignments/media-types/application/vnd.ipfs.ipns-record) protobuf format.
 
 ### `PUT /routing/v1/ipns/{name}`
 
@@ -163,7 +192,7 @@ with one result per line:
 
 :::note
 
-Streaming is opt-in and backwards-compatibile with clients and servers that do
+Streaming is opt-in and backwards-compatible with clients and servers that do
 not support streaming:
 
 - Requests without the `Accept: application/x-ndjson` header MUST default to
@@ -199,59 +228,96 @@ Access-Control-Allow-Origin: *
 Access-Control-Allow-Methods: GET, OPTIONS
 ```
 
-## Known Transfer Protocols
+## Known Schemas
 
-This section contains a non-exhaustive list of known transfer protocols (by name) that may be supported by clients and servers.
+This section contains a non-exhaustive list of known schemas that MAY be supported by clients and servers.
 
-### Bitswap
+### Peer Schema
 
-- Multicodec name: `transport-bitswap`
-- Schema: `bitswap`
-- Specification: [ipfs/specs/BITSWAP.md](https://github.com/ipfs/specs/blob/main/BITSWAP.md)
+The `peer` schema represents an arbitrary peer.
 
-#### Bitswap Read Provider Records
+```json
+{
+  "Schema": "peer",
+  "ID": "bafz...",
+  "Addrs": ["/ip4/..."],
+  "Protocols": ["transport-bitswap", ...]
+  ...
+}
+```
+
+- `ID`: the [Peer ID][peer-id] as Multihash in Base58btc or CIDv1 with libp2p-key codec.
+- `Addrs`: an optional list of known [multiaddrs][multiaddr] for this peer.
+  - If missing or empty, it means the router server is missing that information, and the client should use `ID` to lookup updated peer information.
+- `Protocols`: an optional list of protocols known to be supported by this peer.
+  - If missing or empty, it means the router server is missing that information, and the client should use `ID` and `Addrs` to lookup connect to the peer and use the [libp2p identify protocol](https://github.com/libp2p/specs/tree/master/identify) to learn about supported ones.
+
+:::note
+
+To allow for protocol-specific fields and future-proofing, the parser MUST
+allow for unknown fields, and the clients MUST ignore unknown ones.
+
+Below is an example on how one could include `protocol-a` and `protocol-b`
+protocols that includes an additional fields `protocol-a` and `protocol-b`.
+
+If the client knows the protocol, they are free to use the extra binary
+(base64) or JSON information contained in the additional field. If that is not
+the case, the field MUST be ignored.
 
 ```json
 {
-  "Protocol": "transport-bitswap",
-  "Schema": "bitswap",
-  "ID": "12D3K...",
-  "Addrs": ["/ip4/..."]
+  "Schema": "peer",
+  "ID": "bafz...",
+  "Addrs": ["/ip4/..."],
+  "Protocols": ["transport-bitswap", "protocol-a", "protocol-b", ...],
+  "protocol-a": "[base64-blob]",
+  "protocol-b": { "foo": "bar" }
 }
 ```
 
-- `ID`: the [Peer ID][peer-id] to contact
-- `Addrs`: a list of known multiaddrs for the peer
-  - This list may be incomplete or incorrect and should only be treated as *hints* to improve performance by avoiding extra peer lookups
+:::
+
+### Legacy Schemas
+
+Legacy schemas include `ID` and optional `Addrs` list just like
+the [`peer` schema](#peer-schema) does.
+
+These schemas are deprecated and SHOULD be replaced with `peer` over time, but
+MAY be returned by some legacy endpoints. In such case, a client MAY parse
+them the same way as the `peer` schema.
 
-The server should respect a passed `transport` query parameter by filtering against the `Addrs` list.
+#### Bitswap Schema
 
-### Filecoin Graphsync
+A legacy schema used by some routers to indicate a peer supports retrieval over
+the `/ipfs/bitswap[/*]` libp2p protocol.
+
+```json
+{
+  "Protocol": "transport-bitswap",
+  "Schema": "bitswap",
+  "ID": "bafz...",
+  "Addrs": ["/ip4/..."]
+}
+```
 
-- Multicodec name: `transport-graphsync-filecoinv1`
-- Schema: `graphsync-filecoinv1`
-- Specification: [ipfs/go-graphsync/blob/main/docs/architecture.md](https://github.com/ipfs/go-graphsync/blob/main/docs/architecture.md)
+#### Graphsync Schema
 
-#### Filecoin Graphsync Read Provider Records
+A legacy schema used by some routers to indicate a peer supports retrieval over
+the [graphsync](https://github.com/ipfs/go-graphsync/blob/main/docs/architecture.md)
+libp2p protocol.
 
 ```json
 {
   "Protocol": "transport-graphsync-filecoinv1",
   "Schema": "graphsync-filecoinv1",
-  "ID": "12D3K...",
+  "ID": "bafz...",
   "Addrs": ["/ip4/..."],
   "PieceCID": "<cid>",
   "VerifiedDeal": true,
   "FastRetrieval": true
 }
 ```
 
-- `ID`: the [Peer ID][peer-id] of the provider
-- `Addrs`: a list of known multiaddrs for the provider
-- `PieceCID`: the CID of the [piece](https://spec.filecoin.io/systems/filecoin_files/piece/#section-systems.filecoin_files.piece) within which the data is stored
-- `VerifiedDeal`: whether the deal corresponding to the data is verified
-- `FastRetrieval`: whether the provider claims there is an unsealed copy of the data available for fast retrieval
-
 [multibase]: https://github.com/multiformats/multibase
 [CIDv1]: https://github.com/multiformats/cid#cidv1
 [multiaddr]: https://github.com/multiformats/multiaddr#specification