Merge pull request #161 from ipfs-cluster/v1.1.2

Documentation updates for v1.1.2

Author: hsanjuan

content/documentation/_index.md

@@ -11,7 +11,7 @@ aliases = [
 
 # Documentation
 
-<div class="tipbox tip">Updated to version <a href="https://github.com/ipfs-cluster/ipfs-cluster/blob/master/CHANGELOG.md">1.1.1 (see the Changelog)</a>.</div>
+<div class="tipbox tip">Updated to version <a href="https://github.com/ipfs-cluster/ipfs-cluster/blob/master/CHANGELOG.md">1.1.2 (see the Changelog)</a>.</div>
 
 Welcome to IPFS Cluster documentation. The different sections of the documentation will explain how to setup, start, and operate a Cluster. Operating a production IPFS Cluster can be a daunting task if you are not familiar with concepts around [IPFS](https://ipfs.io) and peer-2-peer networking ([libp2p](https://libp2p.io) in particular). We aim to provide comprehensive documentation and guides but we are always open for improvements: documentation issues can be submitted to the [ipfs-cluster-website repository](https://github.com/ipfs-cluster/ipfs-cluster-website).
 

content/documentation/reference/api.md

@@ -44,7 +44,7 @@ There are considerations to take into account here:
 
 Currently IPFS Cluster supports adding with two DAG-formats (`?format=` query parameter):
 
-* By default it uses the `unixfs` format. In this mode, the request body is expected to be a multipart just like described in [`/api/v0/add` documentation](https://docs.ipfs.io/reference/http/api/#api-v0-add). The `/add` endpoint supports the same optional parameters as IPFS does and produces exactly the same DAG as go-ipfs when adding files. In UnixFS, files uploaded in the request are chunked and a DAG is built replicating the desired folder layout. This is done by the cluster peer.
+* By default it uses the `unixfs` format. In this mode, the request body is expected to be a multipart just like described in [`/api/v0/add` documentation](https://docs.ipfs.tech/reference/kubo/rpc/#api-v0-add). The `/add` endpoint supports the same optional parameters as IPFS does and produces exactly the same DAG as go-ipfs when adding files. In UnixFS, files uploaded in the request are chunked and a DAG is built replicating the desired folder layout. This is done by the cluster peer.
 * Alternatively, the `/add` endpoint also accepts a CAR file with `?format=car` format. In this case, the CAR file already includes the blocks that need to be added to IPFS and Cluster does not do any further processing (similarly to `ipfs dag import`). At the moment, the `/add` endpoint will process only a single CAR file and this file must have only one root (the one that will be pinned). CAR files allow adding arbitrary IPLD-DAGs through the Cluster API.
 
 <div class="tipbox warning">Using the <code>/add</code> endpoint with Nginx in front as a reverse proxy may cause problems. Make sure to add <code>?stream-channels=false</code> to every Add request to avoid them.<br /><br />The problems manifest themselves as "connection reset by peer while reading upstream" errors in the logs. They are caused by read after write on the HTTP request/response cycle: Nginx refuses any application that has started sending the response body to read further from the request body (<a href="https://trac.nginx.org/nginx/ticket/1293" target="_blank">see bug report</a>). IPFS and IPFS Cluster send object updates while adding files, therefore triggering the situation, which is otherwise legal per HTTP specs. The issue depends on Nginx internal buffering and may appear very sporadically or not at all, but it exists.</div>

content/documentation/reference/configuration.md

@@ -135,11 +135,18 @@ The `leave_on_shutdown` option allows a peer to remove itself from the *peerset*
 |     `memory_limit_bytes` | `0` | Controls the maximum amount of RAM memory that the libp2p host should use. When set to `0`, the amount will be set to a 25% of the machine's memory or a minimum of 1GiB. Note that this affects only the libp2p resources and not the overall memory of the cluster node |
 |     `file_descriptors_limit` | `0` | Controls the maximum number of file-descriptors to use. When set to `0`, the limit will be set to 50% of the total amount of file descriptors available to the process. |
 | `}` |||
+| `pubsub {` | | A libp2p pubsub configuration object. This allows to configure pubsub internals. Defaults are optimized for ipfs-cluster pubsub usecase, where metrics and CRDT-heads are broadcasted, and it is not generally fatal if a message gets lost. Deviations from pubsub defaults aim to reduce unnecessary chatter in standard clusters as pubsub ends up using a lot of bandwidth. |
+|     `seen_messages_ttl` | `"30m0s"` | How long before a seen pubsub message can be forgotten. A seen pubsub message is ignored and not rebroadcasted to peers. This should be high enough that a pubsub message has time to reach all peers in the cluster and some more. |
+|     `heartbeat_interval` | `"10s"` | Time between heartbeats. A heartbeat triggers mesh maintenance and emits gossip with our entries. Increasing it reduces chatter. Reducing increases speed. |
+|     `d_factor` | `4` | A factor to multiply default mesh values (Dlo-5, D-6, Dhigh-12, DLazy-6). It generally controls how many peers we are meshed with, and that influences how much it costs us to broadcast a message. Few peers means little bandwidth on this peer, at the expense of other peers having to re-broadcast the message more often to reach full distribution, so more chatter in the end. A higher number means less chatter, but more effort per message. The default of 4 makes D=16. That means we will send every broadcast to 16 peers or so. |
+|     `history_gossip` | `2` | How many of our heartbeats should include IHAVE entries for each known message. Increasing makes chatter consume more bandwidth, but can improves message delivery. |
+|     `history_length` | `6` | For how many heartbeats are message requests from other peers honored. This means that after 6 heartbeats of receiving a pubsub message, we will not be sending it anymore to anyone that requests it. Increasing it improves message delivery, but also network churn since slow peers might be requesting all messages and we would provide them. |
+|     `flood_publish` | `false` | Enabling means that the first pubsub message hop is flooded to all subscribed peers (not just ~D). Improves delivery but might be overkill for peers that are not very well connected or more limited. |
+| `}` |||
 |`dial_peer_timeout` | `"3s"` | How long to wait when dialing a cluster peer before giving up. |
 |`state_sync_interval`| `"10m0s"` | Interval between automatic triggers of [`StateSync`](https://godoc.org/github.com/ipfs-cluster/ipfs-cluster#Cluster.StateSync). |
 |`pin_recover_interval`| `"1h0m0s"` | Interval between automatic triggers of [`RecoverAllLocal`](https://godoc.org/github.com/ipfs-cluster/ipfs-cluster#Cluster.RecoverAllLocal). This will automatically re-try pin and unpin operations that failed. |
-|`replication_factor_min` | `-1` | Specifies the default minimum number of peers that should be pinning an item. -1 == all. |
-|`replication_factor_max` | `-1` | Specifies the default maximum number of peers that should be pinning an item. -1 == all. |
+|`replication_factor_min` | `-1` | Specifies the default minimum number of peers that should be pinning an item. -1 == all. ||`replication_factor_max` | `-1` | Specifies the default maximum number of peers that should be pinning an item. -1 == all. |
 |`monitor_ping_interval` | `"15s"` | Interval for sending a `ping` (used to detect downtimes). |
 |`peer_watch_interval`| `"5s"` | Interval for checking the current cluster peerset and detect if this peer was removed from the cluster (and shut-down). |
 |`mdns_interval` | `"10s"` | Setting it to `"0"` disables mDNS. Setting to a larger value enables mDNS but no longer controls anything. |