diff --git a/docs/.vuepress/config.js b/docs/.vuepress/config.js index 575b0ebf3..07b939a60 100644 --- a/docs/.vuepress/config.js +++ b/docs/.vuepress/config.js @@ -218,6 +218,7 @@ module.exports = { children: [ '/how-to/work-with-blocks', '/how-to/pin-files', + '/how-to/work-with-pinning-services', '/how-to/troubleshoot-file-transfers', '/how-to/take-snapshot', '/how-to/store-play-videos', diff --git a/docs/how-to/pin-files.md b/docs/how-to/pin-files.md index 964e3cda1..a702ee339 100644 --- a/docs/how-to/pin-files.md +++ b/docs/how-to/pin-files.md @@ -6,7 +6,11 @@ description: Learn how to pin files in IPFS in order to keep your files and othe # Pin files using IPFS -Pinning is a very important concept in IPFS. IPFS semantics try to make it feel like every single object is local — there is no "retrieve this file for me from a remote server", just `ipfs cat` or `ipfs get`, which act the same way no matter where the actual object is located. While this is nice, sometimes you want to be able to control what you keep around. Pinning is the mechanism that allows you to tell IPFS to always keep a given object local. IPFS has a fairly aggressive caching mechanism that will keep an object local for a short time after you perform any IPFS operation on it, but these objects may get garbage-collected fairly regularly. To prevent that garbage collection, simply pin the hash you care about. Objects added through `ipfs add` are pinned recursively by default. +Pinning is a very important concept in IPFS. IPFS semantics try to make it feel like every single object is local — there is no "retrieve this file for me from a remote server", just `ipfs cat` or `ipfs get`, which act the same way no matter where the actual object is located. + +While this is nice, sometimes you want to be able to control what you keep around. **Pinning** is the mechanism that allows you to tell IPFS to always keep a given object somewhere — the default being your local node, though this can be different if you use a [third-party remote pinning service](#using-a-pinning-service). IPFS has a fairly aggressive caching mechanism that will keep an object local for a short time after you perform any IPFS operation on it, but these objects may get garbage-collected regularly. To prevent that garbage collection, simply pin the CID you care about, or add it to [MFS](/concepts/file-systems/#mutable-file-system-mfs). Objects added through `ipfs add` are pinned recursively by default. Things in MFS are not pinned by default, but are protected from garbage-collection, so one can think about MFS as a mechanism for implicit pinning. + +Let's look at this example to explore pinning to your local IPFS node in a bit more depth: ``` echo "ipfs rocks" > foo @@ -19,11 +23,11 @@ ipfs pin ls --type=all ## Three kinds of pins -As you may have noticed, the first `ipfs pin rm` command didn't work — it should have warned you that the given hash was "pinned recursively". There are three types of pins in the IPFS world: +As you may have noticed in the example above, the first `ipfs pin rm` command didn't work — it should have warned you that the given hash was _pinned recursively_. What does this mean? There are three types of pins in the IPFS world: -- Direct pins, which pin just a single block, and no others in relation to it -- Recursive pins, which pin a given block and all of its children -- Indirect pins, which are the result of a given block's parent being pinned recursively +- **Direct pins**, which pin just a single block and no others in relation to it. +- **Recursive pins**, which pin a given block and all of its children. +- **Indirect pins**, which are the result of a given block's parent being pinned recursively. A pinned object cannot be garbage-collected — try this for proof: @@ -40,3 +44,17 @@ ipfs pin rm -r ipfs repo gc ipfs cat ``` + +## Local versus remote pinning + +All the information above assumes that you're pinning items locally — that is, to your local IPFS node. That's the default behavior for IPFS, but it's also possible to pin your files to a _remote pinning service_. These third-party services give you the opportunity to pin files not to your own local node, but to nodes that they operate. You don't need to worry about your own node's available disk space or uptime. + +While you can use a remote pinning service's own GUI, CLI, or other dev tools to manage IPFS files pinned to their service, you can also work directly with pinning services using your local IPFS installation — meaning that you don't need to learn a pinning service's unique API or other tooling. + +- The [IPFS Pinning Service API](https://ipfs.github.io/pinning-services-api-spec/) offers a specification that enables developers to integrate any pinning service that supports the spec, or create their own. Thanks to the OpenAPI spec format, both clients and servers can be [generated](https://github.com/ipfs/pinning-services-api-spec#code-generation) from the YAML spec file. + +- If you use go-ipfs 0.8+ from the command line, you have access to `ipfs pin remote` commands acting as a client for interacting with pinning service APIs. Add your favorite pinning service(s), pin CIDs under human-readable names, get pin statuses, and more, straight from the CLI. [Learn how →](/how-to/work-with-pinning-services/) + + diff --git a/docs/how-to/work-with-pinning-services.md b/docs/how-to/work-with-pinning-services.md new file mode 100644 index 000000000..4b88bfddc --- /dev/null +++ b/docs/how-to/work-with-pinning-services.md @@ -0,0 +1,128 @@ +--- +title: Work with pinning services +description: Learn how to use or create remote pinning services with IPFS, the InterPlanetary File System. +--- + +# Work with remote pinning services + +Depending on how you use IPFS, you might find it helpful to use a **remote pinning service** instead of, or in addition to, pinning files on your local IPFS node. Whether it happens remotely or locally, **pinning** an item in IPFS identifies it as something you always wish to keep available, exempting it from the routine _garbage collection_ that IPFS does on infrequently-used items in order to efficiently manage storage space. [Learn more about local pinning →](/how-to/pin-files) + +If you've got just one local IPFS node that's always running, local pinning may be all you need to ensure your important items are persisted and never garbage-collected. However, using a remote pinning service — or creating your own — might be useful to you if: + +- Your local node isn't always online, but you need items to be consistently available. +- You'd like to keep a persistent backup of your local node's files somewhere else. +- You don't have all the disk space you need on your local node. +- You run more than one IPFS node, and would like to use one of them as a "personal pinning service" as your preferred location for permanent storage. + +There are a number of commercial pinning services that make it easy for you to purchase pinning capacity for your important files, some of which include Pinata, Temporal, Infura, and others. Each of these third-party services has its own unique interface for pinning files and managing those pins; this could include a GUI, an API, CLI commands, or other tooling. + +However, you don't need to learn new commands or tools if your pinning service of choice supports the vendor-agnostic [IPFS Pinning Service API](https://ipfs.github.io/pinning-services-api-spec/) specification. Those services are supported within IPFS itself through the command line: `ipfs pin remote --help`. + +As of January 2021, [Pinata](https://pinata.cloud/) supports the [IPFS Pinning Service API endpoint](https://pinata.cloud/documentation#PinningServicesAPI), with more pinning services on the way! [Learn how to create your own →](#create-your-own-pinning-service) + +## Use an existing pinning service + +To add and use a remote pinning service directly in IPFS, you'll first need to have an account with that service. Once you've got an account, follow these steps to add and use it: + + + +### IPFS CLI + +Command-line users benefit from `ipfs pin remote` commands, which simplify remote pinning operations. The built-in pinning service API client executes all the necessary remote calls under the hood. + +#### Adding a new pinning service + +To add a new pinning service, use the following command: + +```console +$ ipfs pin remote service add nickname https://my-pin-service.example.com/api-endpoint myAccessToken +``` + +- `nickname` is a unique name for this particular instantiation of a pinning service. This can be helpful if, for example, you want to add two accounts from the same service. +- `https://my-pin-service.example.com/api-endpoint` is the endpoint supplied to you by the pinning service. Check the service's documentation for more info. +- `myAccessToken` is the unique secret token provided to you by the pinning service. Check the service's documentation for more info. + +#### Using a pinning service + +Here are a few CLI commands to get you started. In all examples, replace `nickname` with the unique name you gave the pinning service when you added it. + +To pin a CID under under a human-readable name: + +```console +$ ipfs pin remote add --service=nickname --name=war-and-peace.txt bafybeib32tuqzs2wrc52rdt56cz73sqe3qu2deqdudssspnu4gbezmhig4 +``` + +To list successful pins: + +```console +$ ipfs pin remote ls --service=nickname +``` + +To list all "pending" pins: + +```console +$ ipfs pin remote ls --service=nickname --status=queued,pinning,failed +``` + +For more commands and general help: + +```console +$ ipfs pin remote --help +``` + +## Create your own pinning service + +Obviously you aren't limited to a static list of pre-approved services. Any remote pinning service compatible with the [IPFS Pinning Service API](https://ipfs.github.io/pinning-services-api-spec) can be added as a custom pinning service — which also means that you can create your own! This might be useful in circumstances like: + +- Designating one of your own IPFS nodes to be a _personal pinning service_ as a preferred location for permanent storage. +- Running a private pinning service for your friends or company. +- Starting your own commercial pinning service. + +As noted above, your service must use the [IPFS Pinning Service API](https://ipfs.github.io/pinning-services-api-spec) in order to be interoperable with client behind `ipfs pin remote` commands. + + +::: tip +If you're interested in creating your own pinning service for your own personal or shared use, you can [generate client and server from the OpenAPI spec](https://github.com/ipfs/pinning-services-api-spec#code-generation), reducing the development time. + +You may also wish to read continuing details on how the API is evolving in the [Pinning Service API Spec GitHub repo](https://github.com/ipfs/pinning-services-api-spec), and be part of the discussion on its further development! +::: + + +