Boost: Testing: Booster-bitswap testing

[LexLuthr]

Overview

booster-bitswap is ready for alpha testing! 🚀
booster-bitswap is a binary that runs alongside the boostd process, to serve retrievals over the Bitswap protocol. The new release of boost provides a number of tools for managing a production grade Bitswap service for a Storage Provider's content.
Note that there is currently no payment method mechanism in booster-bitswap; this endpoint is intended to serve free content.

Why enable retrievals via Bitswap?

Bitswap retrieval introduces interoperability between IPFS and Filecoin, as it enables clients to retrieve Filecoin data over IPFS. This expands the reach of the Filecoin network considerably, increasing the value proposition for users to store data on the Filecoin network. This benefits the whole community, including SPs. Users will be able to access data directly via IPFS, as well as benefit from retrieval markets (e.g. Saturn) and compute over data projects (e.g. Bacalhau).
This is the first step to enable paid retrievals from Saturn etwork and IPFS network directly to the Filecoin SPs.

booster-bitswap modes

There are two primary modes for exposing booster-bitswap to the internet:

1. In private mode the booster-bitswap peer ID is not publicly accessible to the internet. Instead, public Bitswap traffic goes to boostd itself, which then acts as a reverse proxy, forwarding that traffic on to booster-bitswap. This is similar to the way one might configure Nginx as a reverse proxy for an otherwise private web server. private mode is simpler to setup but may produce greater load on boostd as a protocol proxy.

2. In public mode, the public internet firewall must be configured to forward traffic directly to the booster-bitswap instance. boostd is configured to announce the public address of booster-bitswap to the network indexer (the network indexer is the service that clients can query to discover where to retrieve content). public mode offers greater flexibility and performance -- you can even setup booster-bitswap to run over a separate internet connection from boostd. However, it requires additional configuration and changes to your overall network infrastructure.

Expectations

After successful initial testing, Boost team is looking to test the product with a wider range of storage providers. We are inviting all the interested storage providers from the community to participate in this testing before launching this feature. The objective of this testing is to test Bitswap retrieval's robustness and performance with different sized storage providers, deal sources and configuration.

Instructions

The test instructions and the number of tests to be performed are uniquely large and complex in this test plan. Here is a quick summary of what we want to test to help SPs get a basic understanding of what this testing entails.

1. Understand what is booster-bitswap and how it connects Filecoin network with IPFS network
2. Setup booster-bitswap binary and retrieve using booster-bitswap to test basic setup
3. Setup Kubo(Go implementation of IPFS) node
4. Understand steps to maintain Kubo node(s) throughout the testing
5. Configure booster-bitswap to Private mode and test retrieval from IPFS network using Kubo under various conditions
6. Configure booster-bitswap to Public mode and test retrieval from IPFS network using Kubo under various conditions
7. Check that booster-bitswap is now publishing IPFS retrieval address on the indexer

Initializing booster-bitswap

1. Build the booster-bitswap binary

cd boost
git pull
git checkout release/1.5.1-rc3
make booster-bitswap

2. Initialize booster-bitswap:

booster-bitswap init

3. Record the peer ID output by booster-bitswap init -- we will need this peer id later

4. Collect boostd's API Info:

export ENV_BOOST_API_INFO=`boostd auth api-info --perm=admin`

export BOOST_API_INFO=`echo $ENV_BOOST_API_INFO | awk '{split($0,a,"="); print a[2]}'`

5. Run booster-bitswap:

booster-bitswap run --api-boost=$BOOST_API_INFO

To run booster-bitswap with verbose logging set these environment variables:

GOLOG_LOG_LEVEL=remote-blockstore=debug,booster=debug,engine=debug,bitswap-server=debug booster-bitswap run --api-boost=$BOOST_API_INFO

Setup IPFS (Kubo) Node

As explained above, we are trying to test the data retrievals in IPFS directly from Filecoin SPs. Thus, from here on all of our tests will use a Kubo (Go implementation of IPFS) node for retrieving the data. Please setup your Kubo node using the below commands.

$ git clone https://github.com/ipfs/kubo.git
$ cd kubo
$ make install

Alternatively, you can run make build to build the go-ipfs binary (storing it in cmd/ipfs/ipfs) without installing it.

Over the course of this testing, you would need to reset your Kubo node to test retrievals. If a new Kubo node is not initialised for each test then retrieval command will use the local IPFS cache. You can re-initialize a Kubo node with the below instructions.

$ ipfs shutdown
$ rm -rf ~/.ipfs
$ ipfs init
$ ipfs config --json Bootstrap '[]'
$ ipfs daemon

Note: This will generate a new peer ID for your Kubo node. Make sure to use the new ID for the test.

In case a test requires you to run multiple Kubo instances, then you can do so by simply exporting a new and different repo path with export IPFS_PATH=/path/to/ipfsrepo in a new terminal and then initalize a new Kubo instance.

Retrieve from booster-bitswap using IPFS (Kubo)

$ ipfs init
$ ipfs config --json Bootstrap '[]'
$ ipfs daemon
$ ipfs swarm connect /ip4/<booster-bitswap IP>/tcp/<booster-bitswap port>/p2p/<booster-bitswap peer ID>
$ ipfs get <deal data root cid>

Where booster-bitswap peer ID is the peer id recorded when you ran booster-bitswap init and deal data root cit is the CID of a data CID known to be stored on your SP.

Configuring Boost To Serve Retrievals Publicly via Boost proxy (Private Mode)

To setup booster-bitswap in private mode, you'll need to make some changes to boostd's config file and restart boostd.

1. Edit ~/.boost/config.toml and set the peer ID for bitswap.

[DealMaking]
  BitswapPeerID ="{peer id for bosoter bitswap you recorded earlier}"

2. Restart boostd

3. Get boostd's multiaddr:

$ boostd net listen

4. Restart booster-bitswap with the --proxy parameter:

$ booster-bitswap run --api-boost=$BOOST_API_INFO --proxy={boostd multiaddress}

Configuring Boost To Serve Retrievals Publicly without proxy (Public Mode)

To setup booster-bitswap in public mode, you'll need to make some changes to boostd's config file and restart boostd.

1. Configure your public firewall to forward requests on that port to booster-bitswap.
2. Set the following values in your boostd config

 [Dealmaking]
  BitswapPeerID ="{peer id for booster bitswap you recorded earlier}"
  BitswapPublicAddresses = ["/ip4/{booster-bitswap public IP}/tcp/{booster-bitswap public port}"]
  BitswapPrivKeyFile = "{path to libp2p private key file for booster bitswap}"

The libp2p private key file for booster-bitswap is at ~/.booster-bitswap/libp2p.key

3. Restart boostd and booster-bitswap (with the --port argument if you've setup your firewall to forward to something other than the default booster-bitswap port of 8888)

Booster Bitswap Tests (To be run in both Proxied and Non-proxied configuration)

BadBits filtering

1. Host a json file containing BadBits info on a local or remote HTTP server. The file content should look like below:

[{"anchor":"7f26e7813526dfa529020b7611a62e8f2c496558f80a82bc24a95d4e4105125d"}]

2. Check some of your deals and copy the data root CID from them. Convert these data root CIDs to a SHA256() using this online tool. The input would be in info format / - bafykbzacebmouqhg33afbz2sb57ivdfxpw3si6w7q7or32k53axuhhvi3myms/
3. Copy the output and add it to BadBits json.
4. Restart booster-bitswap with custom BadBits list --badbits-lists <URL>
5. Setup a Kubo node (an IPFS node)

$ git clone https://github.com/ipfs/kubo.git
$ cd kubo
$ make install

6. Connect to the booster-bitswap server

$ ipfs swarm connect /ip4/<Booster-bitswap IP>/tcp/<Booster-bitswap Port>/p2p/<Booster-bitswap Peer ID>

7. Try to fetch a deal which we added to BadBits, reset the Kubo node and a deal which is not part of the BadBit list

Request Filtering - Functional

1. Create /retrievalconfig.json with below content

 {
	"AllowDenyList": {
		"Type": "denylist",
			"PeerIDs": []
	},
        "UnderMaintenance": false
}

2. Reset your Kubo node
3. Try to fetch a root CID of a deal based on the different condition described in the reporting metrics table.
   $ ipfs get <deal data root cid>
4. Make sure to re-initialize you Kubo node after each test to avoid using a local copy of the data.
5. Once all the conditions on the tests are covered, please check https://cid.contact to verify that your Bitswap address is being published. You can query one of the data CIDs from any deal that you are holding and result should show your Bitswap address as one of the options to retrieve the deal.

Report your results

• Please fill out this report template with your results, as well as any qualitative feedback that you may have. You need to check if Kubo is able to retrieve the data under various conditions listed below.
• If the expected condition does not match your result then please share your configuration details and the results.

Title	Expected Result	Result
Miner ID
Using booster-bitswap fetch	Successfully retrieved
Private mode - No filters	Successfully retrieved
Private mode - Custom Badbits list - Fetch data CID in badbits list	Failed to retrieve
Private mode - Custom Badbits list - Fetch data CID not in badbits list	Successfully retrieved
Private mode - Maintenance set to "True"	Failed to retrieve
Private mode - Maintenance is "False", "Type" is "denylist", "PeerIDs" is empty	Successfully retrieved
Private mode - Maintenance is "False", "Type" is "denylist", "PeerIDs" has Kubo Peer ID	Failed to retrieve
Private mode - Maintenance is "False", "Type" is "denylist", "PeerIDs" has Kubo Peer ID and retrieve using another Kubo instance	Successfully retrieved
Private mode - Maintenance is "False", "Type" is "allowlist", "PeerIDs" is empty	Failed to retrieve
Private mode - Maintenance is "False", "Type" is "allowlist", "PeerIDs" has Kubo Peer ID	Successfully retrieved
Private mode - Maintenance is "False", "Type" is "allowlist", "PeerIDs" has Kubo Peer ID and retrieve using another Kubo instance	Failed to retrieve
Public mode - No filters	Successfully retrieved
Public mode - Custom Badbits list - Fetch data CID in badbits list	Failed to retrieve
Public mode - Custom Badbits list - Fetch data CID not in badbits list	Successfully retrieved
Public mode - Maintenance set to "True"	Failed to retrieve
Public mode - Maintenance is "False", "Type" is "denylist", "PeerIDs" is empty	Successfully retrieved
Public mode - Maintenance is "False", "Type" is "denylist", "PeerIDs" has Kubo Peer ID	Failed to retrieve
Public mode - Maintenance is "False", "Type" is "denylist", "PeerIDs" has Kubo Peer ID and retrieve using another Kubo instance	Successfully retrieved
Public mode - Maintenance is "False", "Type" is "allowlist", "PeerIDs" is empty	Failed to retrieve
Public mode - Maintenance is "False", "Type" is "allowlist", "PeerIDs" has Kubo Peer ID	Successfully retrieved
Public mode - Maintenance is "False", "Type" is "allowlist", "PeerIDs" has Kubo Peer ID and retrieve using another Kubo instance	Failed to retrieve
Public mode - SP's Bitswap address is published on the Indexer	Published

Reminders

• Risk Mitigation
  • Please backup everything in the repositories before making any changes.
  • In addition to reporting issues you see in the comments below, please fill out a bug report using this template
• Questions
  • Start here
  • @LexLuthr and @brendalee will be monitoring #boost-help and #boost on Filecoin slack and Github