feat: E2E test suite with puppeteer+ipfsd-ctl (#1353)

This adds end to end tests against real HTTP API.
Uses jest-puppeteer and ipfsd-ctl for orchestration.

By default it runs headless chromium against go-ipfs,
but can be made to run test matrix against js-ipfs with
E2E_IPFSD_TYPE=js or arbitrary HTTP API via E2E_API_URL

Author: lidel

.circleci/config.yml

@@ -16,7 +16,9 @@ jobs:
       - run:
           command: npm run build
       - run:
-          command: npm run test:e2e:ci
+          command: E2E_IPFSD_TYPE=go npm run test:e2e
+      - run:
+          command: E2E_IPFSD_TYPE=js npm run test:e2e
       - run:
           command: npm run bundlesize
       - persist_to_workspace:

README.md

@@ -105,29 +105,90 @@ The WebUI uses Jest to run the isolated unit tests. Unit test files are located
 
 ## End-to-end tests
 
-The end-to-end tests (e2e) test the full app in a headless Chromium browser. They require an http server to be running to serve the app.
+The end-to-end tests (e2e) test the full app in a headless Chromium browser. They spawn real IPFS node for HTTP API and a static HTTP server to serve the app.
+The purpose of those tests is not being comprehensible, but act as a quick regression and integration suite.
 
-In dev, run `npm start` in another shell before starting the tests
+Make sure `npm run build` is run before starting E2E tests:
 
+```console
+$ npm run build
+$ npm run test:e2e # end-to-end smoke tests (fast, headless, use go-ipfs)
 ```
-# Run the end-to-end tests
-> npm run test:e2e
+
+### Customizing E2E Tests
+
+Default behavior can be tweaked via env variables below.
+
+#### `E2E_IPFSD_TYPE`
+
+Variable named `E2E_IPFSD_TYPE` defines which IPFS backend should be used for end-to-end tests.
+
+CI setup of ipfs-webui repo runs tests against both JS and GO implementations:
+
+```console
+$ E2E_IPFSD_TYPE=go npm run test:e2e # 'go' is the default if missing
+$ E2E_IPFSD_TYPE=js npm run test:e2e
+```
+
+It is possible to test against arbitrary versions by tweaking `ipfs` (js-ipfs)
+ and `go-ipfs-dep` (go-ipfs) in `devDependencies` section of `package.json` and applying the change via `npm i`
+
+#### `E2E_API_URL`
+
+Instead of spawning a disposable node and repo for tests, one can point the E2E test suite at arbitrary HTTP API running on localhost:
+
+```console
+$ E2E_API_URL=http://127.0.0.1:5001 npm run test:e2e
 ```
 
-By default the test run headless, so you won't see the the browser. To debug test errors, it can be helpful to see the robot clicking around the site. To disable headless mode and see the browser, set the environment variable `DEBUG=true`
+**Caveat 1:** HTTP API used in tests needs to run on the local machine for Peers screen to pass (they test manual swarm connect to ephemeral `/ip4/120.0.0.1/..` multiaddr)
 
+**Caveat 2:** CORS requests from `http://localhost:3001` (static server hosting dev version of webui) need to be added to `Access-Control-Allow-Origin` whitelist array in node's config:
+
+```json
+"API": {
+  "HTTPHeaders": {
+    "Access-Control-Allow-Origin": [
+      "http://localhost:3001"
+    ]
+  }
+}
 ```
-# See the end-to-end tests in a browser
-> DEBUG=true npm run test:e2e
+
+Can be done ad-hoc via command line:
+
+```console
+$ ipfs config --json API.HTTPHeaders.Access-Control-Allow-Origin '["http://localhost:3001"]'
+$ ipfs config --json API.HTTPHeaders.Access-Control-Allow-Methods '["PUT", "GET", "POST"]'
 ```
 
-In a **continuous integration** environment we lint the code, run the unit tests, build the app, start an http server and run the unit e2e tests.
+### Debugging E2E tests
+
+#### Show the browser
+
+By default, the test run headless, so you won't see the browser. To debug test errors, it can be helpful to see the robot clicking around the site.
+To disable headless mode and see the browser, set the environment variable `DEBUG=true`:
+
+```console
+$ DEBUG=true npm run test:e2e # e2e in slowed down mode in a browser window
+```
+
+#### Breakpoints
+
+It is possible to set a "breakpoint" via `await jestPuppeteer.debug()` to stop tests at a specific line:
+
+```js
+jest.setTimeout(600000) // increase test suite timeout
+await jestPuppeteer.debug() // puppeteer will pause here
+```
+
+In a **continuous integration** environment we lint the code, run the unit tests, build the app, start an http server and run the unit e2e tests:
 
 ```sh
 > npm run lint
 > npm test
 > npm run build
-> npm run test:ci:e2e
+> npm run test:e2e
 ```
 
 ## Coverage