Skip to content

Commit

Permalink
ADR: Module Configuration (#60)
Browse files Browse the repository at this point in the history
* chore: enable Prettier for Markdown files

* build: disable git's autocrlf on Windows CI

Checkout the code as-is with LF line endings to avoid Prettier errors.

* docs: template for Architecture Decision Records

* docs: add Module Configuration ADR
  • Loading branch information
bajtos authored Jul 27, 2022
1 parent dcd883b commit 62b5de8
Show file tree
Hide file tree
Showing 11 changed files with 269 additions and 59 deletions.
4 changes: 4 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,10 @@ jobs:
os: [macos-latest, ubuntu-latest, windows-latest]

steps:
- name: Fix CRLF handling on Windows
if: matrix.os == 'windows-latest'
run: git config --global core.autocrlf false

- name: Check out Git repository
uses: actions/checkout@v2

Expand Down
9 changes: 9 additions & 0 deletions .prettierignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# We are using https://standardjs.com coding style. It is unfortunately incompatible with Prettier;
# see https://github.com/standard/standard/issues/996 and the linked issues.
# Standard does not handle Markdown files though, so we want to use Prettier formatting.
# The goal of the configuration below is to configure Prettier to lint only Markdown files.
**/*
!**/*.md

# Let's keep LICENSE.md in the same formatting as we use in other PL repositories
LICENSE.md
2 changes: 2 additions & 0 deletions .prettierrc.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
printWidth: 80
proseWrap: always
5 changes: 2 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# Filecoin Station



[![](https://img.shields.io/badge/made%20by-Protocol%20Labs-blue.svg?style=flat-square)](https://protocol.ai/)
[![](https://img.shields.io/badge/project-Filecoin-blue.svg?style=flat-square)](https://filecoin.io/)
[![ci](https://github.com/filecoin-project/filecoin-station/actions/workflows/ci.yml/badge.svg)](https://github.com/filecoin-project/filecoin-station/actions/workflows/ci.yml)
Expand All @@ -12,7 +10,8 @@ A longer description goes here.

## Install

Release notes and all versions of Filecoin Station can be found on the [releases page](https://github.com/filecoin-project/filecoin-station/releases).
Release notes and all versions of Filecoin Station can be found on the
[releases page](https://github.com/filecoin-project/filecoin-station/releases).

Each release has packages for desktop platforms:

Expand Down
37 changes: 25 additions & 12 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,35 @@

## Reporting a Vulnerability

For reporting security vulnerabilities/bugs, please consult our Security Policy and Responsible Disclosure Program information at https://github.com/filecoin-project/community/blob/master/SECURITY.md. Security vulnerabilities should be reported via our [Vulnerability Reporting channels](https://github.com/filecoin-project/community/blob/master/SECURITY.md#vulnerability-reporting) and will be eligible for a [Bug Bounty](https://security.filecoin.io/bug-bounty/).

Please try to provide a clear description of any bugs reported, along with how to reproduce the bug if possible. More detailed bug reports (especially those with a PoC included) will help us move forward much faster. Additionally, please avoid reporting bugs that already have open issues. Take a moment to search the issue list of the related GitHub repositories before writing up a new report.
For reporting security vulnerabilities/bugs, please consult our Security Policy
and Responsible Disclosure Program information at
https://github.com/filecoin-project/community/blob/master/SECURITY.md. Security
vulnerabilities should be reported via our
[Vulnerability Reporting channels](https://github.com/filecoin-project/community/blob/master/SECURITY.md#vulnerability-reporting)
and will be eligible for a
[Bug Bounty](https://security.filecoin.io/bug-bounty/).

Please try to provide a clear description of any bugs reported, along with how
to reproduce the bug if possible. More detailed bug reports (especially those
with a PoC included) will help us move forward much faster. Additionally, please
avoid reporting bugs that already have open issues. Take a moment to search the
issue list of the related GitHub repositories before writing up a new report.

Here are some examples of bugs we would consider to be security vulnerabilities:

* If you can spend from a `multisig` wallet you do not control the keys for.
* If you can cause a miner to be slashed without them actually misbehaving.
* If you can maintain power without submitting windowed posts regularly.
* If you can craft a message that causes lotus nodes to panic.
* If you can cause your miner to win significantly more blocks than it should.
* If you can craft a message that causes a persistent fork in the network.
* If you can cause the total amount of Filecoin in the network to no longer be 2 billion.
- If you can spend from a `multisig` wallet you do not control the keys for.
- If you can cause a miner to be slashed without them actually misbehaving.
- If you can maintain power without submitting windowed posts regularly.
- If you can craft a message that causes lotus nodes to panic.
- If you can cause your miner to win significantly more blocks than it should.
- If you can craft a message that causes a persistent fork in the network.
- If you can cause the total amount of Filecoin in the network to no longer be 2
billion.

This is not an exhaustive list, but should provide some idea of what we consider as a security vulnerability, .
This is not an exhaustive list, but should provide some idea of what we consider
as a security vulnerability, .

## Reporting a non security bug

For non-security bugs, please simply file a GitHub [issue](https://github.com/filecoin-project/filecoin-station/issues/new).
For non-security bugs, please simply file a GitHub
[issue](https://github.com/filecoin-project/filecoin-station/issues/new).
28 changes: 16 additions & 12 deletions docs/DEVELOPING.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,31 +41,35 @@ Repositories:

1. Install the latest LTS version of Node.js
2. Clone this repository
3. Run `npm install` to install all dependencies, including electron components and Saturn L2 Node.
3. Run `npm install` to install all dependencies, including electron components
and Saturn L2 Node.

### Making changes

- Run `npm start` to open the Electron app and load the WebUI via the Vite DEV server. Changes made
inside `renderer` files will be immediately applied in the running app.
- Run `npm start` to open the Electron app and load the WebUI via the Vite DEV
server. Changes made inside `renderer` files will be immediately applied in
the running app.

- Run `npm run lint` to check for coding style issues and type errors. Run `npm run lint:fix` to fix
formatting issues.
- Run `npm run lint` to check for coding style issues and type errors. Run
`npm run lint:fix` to fix formatting issues.

- Run `npm test` to run all automated tests (unit/integration/end-to-end).

### Building the app package

Run `npm run package` to create the app package for your local platform. Check the `dist` directory
for app artefacts, for example a DMG installer for macOS.
Run `npm run package` to create the app package for your local platform. Check
the `dist` directory for app artefacts, for example a DMG installer for macOS.

## CommonJS vs ES Modules

Electron does not support ESM for the backend code. We use CommonJS (`require`) in those files.
Electron does not support ESM for the backend code. We use CommonJS (`require`)
in those files.

The front-end is written in TypeScript; there is no such limitation. Therefore inside `renderer`, we
use ESM to get better developer experience and tooling support.
The front-end is written in TypeScript; there is no such limitation. Therefore
inside `renderer`, we use ESM to get better developer experience and tooling
support.

## Updating to a new major version of Electron/Chromium

After updating to a newer Chromium version, update the compilation target for `renderer` files in
[vite.config.js](./../vite.config.js).
After updating to a newer Chromium version, update the compilation target for
`renderer` files in [vite.config.js](./../vite.config.js).
81 changes: 51 additions & 30 deletions docs/RELEASE_PROCESS.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,12 @@
## Overview

Filecoin Station uses [electron-build/auto-update](https://www.electron.build/auto-update) and a modified flow from [Electron Builder Action](https://github.com/samuelmeuli/action-electron-builder#electron-builder-action):
Filecoin Station uses
[electron-build/auto-update](https://www.electron.build/auto-update) and a
modified flow from
[Electron Builder Action](https://github.com/samuelmeuli/action-electron-builder#electron-builder-action):

- `electron-builder` will create `.exe`, `.dmg` and `.AppImage` packages and output them to `./dist`
- `electron-builder` will create `.exe`, `.dmg` and `.AppImage` packages and
output them to `./dist`
- ℹ️ the type of produced artifact depends on OS used for build
- ℹ️ locally built versions are neither signed nor notarized
- when running on CI (`.github/workflows/ci.yml`) our automation will
Expand All @@ -11,7 +15,8 @@ Filecoin Station uses [electron-build/auto-update](https://www.electron.build/au
- do the necessary signing and notarization (macOS)
- ℹ️ builds from forks are neither signed nor notarized
- attach created packages to GitHub Action's Summary page (valid for 90 days)
- detect build for a tag, and create release draft with created packages and `.yml` auto-update manifests
- detect build for a tag, and create release draft with created packages and
`.yml` auto-update manifests

## Making a new release

Expand All @@ -23,56 +28,72 @@ Filecoin Station uses [electron-build/auto-update](https://www.electron.build/au
4. Update the package version using `npm --no-git-tag-version version A.B.C`
5. Commit the changes, use `release: vA.B.C` as the commit message
6. Open a pull request, wait for CI to finish, and then land the PR.
7. Pull the latest `main` branch to your machine, create a new tag and push it to GitHub:

```
git checkout main
git pull
git tag -a -s vA.B.C
# Set the commit message to "vA.B.C"
git push origin vA.B.C
```
3. Wait for GitHub Actions CI to upload the binaries to a draft release [here](https://github.com/filecoin-project/filecoin-station/releases) (a new one will be created if you haven't drafted one).
4. Review and publish the release draft.
7. Pull the latest `main` branch to your machine, create a new tag and push it
to GitHub:

```
git checkout main
git pull
git tag -a -s vA.B.C
# Set the commit message to "vA.B.C"
git push origin vA.B.C
```

8. Wait for GitHub Actions CI to upload the binaries to a draft release
[here](https://github.com/filecoin-project/filecoin-station/releases) (a new
one will be created if you haven't drafted one).
9. Review and publish the release draft.
- Once a release is published, users will receive the app update
- The `latest.yml, latest-mac.yml, latest-linux.yml` files attached to [/releases](https://github.com/filecoin-project/filecoin-station/releases) are used by the desktop app to determine when an app update is available.
5. Done!
- The `latest.yml, latest-mac.yml, latest-linux.yml` files attached to
[/releases](https://github.com/filecoin-project/filecoin-station/releases)
are used by the desktop app to determine when an app update is available.
10. Done!

### Old instructions that require admin access to the repo

1. Update `electron*` dependencies, if necessary.
2. Commit the changes.
3. Update the version using `npm version [major|minor|patch]` (it will create a new tag `vA.B.C`, note it down)
4. Publish local changes and the tag to the GitHub repo: `git push && git push origin vA.B.C`.
5. Wait for GitHub Actions CI to upload the binaries to a draft release [here](https://github.com/filecoin-project/filecoin-station/releases) (a new one will be created if you haven't drafted one).
3. Update the version using `npm version [major|minor|patch]` (it will create a
new tag `vA.B.C`, note it down)
4. Publish local changes and the tag to the GitHub repo:
`git push && git push origin vA.B.C`.
5. Wait for GitHub Actions CI to upload the binaries to a draft release
[here](https://github.com/filecoin-project/filecoin-station/releases) (a new
one will be created if you haven't drafted one).
6. Review and publish the release draft.
- Once a release is published, users will receive the app update
- The `latest.yml, latest-mac.yml, latest-linux.yml` files attached to [/releases](https://github.com/filecoin-project/filecoin-station/releases) are used by the desktop app to determine when an app update is available.
- The `latest.yml, latest-mac.yml, latest-linux.yml` files attached to
[/releases](https://github.com/filecoin-project/filecoin-station/releases)
are used by the desktop app to determine when an app update is available.
7. Done!

### Staged rollouts

To do a [staged rollout](https://www.electron.build/auto-update#staged-rollouts)

1. download `.yml` manifest files
2. append `stagingPercentage: 10` (% of userbase)
3. remove old `.yml` and re-upload patched ones

### Disabling rollout of a broken release

If your release is broken only on specific OS, set `stagingPercentage: 0` in respective `.yml`
If your release is broken only on specific OS, set `stagingPercentage: 0` in
respective `.yml`

This will make clients ignore the broken release. If you do this, ix has to be released under a new version.
This will make clients ignore the broken release. If you do this, ix has to be
released under a new version.

### Apple Signing and Notarization

CI builds from main and other local branches
is signed and notarized using Protocol Labs Developer account.
CI builds from main and other local branches is signed and notarized using
Protocol Labs Developer account.

Signing happens when env variables with certificate info is present:
- `MAC_CERTS` Certificate chain downloaded from apple website, as Base64 (`base64 -i certs.p12 > certs.txt`)
- `MAC_CERTS_PASSWORD` Certificate chain password

Notarization is possible only if signing was successful.
It uses a separate set of credentials: `APPLE_ID` + `APPLE_ID_PASS`
(not a global account password, but app-specific one)
- `MAC_CERTS` Certificate chain downloaded from apple website, as Base64
(`base64 -i certs.p12 > certs.txt`)
- `MAC_CERTS_PASSWORD` Certificate chain password

Notarization is possible only if signing was successful. It uses a separate set
of credentials: `APPLE_ID` + `APPLE_ID_PASS` (not a global account password, but
app-specific one)
100 changes: 100 additions & 0 deletions docs/architecture-decision-records/2022-07-module-configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# Station Module Configuration

- Status: ACCEPTED

## Context

Station modules want to allow Station users to tweak certain aspects of their
runtime behaviour.

For example, Saturn L2 Node lets users control how much disk space they are
willing to offer for the cache.

This configuration is module-specific. Each module has different options.

The Station communicates the configuration to the Module via environment
variables passed to the module backend process.

## Options Considered

### (1) Each module provides config WebUI

The initial idea was to let each Station Module implement its configuration
screen as part of the module WebUI and persist the configuration in a file. The
Station would provide the directory where to keep the config file.

We quickly realised this was not optimal. The Saturn module is written in Go and
expects users to provide all configuration options via environment variables in
the spirit of [The Twelve-Factor App](https://12factor.net). Implementing a
configuration layer with GUI requires a lot of work inside Saturn L2 that's not
adding much value to Saturn:

1. A REST API to read & write the configuration from the WebUI.
2. Code to persist the configuration in a file. Also, handle possible conflicts
between the config file and environment variables.
3. The config editor GUI.
4. Support for changing configuration options at runtime with zero downtime.

### (2) Module provides config schema; Station provides configuration WebUI

A different approach is to mimic what VisualStudio Code offers to extensions:

- Modules provide schema describing their configuration, and
- Station builds a configuration GUI based on the module-provided schema.

This creates a value proposition offered by the Station to future Module
authors: modules don't have to implement a user-friendly configuration GUI; the
Station will provide it for free.

There are some caveats, though. Supporting complex configuration formats is a
lot of work. VS Code has been evolving the configuration editor for years, and
there are still edge cases where the user has to edit the configuration in the
JSON file.

Some edge cases to consider:

- A config option is an array of primitive values (an array of strings, an array
of numbers)
- A config option is an enum - there is a set of valid values to pick from.
- Combine 1+2, and we have an array of enum values.
- A config option is an object (a key-value map). In some cases, sub-options are
well defined - we know what the valid keys are and what value types each key
accepts. In other cases, this can be entirely dynamic.

## Decision

We have decided to implement the second option, where the Station provides the
configuration GUI for Modules.

We will start by supporting:

- string keys mapped directly to environment variables
- string or integer values
- optional and required config options
- string descriptions of config options

We will implement support for more complex configurations later.

To keep our config schema easy to extend and forward-compatible, it should be a
superset of
[JSON Schema](https://json-schema.org/understanding-json-schema/reference/index.html).

## Consequences

The configuration WebUI will be consistent across all modules and follow the
Station visual style.

Developers in the Filecoin ecosystem have another incentive to deploy their app
to Filecoin Station - we will take care of user-friendly configuration
management.

In the future, we may need to spend some development cycles adding support for
more complex configuration schemes.

Config schema evolution is another area to explore after the initial release.
How can a module rename a config key, add a new config option, or update a
default value?

## Links & References

- [VS Code reference for extensions contributing configuration](https://code.visualstudio.com/api/references/contribution-points#contributes.configuration)
Loading

0 comments on commit 62b5de8

Please sign in to comment.