|
1 |
| -# js-ipfs-migrator |
| 1 | +# Migration tool for JS IPFS Repo |
| 2 | + |
| 3 | +[](http://ipn.io) |
| 4 | +[](http://ipfs.io/) |
| 5 | +[](http://webchat.freenode.net/?channels=%23ipfs) |
| 6 | +[](https://github.com/RichardLitt/standard-readme) |
| 7 | +[](https://github.com/feross/standard) |
| 8 | + |
| 9 | + |
| 10 | + |
| 11 | +> Migration framework for versioning of JS IPFS Repo |
| 12 | +
|
| 13 | +This package takes inspiration from similar tool used by Go-IPFS: [fs-repo-migrations](https://github.com/ipfs/fs-repo-migrations/) |
| 14 | + |
| 15 | +## Lead Maintainer |
| 16 | + |
| 17 | +??? |
| 18 | + |
| 19 | +## Table of Contents |
| 20 | + |
| 21 | +- [Background](#background) |
| 22 | +- [Install](#install) |
| 23 | + - [npm](#npm) |
| 24 | + - [Use in Node.js](#use-in-nodejs) |
| 25 | + - [Use in a browser with browserify, webpack or any other bundler](#use-in-a-browser-with-browserify-webpack-or-any-other-bundler) |
| 26 | + - [Use in a browser Using a script tag](#use-in-a-browser-using-a-script-tag) |
| 27 | +- [Usage](#usage) |
| 28 | + - [Writing migration](#writing-migration) |
| 29 | + - [Migrations matrix](#migrations-matrix) |
| 30 | +- [API](#api) |
| 31 | +- [CLI](#cli) |
| 32 | +- [Contribute](#contribute) |
| 33 | +- [License](#license) |
| 34 | + |
| 35 | +## Background |
| 36 | + |
| 37 | +As JS-IPFS evolves and new technologies, algorithms and data structures are being incorporated it is necessary to |
| 38 | +enable users easy transition between versions. Different versions of JS-IPFS may expect different structure or content |
| 39 | +of the IPFS repo (see: [IPFS repo spec](https://github.com/ipfs/specs/tree/master/repo), [JS implementation](https://github.com/ipfs/js-ipfs-repo) ). |
| 40 | +For that reason IPFS repo is versioned and this package provides framework to create migrations which transits |
| 41 | +one version of IPFS repo into next/previous one. |
| 42 | + |
| 43 | +This framework provides: |
| 44 | + * Handles locking/unlocking of repository |
| 45 | + * Define migrations API |
| 46 | + * Executes and reports migrations in both direction: forward and backword |
| 47 | + * Simplify creation of new migrations |
| 48 | + |
| 49 | +## Install |
| 50 | + |
| 51 | +### npm |
| 52 | + |
| 53 | +```sh |
| 54 | +> npm install ipfs-repo-migrations |
| 55 | +``` |
| 56 | + |
| 57 | +### Use in Node.js |
| 58 | + |
| 59 | +```js |
| 60 | +const migrations = require('ipfs-repo-migrations') |
| 61 | +``` |
| 62 | + |
| 63 | +### Use in a browser with browserify, webpack or any other bundler |
| 64 | + |
| 65 | +```js |
| 66 | +const migrations = require('ipfs-repo-migrations') |
| 67 | +``` |
| 68 | + |
| 69 | +## Usage |
| 70 | + |
| 71 | +Example: |
| 72 | + |
| 73 | +```js |
| 74 | +const migrations = require('ipfs-repo-migrations') |
| 75 | +const getVersion = require('ipfs-repo-migrations/repo/version') |
| 76 | + |
| 77 | +const repoPath = 'some/repo/path' |
| 78 | +const repoVersion = await getVersion(repoPath) |
| 79 | + |
| 80 | +if(repoVersion < migrations.getLatestMigrationVersion()){ |
| 81 | + // Old repo! Lets migrate to latest version! |
| 82 | + await migrations.migrate(repoPath) |
| 83 | +} |
| 84 | +``` |
| 85 | + |
| 86 | +### Writing migration |
| 87 | + |
| 88 | +#### Architecture of migrations |
| 89 | + |
| 90 | +All migrations are placed in `/migrations` folder. Each folder there represents one migration that behaves as stand-alone |
| 91 | +package that has its own `package.json` file that states migration's dependencies, has its own tests and follows migration |
| 92 | +API. |
| 93 | + |
| 94 | +All migrations are collected in `/migrations/index.js`, which should not be edited manually is it is regenerated on |
| 95 | +every run of `jsipfs-migrations add` (or the manual changes should follow same style of modifications). |
| 96 | +The order of migrations is important and the migrations have to be sorted in the growing order. |
| 97 | + |
| 98 | +Each migration has to follow this API. It has to export object in its `index.js` that has following properties: |
| 99 | + |
| 100 | + * `version` (int) - Number that represents the version into which will be the repo migrated to (eq. `migration-8` will move the repo into version 8). |
| 101 | + * `description` (string) - Brief description of what the migrations does. |
| 102 | + * `reversible` (bool) - Specify if it is possible to revert this migration. |
| 103 | + * `migrate` (function) - Function that on execution will perform the migration, see signature of this function bellow. |
| 104 | + * `revert` (function) - If `reversible == True` then this function will be used to revert the migration to previous version. |
| 105 | + |
| 106 | +##### `migrate(repoPath, isBrowser)` |
| 107 | + |
| 108 | +_Do not confuse this function with the `require('ipfs-repo-migrations').migrate()` function that drives the whole migration process!_ |
| 109 | + |
| 110 | +Arguments: |
| 111 | + * `repoPath` (string) - absolute path to the root of the repo |
| 112 | + * `isBrowser` (bool) - indicates if the migration is run in browser environment in oppose to NodeJS |
| 113 | + |
| 114 | +##### `revert(repoPath, isBrowser)` |
| 115 | + |
| 116 | +_Do not confuse this function with the `require('ipfs-repo-migrations').revert()` function that drives the whole backward migration process!_ |
| 117 | + |
| 118 | +Arguments: |
| 119 | + * `repoPath` (string) - path to the root of the repo |
| 120 | + * `isBrowser` (bool) - indicates if the migration is run in browser environment in oppose to NodeJS |
| 121 | + |
| 122 | +#### Browser vs. NodeJS environments |
| 123 | + |
| 124 | +Migration might need to distinguish in what environment it runs (browser vs. NodeJS), for this reason there is the argument |
| 125 | +`isBrowser` passed to migrations functions. But with simple migrations it should not be necessary to distinguish between |
| 126 | +these environments as datastore implementation will handle the main differences. |
| 127 | + |
| 128 | +There are currently two main datastore implementations: |
| 129 | + 1. [`datastore-fs`](https://github.com/ipfs/js-datastore-fs) that is backed by file system and is used mainly in NodeJS environment |
| 130 | + 2. [`datastore-level`](https://github.com/ipfs/js-datastore-level) that is backed by LevelDB and is used mainly in browser environment |
| 131 | + |
| 132 | + Both implementations share same API and hence are interchangeable. |
| 133 | + |
| 134 | + When the migration is run in browser environment the `datastore-fs` is automatically replaced with `datastore-level` even |
| 135 | + when it is directly imported (`require('datastore-fs')` will return `datastore-level` in browser). Because of this mechanism |
| 136 | + with simple migrations you should not worry about difference between `datastore-fs` and `datastore-level`. |
| 137 | + |
| 138 | +#### Guidelines |
| 139 | + |
| 140 | +The recommended way to write a new migration is to first bootstrap an dummy migration using the CLI: |
| 141 | + |
| 142 | +```sh |
| 143 | +> jsipfs-migrations add |
| 144 | +``` |
| 145 | + |
| 146 | +Afterwards new folder is created with bootstrapped migration. You can then simply fill in the required fields and |
| 147 | +write the rest of migration! |
| 148 | + |
| 149 | +The `node_modules` of the migration should be committed to the repo to ensure that the dependencies are resolved even in |
| 150 | +far future, when the package might be removed from registry. |
| 151 | + |
| 152 | +#### Tests |
| 153 | + |
| 154 | +**TODO** |
| 155 | + |
| 156 | +#### Empty migrations |
| 157 | + |
| 158 | +For inter-operable reasons with Go-IPFS it might be necessary just to bump a version of repo without any actual |
| 159 | +modification as there might not be any changes needed in JS implementation. For that you can create "empty migration". |
| 160 | + |
| 161 | +The easiest way is to use the CLI for that: |
| 162 | + |
| 163 | +```sh |
| 164 | +> jsipfs-migrations add --empty |
| 165 | +``` |
| 166 | + |
| 167 | +This will create empty migration with next version in line. |
| 168 | + |
| 169 | +### Migrations matrix |
| 170 | + |
| 171 | +**TODO** |
| 172 | + |
| 173 | +## API |
| 174 | + |
| 175 | +### `migrate(path[, toVersion[, progressCb[, isDryRun]]]) -> Promise<void>` |
| 176 | + |
| 177 | +Executes forward migration to specific version or if not specified to the latest version. |
| 178 | + |
| 179 | +**Arguments:** |
| 180 | + |
| 181 | + * `path` (string, mandatory) - path to the repo to be migrated |
| 182 | + * `toVersion` (int, optional) - version to which the repo should be migrated to. If left out the version of latest migration is used. |
| 183 | + * `progressCb` (function, optional) - callback that is called after finishing execution of each migration to report progress. |
| 184 | + * `isDryRun` (bool, optional) - flag that indicates if it is a dry run that should imitate running migration without actually any change. |
| 185 | + |
| 186 | +#### `progressCb(migration, counter, totalMigrations)` |
| 187 | + |
| 188 | +Signature of the progress callback. |
| 189 | + |
| 190 | +**Arguments:** |
| 191 | + * `migration` (object) - object of migration that just successfully finished running. See [Architecture of migrations](#architecture-of-migrations) for details. |
| 192 | + * `counter` (int) - current number of migration in the planned migrations streak. |
| 193 | + * `totalMigrations` (int) - total count of migrations that are planned to be run. |
| 194 | + |
| 195 | +### `revert(path, toVersion[, progressCb[, isDryRun]]) -> Promise<void>` |
| 196 | + |
| 197 | +Executes backward migration to specific version. |
| 198 | + |
| 199 | +**Arguments:** |
| 200 | + |
| 201 | + * `path` (string, mandatory) - path to the repo to be reverted |
| 202 | + * `toVersion` (int, mandatory) - version to which the repo should be reverted to. |
| 203 | + * `progressCb` (function, optional) - callback that is called after finishing execution of each migration to report progress. |
| 204 | + * `isDryRun` (bool, optional) - flag that indicates if it is a dry run that should imitate running migration without actually any change. |
| 205 | + |
| 206 | +### `getLatestMigrationVersion() -> int` |
| 207 | + |
| 208 | +Return the version of latest migration. |
| 209 | + |
| 210 | +## CLI |
| 211 | + |
| 212 | +The package comes also with CLI that is exposed as NodeJS binary with name `jsipfs-repo-migrations`. |
| 213 | +It has several commands: |
| 214 | + |
| 215 | + * `migrate` - performs forward migration to specific or latest version. |
| 216 | + * `revert` - performs backward migration to specific version. |
| 217 | + * `status` - check repo for migrations that should be run. |
| 218 | + * `add` - bootstraps new migration. |
| 219 | + |
| 220 | +For further details see the `--help` pages. |
| 221 | + |
| 222 | +## Contribute |
| 223 | + |
| 224 | +There are some ways you can make this module better: |
| 225 | + |
| 226 | +- Consult our [open issues](https://github.com/ipfs/js-ipfs-repo/issues) and take on one of them |
| 227 | +- Help our tests reach 100% coverage! |
| 228 | + |
| 229 | +This repository falls under the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md). |
| 230 | + |
| 231 | +[](https://github.com/ipfs/community/blob/master/contributing.md) |
| 232 | + |
| 233 | +## License |
| 234 | + |
| 235 | +[MIT](LICENSE) |
0 commit comments