Skip to content
This repository was archived by the owner on Oct 1, 2021. It is now read-only.

Commit 5a537ce

Browse files
AuHauAuHau
committed
Writing proper README
License: MIT Signed-off-by: Adam Uhlir <[email protected]>
1 parent 6a1a379 commit 5a537ce

File tree

4 files changed

+292
-53
lines changed

4 files changed

+292
-53
lines changed

‎README.md

+235-1
Original file line numberDiff line numberDiff line change
@@ -1 +1,235 @@
1-
# js-ipfs-migrator
1+
# Migration tool for JS IPFS Repo
2+
3+
[![](https://img.shields.io/badge/made%20by-Protocol%20Labs-blue.svg?style=flat-square)](http://ipn.io)
4+
[![](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](http://ipfs.io/)
5+
[![](https://img.shields.io/badge/freenode-%23ipfs-blue.svg?style=flat-square)](http://webchat.freenode.net/?channels=%23ipfs)
6+
[![standard-readme compliant](https://img.shields.io/badge/standard--readme-OK-green.svg?style=flat-square)](https://github.com/RichardLitt/standard-readme)
7+
[![js-standard-style](https://img.shields.io/badge/code%20style-standard-brightgreen.svg?style=flat-square)](https://github.com/feross/standard)
8+
![](https://img.shields.io/badge/npm-%3E%3D3.0.0-orange.svg?style=flat-square)
9+
![](https://img.shields.io/badge/Node.js-%3E%3D10.0.0-orange.svg?style=flat-square)
10+
11+
> Migration framework for versioning of JS IPFS Repo
12+
13+
This package takes inspiration from similar tool used by Go-IPFS: [fs-repo-migrations](https://github.com/ipfs/fs-repo-migrations/)
14+
15+
## Lead Maintainer
16+
17+
???
18+
19+
## Table of Contents
20+
21+
- [Background](#background)
22+
- [Install](#install)
23+
- [npm](#npm)
24+
- [Use in Node.js](#use-in-nodejs)
25+
- [Use in a browser with browserify, webpack or any other bundler](#use-in-a-browser-with-browserify-webpack-or-any-other-bundler)
26+
- [Use in a browser Using a script tag](#use-in-a-browser-using-a-script-tag)
27+
- [Usage](#usage)
28+
- [Writing migration](#writing-migration)
29+
- [Migrations matrix](#migrations-matrix)
30+
- [API](#api)
31+
- [CLI](#cli)
32+
- [Contribute](#contribute)
33+
- [License](#license)
34+
35+
## Background
36+
37+
As JS-IPFS evolves and new technologies, algorithms and data structures are being incorporated it is necessary to
38+
enable users easy transition between versions. Different versions of JS-IPFS may expect different structure or content
39+
of the IPFS repo (see: [IPFS repo spec](https://github.com/ipfs/specs/tree/master/repo), [JS implementation](https://github.com/ipfs/js-ipfs-repo) ).
40+
For that reason IPFS repo is versioned and this package provides framework to create migrations which transits
41+
one version of IPFS repo into next/previous one.
42+
43+
This framework provides:
44+
* Handles locking/unlocking of repository
45+
* Define migrations API
46+
* Executes and reports migrations in both direction: forward and backword
47+
* Simplify creation of new migrations
48+
49+
## Install
50+
51+
### npm
52+
53+
```sh
54+
> npm install ipfs-repo-migrations
55+
```
56+
57+
### Use in Node.js
58+
59+
```js
60+
const migrations = require('ipfs-repo-migrations')
61+
```
62+
63+
### Use in a browser with browserify, webpack or any other bundler
64+
65+
```js
66+
const migrations = require('ipfs-repo-migrations')
67+
```
68+
69+
## Usage
70+
71+
Example:
72+
73+
```js
74+
const migrations = require('ipfs-repo-migrations')
75+
const getVersion = require('ipfs-repo-migrations/repo/version')
76+
77+
const repoPath = 'some/repo/path'
78+
const repoVersion = await getVersion(repoPath)
79+
80+
if(repoVersion < migrations.getLatestMigrationVersion()){
81+
// Old repo! Lets migrate to latest version!
82+
await migrations.migrate(repoPath)
83+
}
84+
```
85+
86+
### Writing migration
87+
88+
#### Architecture of migrations
89+
90+
All migrations are placed in `/migrations` folder. Each folder there represents one migration that behaves as stand-alone
91+
package that has its own `package.json` file that states migration's dependencies, has its own tests and follows migration
92+
API.
93+
94+
All migrations are collected in `/migrations/index.js`, which should not be edited manually is it is regenerated on
95+
every run of `jsipfs-migrations add` (or the manual changes should follow same style of modifications).
96+
The order of migrations is important and the migrations have to be sorted in the growing order.
97+
98+
Each migration has to follow this API. It has to export object in its `index.js` that has following properties:
99+
100+
* `version` (int) - Number that represents the version into which will be the repo migrated to (eq. `migration-8` will move the repo into version 8).
101+
* `description` (string) - Brief description of what the migrations does.
102+
* `reversible` (bool) - Specify if it is possible to revert this migration.
103+
* `migrate` (function) - Function that on execution will perform the migration, see signature of this function bellow.
104+
* `revert` (function) - If `reversible == True` then this function will be used to revert the migration to previous version.
105+
106+
##### `migrate(repoPath, isBrowser)`
107+
108+
_Do not confuse this function with the `require('ipfs-repo-migrations').migrate()` function that drives the whole migration process!_
109+
110+
Arguments:
111+
* `repoPath` (string) - absolute path to the root of the repo
112+
* `isBrowser` (bool) - indicates if the migration is run in browser environment in oppose to NodeJS
113+
114+
##### `revert(repoPath, isBrowser)`
115+
116+
_Do not confuse this function with the `require('ipfs-repo-migrations').revert()` function that drives the whole backward migration process!_
117+
118+
Arguments:
119+
* `repoPath` (string) - path to the root of the repo
120+
* `isBrowser` (bool) - indicates if the migration is run in browser environment in oppose to NodeJS
121+
122+
#### Browser vs. NodeJS environments
123+
124+
Migration might need to distinguish in what environment it runs (browser vs. NodeJS), for this reason there is the argument
125+
`isBrowser` passed to migrations functions. But with simple migrations it should not be necessary to distinguish between
126+
these environments as datastore implementation will handle the main differences.
127+
128+
There are currently two main datastore implementations:
129+
1. [`datastore-fs`](https://github.com/ipfs/js-datastore-fs) that is backed by file system and is used mainly in NodeJS environment
130+
2. [`datastore-level`](https://github.com/ipfs/js-datastore-level) that is backed by LevelDB and is used mainly in browser environment
131+
132+
Both implementations share same API and hence are interchangeable.
133+
134+
When the migration is run in browser environment the `datastore-fs` is automatically replaced with `datastore-level` even
135+
when it is directly imported (`require('datastore-fs')` will return `datastore-level` in browser). Because of this mechanism
136+
with simple migrations you should not worry about difference between `datastore-fs` and `datastore-level`.
137+
138+
#### Guidelines
139+
140+
The recommended way to write a new migration is to first bootstrap an dummy migration using the CLI:
141+
142+
```sh
143+
> jsipfs-migrations add
144+
```
145+
146+
Afterwards new folder is created with bootstrapped migration. You can then simply fill in the required fields and
147+
write the rest of migration!
148+
149+
The `node_modules` of the migration should be committed to the repo to ensure that the dependencies are resolved even in
150+
far future, when the package might be removed from registry.
151+
152+
#### Tests
153+
154+
**TODO**
155+
156+
#### Empty migrations
157+
158+
For inter-operable reasons with Go-IPFS it might be necessary just to bump a version of repo without any actual
159+
modification as there might not be any changes needed in JS implementation. For that you can create "empty migration".
160+
161+
The easiest way is to use the CLI for that:
162+
163+
```sh
164+
> jsipfs-migrations add --empty
165+
```
166+
167+
This will create empty migration with next version in line.
168+
169+
### Migrations matrix
170+
171+
**TODO**
172+
173+
## API
174+
175+
### `migrate(path[, toVersion[, progressCb[, isDryRun]]]) -> Promise<void>`
176+
177+
Executes forward migration to specific version or if not specified to the latest version.
178+
179+
**Arguments:**
180+
181+
* `path` (string, mandatory) - path to the repo to be migrated
182+
* `toVersion` (int, optional) - version to which the repo should be migrated to. If left out the version of latest migration is used.
183+
* `progressCb` (function, optional) - callback that is called after finishing execution of each migration to report progress.
184+
* `isDryRun` (bool, optional) - flag that indicates if it is a dry run that should imitate running migration without actually any change.
185+
186+
#### `progressCb(migration, counter, totalMigrations)`
187+
188+
Signature of the progress callback.
189+
190+
**Arguments:**
191+
* `migration` (object) - object of migration that just successfully finished running. See [Architecture of migrations](#architecture-of-migrations) for details.
192+
* `counter` (int) - current number of migration in the planned migrations streak.
193+
* `totalMigrations` (int) - total count of migrations that are planned to be run.
194+
195+
### `revert(path, toVersion[, progressCb[, isDryRun]]) -> Promise<void>`
196+
197+
Executes backward migration to specific version.
198+
199+
**Arguments:**
200+
201+
* `path` (string, mandatory) - path to the repo to be reverted
202+
* `toVersion` (int, mandatory) - version to which the repo should be reverted to.
203+
* `progressCb` (function, optional) - callback that is called after finishing execution of each migration to report progress.
204+
* `isDryRun` (bool, optional) - flag that indicates if it is a dry run that should imitate running migration without actually any change.
205+
206+
### `getLatestMigrationVersion() -> int`
207+
208+
Return the version of latest migration.
209+
210+
## CLI
211+
212+
The package comes also with CLI that is exposed as NodeJS binary with name `jsipfs-repo-migrations`.
213+
It has several commands:
214+
215+
* `migrate` - performs forward migration to specific or latest version.
216+
* `revert` - performs backward migration to specific version.
217+
* `status` - check repo for migrations that should be run.
218+
* `add` - bootstraps new migration.
219+
220+
For further details see the `--help` pages.
221+
222+
## Contribute
223+
224+
There are some ways you can make this module better:
225+
226+
- Consult our [open issues](https://github.com/ipfs/js-ipfs-repo/issues) and take on one of them
227+
- Help our tests reach 100% coverage!
228+
229+
This repository falls under the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md).
230+
231+
[![](https://cdn.rawgit.com/jbenet/contribute-ipfs-gif/master/img/contribute.gif)](https://github.com/ipfs/community/blob/master/contributing.md)
232+
233+
## License
234+
235+
[MIT](LICENSE)

‎package.json

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"name": "ipfs-repo-migrations",
33
"version": "0.1.0",
4-
"description": "Tool for versioning and migrations of IPFS repositories",
4+
"description": "Migration framework for versioning of JS IPFS Repo",
55
"leadMaintainer": "Adam Uhlir <[email protected]>",
66
"main": "src/index.js",
77
"files": [

‎src/index.js

+53-48
Original file line numberDiff line numberDiff line change
@@ -43,37 +43,39 @@ async function migrate (path, toVersion, progressCb, isDryRun) {
4343
let lock
4444
if (!isDryRun) lock = await repoLock.lock(currentVersion, path)
4545

46-
if (currentVersion === toVersion) {
47-
log('Nothing to migrate, skipping migrations.')
48-
return
49-
}
50-
let counter = 0
51-
let totalMigrations = toVersion - currentVersion
52-
for (let migration of migrations) {
53-
if (toVersion !== undefined && migration.version > toVersion) {
54-
break
46+
try {
47+
if (currentVersion === toVersion) {
48+
log('Nothing to migrate, skipping migrations.')
49+
return
5550
}
51+
let counter = 0
52+
let totalMigrations = toVersion - currentVersion
53+
for (let migration of migrations) {
54+
if (toVersion !== undefined && migration.version > toVersion) {
55+
break
56+
}
5657

57-
if (migration.version > currentVersion) {
58-
counter++
59-
log(`Migrating version ${migration.version}`)
60-
if (!isDryRun) {
61-
try {
62-
await migration.migrate(path, isBrowser)
63-
} catch (e) {
64-
e.message = `During migration to version ${migration.version} exception was raised: ${e.message}`
65-
throw e
58+
if (migration.version > currentVersion) {
59+
counter++
60+
log(`Migrating version ${migration.version}`)
61+
if (!isDryRun) {
62+
try {
63+
await migration.migrate(path, isBrowser)
64+
} catch (e) {
65+
e.message = `During migration to version ${migration.version} exception was raised: ${e.message}`
66+
throw e
67+
}
6668
}
69+
typeof progressCb === 'function' && progressCb(migration, counter, totalMigrations) // Reports on migration process
70+
log(`Migrating to version ${migration.version} finished`)
6771
}
68-
typeof progressCb === 'function' && progressCb(migration, counter, totalMigrations) // Reports on migration process
69-
log(`Migrating to version ${migration.version} finished`)
7072
}
71-
}
72-
73-
if (!isDryRun) await repoVersion.setVersion(path, toVersion || getLatestMigrationVersion())
74-
log('All migrations successfully migrated ', toVersion !== undefined ? `to version ${toVersion}!` : 'to latest version!')
7573

76-
if (!isDryRun) await lock.close()
74+
if (!isDryRun) await repoVersion.setVersion(path, toVersion || getLatestMigrationVersion())
75+
log('All migrations successfully migrated ', toVersion !== undefined ? `to version ${toVersion}!` : 'to latest version!')
76+
} finally {
77+
if (!isDryRun) await lock.close()
78+
}
7779
}
7880

7981
exports.migrate = migrate
@@ -112,34 +114,37 @@ async function revert (path, toVersion, progressCb, isDryRun) {
112114

113115
let lock
114116
if (!isDryRun) lock = await repoLock.lock(currentVersion, path)
115-
let counter = 0
116-
let totalMigrations = currentVersion - toVersion
117-
const reversedMigrationArray = migrations.reverse()
118-
for (let migration of reversedMigrationArray) {
119-
if (migration.version <= toVersion) {
120-
break
121-
}
122117

123-
if (migration.version <= currentVersion) {
124-
counter++
125-
log(`Reverting migration version ${migration.version}`)
126-
if (!isDryRun) {
127-
try {
128-
await migration.revert(path, isBrowser)
129-
} catch (e) {
130-
e.message = `During reversion to version ${migration.version} exception was raised: ${e.message}`
131-
throw e
118+
try {
119+
let counter = 0
120+
let totalMigrations = currentVersion - toVersion
121+
const reversedMigrationArray = migrations.reverse()
122+
for (let migration of reversedMigrationArray) {
123+
if (migration.version <= toVersion) {
124+
break
125+
}
126+
127+
if (migration.version <= currentVersion) {
128+
counter++
129+
log(`Reverting migration version ${migration.version}`)
130+
if (!isDryRun) {
131+
try {
132+
await migration.revert(path, isBrowser)
133+
} catch (e) {
134+
e.message = `During reversion to version ${migration.version} exception was raised: ${e.message}`
135+
throw e
136+
}
132137
}
138+
typeof progressCb === 'function' && progressCb(migration, counter, totalMigrations) // Reports on migration process
139+
log(`Reverting to version ${migration.version} finished`)
133140
}
134-
typeof progressCb === 'function' && progressCb(migration, counter, totalMigrations) // Reports on migration process
135-
log(`Reverting to version ${migration.version} finished`)
136141
}
137-
}
138142

139-
if (!isDryRun) await repoVersion.setVersion(path, toVersion)
140-
log(`All migrations successfully reverted to version ${toVersion}!`)
141-
142-
if (!isDryRun) await lock.close()
143+
if (!isDryRun) await repoVersion.setVersion(path, toVersion)
144+
log(`All migrations successfully reverted to version ${toVersion}!`)
145+
} finally {
146+
if (!isDryRun) await lock.close()
147+
}
143148
}
144149

145150
exports.revert = revert

‎src/migration-templates.js

+3-3
Original file line numberDiff line numberDiff line change
@@ -22,11 +22,11 @@ module.exports = {
2222

2323
indexJs: `'use strict'
2424
25-
const Dastore = require('datastore-fs')
25+
const Datastore = require('datastore-fs')
2626
const log = require('debug')('jsipfs-repo-migrations:migration-{{version}}')
2727
2828
async function migrate(repoPath, isBrowser) {
29-
const store = new Dastore(repoPath, {extension: '', createIfMissing: false})
29+
const store = new Datastore(repoPath, {extension: '', createIfMissing: false})
3030
store.open()
3131
3232
try {
@@ -37,7 +37,7 @@ async function migrate(repoPath, isBrowser) {
3737
}
3838
3939
async function revert(repoPath, isBrowser) {
40-
const store = new Dastore(repoPath, {extension: '', createIfMissing: false})
40+
const store = new Datastore(repoPath, {extension: '', createIfMissing: false})
4141
store.open()
4242
4343
try {

0 commit comments

Comments
 (0)