Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding file access mode #549

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ It is a subset of MongoDB's API (the most used operations).
You can use NeDB as an in-memory only datastore or as a persistent datastore. One datastore is the equivalent of a MongoDB collection. The constructor is used as follows `new Datastore(options)` where `options` is an object with the following fields:

* `filename` (optional): path to the file where the data is persisted. If left blank, the datastore is automatically considered in-memory only. It cannot end with a `~` which is used in the temporary files NeDB uses to perform crash-safe writes.
* `fileMode` (optional): file access mode for newly created files. Should be specified as number, as ex: 0o640. If not set, NodeJS default behaviour is used.
* `inMemoryOnly` (optional, defaults to `false`): as the name implies.
* `timestampData` (optional, defaults to `false`): timestamp the insertion and last update of all documents, with the fields `createdAt` and `updatedAt`. User-specified values override automatic generation, usually useful for testing.
* `autoload` (optional, defaults to `false`): if used, the database will automatically be loaded from the datafile upon creation (you don't need to call `loadDatabase`). Any command issued before load is finished is buffered and will be executed when load is done.
Expand Down Expand Up @@ -106,6 +107,10 @@ db.robots = new Datastore('path/to/robots.db');
// You need to load each database (here we do it asynchronously)
db.users.loadDatabase();
db.robots.loadDatabase();

// Type 5: Specifying file access mode
var Datastore = require('nedb')
, db = new Datastore({ filename: 'path/to/datafile', fileMode: 0o600 });
```

### Persistence
Expand Down
2 changes: 2 additions & 0 deletions lib/datastore.js
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ var customUtils = require('./customUtils')
* @param {Function} options.afterSerialization/options.beforeDeserialization Optional, serialization hooks
* @param {Number} options.corruptAlertThreshold Optional, threshold after which an alert is thrown if too much data is corrupt
* @param {Function} options.compareStrings Optional, string comparison function that overrides default for sorting
* @param {Number} options.fileMode Optional, file access mode for creating files that overrides NodeJS default file mode (example: 0o600)
*
* Event Emitter - Events
* * compaction.done - Fired whenever a compaction operation was finished
Expand All @@ -39,6 +40,7 @@ function Datastore (options) {
this.inMemoryOnly = options.inMemoryOnly || false;
this.autoload = options.autoload || false;
this.timestampData = options.timestampData || false;
this.fileMode = options.fileMode || false;
}

// Determine whether in memory or persistent
Expand Down
3 changes: 2 additions & 1 deletion lib/persistence.js
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ function Persistence (options) {
this.db = options.db;
this.inMemoryOnly = this.db.inMemoryOnly;
this.filename = this.db.filename;
this.fileMode = this.db.fileMode;
this.corruptAlertThreshold = options.corruptAlertThreshold !== undefined ? options.corruptAlertThreshold : 0.1;

if (!this.inMemoryOnly && this.filename && this.filename.charAt(this.filename.length - 1) === '~') {
Expand Down Expand Up @@ -134,7 +135,7 @@ Persistence.prototype.persistCachedDatabase = function (cb) {
}
});

storage.crashSafeWriteFile(this.filename, toPersist, function (err) {
storage.crashSafeWriteFile(this.filename, this.db.fileMode, toPersist, function (err) {
if (err) { return callback(err); }
self.db.emit('compaction.done');
return callback(null);
Expand Down
20 changes: 14 additions & 6 deletions lib/storage.js
Original file line number Diff line number Diff line change
Expand Up @@ -38,24 +38,26 @@ storage.ensureFileDoesntExist = function (file, callback) {
/**
* Flush data in OS buffer to storage if corresponding option is set
* @param {String} options.filename
* @param {Number} options.fileMode Optional, file access mode, defaults to do not change NodeJS behaviour
* @param {Boolean} options.isDir Optional, defaults to false
* If options is a string, it is assumed that the flush of the file (not dir) called options was requested
*/
storage.flushToStorage = function (options, callback) {
var filename, flags;
var filename, fileMode, flags;
if (typeof options === 'string') {
filename = options;
flags = 'r+';
} else {
filename = options.filename;
fileMode = options.fileMode;
flags = options.isDir ? 'r' : 'r+';
}

// Windows can't fsync (FlushFileBuffers) directories. We can live with this as it cannot cause 100% dataloss
// except in the very rare event of the first time database is loaded and a crash happens
if (flags === 'r' && (process.platform === 'win32' || process.platform === 'win64')) { return callback(null); }

fs.open(filename, flags, function (err, fd) {
var fn = function (err, fd) {
if (err) { return callback(err); }
fs.fsync(fd, function (errFS) {
fs.close(fd, function (errC) {
Expand All @@ -69,17 +71,23 @@ storage.flushToStorage = function (options, callback) {
}
});
});
});
};
if (fileMode) {
fs.open(filename, flags, fileMode, fn);
} else {
fs.open(filename, flags, fn);
}
};


/**
* Fully write or rewrite the datafile, immune to crashes during the write operation (data will not be lost)
* @param {String} filename
* @param {String} data
* @param {Number} fileMode
* @param {Function} cb Optional callback, signature: err
*/
storage.crashSafeWriteFile = function (filename, data, cb) {
storage.crashSafeWriteFile = function (filename, fileMode, data, cb) {
var callback = cb || function () {}
, tempFilename = filename + '~';

Expand All @@ -95,13 +103,13 @@ storage.crashSafeWriteFile = function (filename, data, cb) {
});
}
, function (cb) {
storage.writeFile(tempFilename, data, function (err) { return cb(err); });
storage.writeFile(tempFilename, data, fileMode ? { mode: fileMode} : {}, function (err) { return cb(err); });
}
, async.apply(storage.flushToStorage, tempFilename)
, function (cb) {
storage.rename(tempFilename, filename, function (err) { return cb(err); });
}
, async.apply(storage.flushToStorage, { filename: path.dirname(filename), isDir: true })
, async.apply(storage.flushToStorage, { filename: path.dirname(filename), fileMode: fileMode, isDir: true })
], function (err) { return callback(err); })
};

Expand Down