clear without limit

[ronag]

rocksdb has a very nice DeleteRange API which is non blocking and would be great for clear. However, there are two problems:

• Open end is not supported. (Could be hack emulate with '\fffffffffffffff')
• Limit is not supported.

Any chance of making these optional in the level API?

[ronag]

Optionally if we could add a deleteRange API call and discourage clear.

[vweevers]

Why is DeleteRange better? Especially if it does not support these options?

[ronag]

Because it's non blocking. It can apply the delete lazily while applying it to all preceding reads in memory. Also you can do delete range in a batch.

[vweevers]

Not sure I understand what you mean by non blocking.

[ronag]

Not sure I understand what you mean by non blocking.

It doesn't block the thread due to IO.

[vweevers]

From the JS side that doesn't matter. Unless your concern is not having enough threads in the async worker pool (for things other than level)?

[ronag]

I'm actually trying to do all the write operations in the main js thread due to a few different reasons:

• Avoid overhead of marshalling data to the thread pool.
• Increase consistency with reads, i.e. writes are immediately visible for preceding read calls. Without having to add a lot of complicated write barriers and read queuing in the binding. (Snapshots caveats classic-level#28)
• Avoid problems with writes being applied out of order due to thread pool scheduling. (Are writes ordered? classic-level#29)
• Range deletes can be done in a batch.

In rocksdb and I believe also leveldb writes are non blocking as long as the writing background threads can keep up.

[vweevers]

Not sure if there's anything left to answer here. I'll address some specific points:

Open end is not supported

That simply makes it incompatible with the abstract-level API. Unlike limit, the use cases for ltgt options are (I hope) obvious and these options are a core part of level. So can't make this an optional feature.

Limit is not supported

See #38 (comment) - which was a duplicate issue?

Optionally if we could add a deleteRange API call and discourage clear.

I don't want to add RocksDB-specific features that are a limited subset of existing functionality. Your implementation is free to have additional methods ofc (like deleteRange). Perhaps a better way forward is to not follow the abstract-level interface at all, if the requirements are too different.

Increase consistency with reads, i.e. writes are immediately visible for preceding read calls. Without having to add a lot of complicated write barriers and read queuing in the binding.

You have a fundamentally different idea of what the order of operations should be. Which is fine, but it's not level. Level promises consistency in a simple way: want to know that a write finished? Await it. Subsequent reads are then guaranteed to include that written data.

Avoid problems with writes being applied out of order due to thread pool scheduling.

Same applies here: userland code can decide (to care about) the order, by awaiting.