Skip to content
This repository was archived by the owner on Sep 13, 2023. It is now read-only.

return error result only from Iterator.next() #14

Merged
merged 14 commits into from
Dec 19, 2018

Conversation

mykmelez
Copy link

This is an alternative to #13 that minimizes the number of methods that return an error result down to just the Iterator.next() implementation in Iter by using std::iter::once() to create an iterator that returns an error result when a failure occurs in the Cursor.iter*() methods.

A potential downside of this approach is that it returns an Iterator trait object from the Cursor.iter*() methods, which could have runtime performance implications in theory (although I'm unsure that it does in practice).

(I've previously requested integration of these changes upstream in danburkert#45 and would like to move forward with these changes downstream while awaiting that integration.)

@mykmelez mykmelez requested a review from ncalexan November 19, 2018 22:29
@mykmelez
Copy link
Author

@ncalexan I've submitted this separately from #13 and requested review from you for both branches because I'd like your input on the tradeoff between the ergonomic benefits of minimizing the number of fallible methods vs. the potential performance (or other) implications of using a trait object in this branch.

(Also happy for any insight you have into another way we could make the Cursor.iter*() methods infallible without using a trait object.)

Copy link
Member

@ncalexan ncalexan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't review this thoroughly, 'cuz I want to talk through options. It's weird to me that any of these iterators can produce errors. iter_dup_of with an unknown key isn't an error; it should always yield an empty iterator. iter_from shouldn't error if the value isn't found, it should produce an empty iterator. And if these things for reasons I don't understand do error, it should be on the use of the iterator (.next()), not at creation time. (Almost certainly.)

Let's talk about this more soon.

src/cursor.rs Outdated
@@ -57,12 +57,12 @@ pub trait Cursor<'txn> {
/// For databases with duplicate data items (`DatabaseFlags::DUP_SORT`), the
/// duplicate data items of each key will be returned before moving on to
/// the next key.
fn iter_from<K>(&mut self, key: K) -> Result<Iter<'txn>> where K: AsRef<[u8]> {
fn iter_from<K>(&mut self, key: K) -> Box<Iterator<Item=Result<(&'txn [u8], &'txn [u8])>>> where K: AsRef<[u8]> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The way to avoid the Box is easy enough -- push the branch down into the underlying Iter struct. That is, make it:

/// An iterator over the values in an LMDB database.
pub enum Iter<'txn> {
    Err(Result::E),
    Ok(...),
}

where the bits that are currently in Iter are in the Ok branch as well.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The way to avoid the Box is easy enough -- push the branch down into the underlying Iter struct.

Ok, I've done this in a558fab.

src/cursor.rs Outdated
match self.get(Some(key.as_ref()), None, ffi::MDB_SET_RANGE) {
Ok(_) | Err(Error::NotFound) => (),
Err(error) => return Err(error),
Err(error) => return Box::new(iter::once(Err(error))),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, I see how you will only get an error once (and then the iterator ends). With the other approach (Item = Result<...>), how do you ensure you don't get Err forever? This feels similar to fuse in spirit.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, I see how you will only get an error once (and then the iterator ends). With the other approach (Item = Result<...>), how do you ensure you don't get Err forever? This feels similar to fuse in spirit.

I don't currently ensure that; rather, I leave it to the caller to decide what to do on error. I'm not entirely sure if it's possible for LMDB to return a value after returning an error. If so, then consumers should be able to continue iterating in that case.

Otherwise, consumers that call Iterator.next() directly can easily stop iterating at the first error, since they need to match on the return value anyway to distinguish between the Some(Ok), Some(Err) and None types; and collections can easily automatically stop collecting at the first error; so it seems like consumers wouldn't gain much by the Iter returning None after that.

Instead of boxing Iter/IterDup to generalize across both successful and failed attempts to get an iterator, we make Iter and IterDup be enums with Ok and Err variants, where the Ok variant behaves like the current implementations, and the Err variant always returns an error.
@mykmelez
Copy link
Author

mykmelez commented Dec 1, 2018

I didn't review this thoroughly, 'cuz I want to talk through options. … Let's talk about this more soon.

Does this implementation match your expectations based on the conversation we had?

@mykmelez
Copy link
Author

mykmelez commented Dec 1, 2018

Travis failed only on Rust 1.20.0 with errors like:

error[E0308]: mismatched types
   --> src/cursor.rs:265:13
    |
265 |             Iter::Ok { cursor, ref mut op, next_op, _marker } => {
    |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected mutable reference, found enum `cursor::Iter`
    |
    = note: expected type `&mut cursor::Iter<'txn>`
               found type `cursor::Iter<'_>`

I'm unsure what to do about that. Perhaps requiring newer Rust is the simplest solution.

@mykmelez
Copy link
Author

mykmelez commented Dec 1, 2018

I'm unsure what to do about that. Perhaps requiring newer Rust is the simplest solution.

af923dc fixes Rust 1.20.0+ by explicitly identifying references in the match patterns.

@mykmelez
Copy link
Author

@ncalexan Does this look good with the latest set of changes in response to your review comments?

Copy link
Member

@ncalexan ncalexan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, this makes much more sense to me now! Sorry for delayed review.

(I reviewed the squashed diff 'cuz there was a lot of inter-commit churn; I expect that's okay in this situation.)

@mykmelez mykmelez merged commit cfaf37d into mozilla:master Dec 19, 2018
mykmelez added a commit to mykmelez/lmdb-rs that referenced this pull request Dec 19, 2018
mykmelez added a commit that referenced this pull request Dec 20, 2018
update minor version for breaking change in #14
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants