meta: Expose `Bytes` vtable

[carllerche]

At some point in the future, we will want to expose the vtable backing Bytes. This issue tracks relevant discussion as well as API issues that relate to the vtable.

• Convert Bytes -> BytesMut 0.5 release prevents going back from Bytes to BytesMut #350
• mmap [Question] Next steps about vtable and mmap? #359
• try_unsplit Add Bytes::try_unsplit() #287

[rrichardson]

I've been researching a bunch of relevant tickets and solutions for the last couple days, I'm going to attempt to summarize all of the issues/PRs so that we can discuss them in one place and hopefully move forward with a solution.

Introduction

Bytes has become the de-facto ref-counted Buffer for asynchronous services and service frameworks. Interop with most high-level frameworks in the Tokio ecosystem requires using Bytes. However, Bytes features a limited set of allocation modes, so it is incompatible with many high-performance memory management systems. Recent interest in io_uring and it's shared ownership model has highlighted this. Also, there have been a couple requests made to interop with Memory Mapped data as well. In addition, Recycling Buffer Pools/Freelists need a way to "drop" themselves back into their pool instead of De-allocating.

Why should I use Bytes over an Arc<Vec<u8>>? The answer is in Bytes' vtable. It allows differently behaving Bytes instances to offer the same homogenous interface. This enables copy-on-write functionality, as well as supporting common operations on buffers that Rust won't normally allow, such as splitting a partially filled buffer to hand to a consumer while the other portion of the buffer continues to be written.

One might go so far as to say that Bytes is actually two orthogonal sets of functionality under one struct

1. Buffer IO Operations (Read/Write/Cursor)
2. Buffer Lifecycle Management.

Level Setting

There have been repeated requests for 2 umbrellas of operations:

1. Constructing RefCounted instances that enclose 3rdPartyBuffers
   1. 3rdPartyBuffer -> Bytes
   2. 3rdPartyBuffer -> BytesMut
   3. BytesMut -> 3rdPartyBuffer
   4. Bytes -> 3rdPartyBuffer
2. Conversion and Interop between Bytes and BytesMut
   1. split (checked and unchecked)
      1. (Bytes -> BytesMut
      2. BytesMut -> Bytes
   2. unsplit (allocating, try_*)
      1. (Bytes|BytesMut) -> (Bytes | BytesMut) -> (Bytes | BytesMut)
   3. try_mut (Bytes -> BytesMut)

Current VTable Implementation

The VTable implementation currently looks like:

struct Vtable {
    /// fn(data, ptr, len)  to increment a refcount (which may require restructuring)
    pub clone: unsafe fn(&AtomicPtr<()>, *const u8, usize) -> Bytes,
    /// fn(data, ptr, len) To convert a Bytes to a Vec<u>
    pub to_vec: unsafe fn(&AtomicPtr<()>, *const u8, usize) -> Vec<u8>,
    /// fn(data, ptr, len) To decrement or deallocate a Bytes instance
    pub drop: unsafe fn(&mut AtomicPtr<()>, *const u8, usize),
}

New Functionality Required

As I will explain below: Exposing the vtable itself is not quite sufficient to meet the new requirements because it has no way to:

1. Construct a Bytes* instance from a 3rdPartyBuffer.
2. Unwrap a Bytes* instance back into a 3rdPartyBuffer.

As we all know, exposing a plain struct as a public interface is generally considered a bad idea. If we ever need to add/remove/alter fields to the vtable, it will break the API.

In addition, there is some buffer lifecycle management code that might be better to be stored in the VTable (see BytesMut::resize and Bytes::truncate)

For this reason, we need some added functionality that is specific to each underlying buffer type. (Either in the vtable or somewhere nearby)

1. resize :: (*instance, ptr, len) -> Result<(), usize)
   1. This should be called before the actual slice resizing, to prepare the underlying buffer.
   2. It should return Err if it is unable to accomodate (some buffer schemes can't resize)
2. to_byte :: (*instance) -> (*instance, ptr, len)
   1. supply the necessary parts of a Byte or ByteMut so the Bytes impl can c
3. from_byte: `(*instance, ptr, len) -> T
   1. Given the components of a Bytes struct, re-create the original 3rdPartyBuffer
4. (optionally) type_id :: TypeId
   1. We'd use this to ensure that the 3rdPartyBuffer that was the source, is also the dest if we want to "downcast" this instance back to the 3rdPartyBuffer

Proposed Designs

There are 2 lobes to this:

1. How to expose the functionality in a 3rd Party API
2. How should the Bytes/Mut API change to reflect the T <--> Bytes/Mut functionality.

Item 2 should be rather trivial, and would likely be subject to bike-shedding, so this post will only discuss 1.

1. VTableBuilder Proposal

This was proposed by @Matthias247 in #287 (comment)

This mentions explicitly constructing a VTable object, which we may not want, we could alternatively use the Vtable builder as an alternate Bytes constructor, so we don't have to make the VTable struct public.

    let mybytes = Bytes::with_vtable()
        .clone_fn(my_clone_fn)
        .resize_fn(my_resize_fn)
        .drop_fn(my_drop_fn)
        .from_bytes_fn(MyBufferType::from_bytes_parts)
        .build(const* MyBufferType);

The idea would be to offer an API that allows the user to supply fn impls/pointers for each of the known fields of the vtable, but we would still be free to add/alter fields in the VTable.

Pros

1. Easiest solution to implement for the Bytes crate.

Cons

1. Exposes quite a few internal details of the VTable. If we do need to alter VTable, it could still break compatibility.
2. The actual implementations of the functions could be tricky to implement. as an example

2. dyn Trait Approach

This was proposed by several people, the most compelling example was by @quark-zju in their implementation of mini-bytes. #359 (comment)

This implementation very simply and elegantly allows a Byte to use the buffer contents of a 3rdPartyBuffer as well as control its lifetime in a refcounted way, and with very little hassle on the part of the 3rd party.

(Note that the above example doesn't defer the clone/drop to the 3rdPartyBuffer instance. So it can't offer the optimizations that Bytes currently offers, however, that can easily be added)

The idea is this:

    trait ManagedBuffer {
        fn  get_slice() -> &[u8]; 
        fn inc_ref() -> Option<Self>;
        fn dec_ref() -> Option<Self>;
        fn resize(&self, usize) -> Option<Self>; 
        ... 
    }
    
    struct Bytes {
        ptr: *const u8,
        len: usize,
        owner: Box<dyn ManagedBuffer>,
    }
    
    impl Bytes {
        pub fn from_buffer_manager(bman: impl BufferManager);
    }

So you impl BufferManager for your 3rdPartyBuffer, and use it to construct

Others have suggested a similar approach, but to store the object as an Arc<dyn Any + Send + Sync + 'static> so that downcasting can be simply accomplished by the Arc implementation

Pros

1. Intuitive for most developers
2. Simplifies the implementation of Bytes
3. Lets rustc manage the VTable, so we don't have to.
4. The operation to convert Bytes back into 3rdPartyBuffer is trivial and universal.

Cons

1. Significant change vs the current model.
2. Lifecycle management might git a bit more complicated.

3. Trait as VTable Builder Approach

This is sort of a hybrid of approaches 1 and 2. It offers more flexibility than both, but at the possible expense of greater complexity. This was implemented in PR (#567) by @HyeonuPark .

This approach features an expanded VTable, as well as a stateless Trait design. This differs from approach 2 because the trait exists to provide functionality to the VTable, as well as provide functionality to re-construct/downcast the Bytes type back to T.

Note that the above PR features a clone method that looks like: unsafe fn clone(data: &AtomicPtr<()>, ptr: *const u8, len: usize) -> Bytes; With the current Bytes API, this would require that the implementor of the BytesImpl trait know the internal construction of a Bytes object. That can be fixed with the addition of a fn from_parts(...) -> Self method to the Bytes impl.

Pros

1. No mention of the VTable to the public.
2. A expanding the trait for this won't break compatibility (if the new methods have default impls)

Cons

1. Higher complexity for both the Bytes impl and the 3rd party.

4. The In-a-Perfect-World Approach

As mentioned in the intro, Bytes encompasses two somewhat orthogonal sets of functionality. Buffer Lifecycle Management, and Buffer Read/Write operations.

This design follows the analogy of the Future and FutureExt traits. The impl Future does the dirty work of concurrency management, but can be implemented for a wide variety of use-cases. Then the FutureExt trait offers the high-level functionality.

So all Buffer lifecycle management, copy on write etc, would be implemented by the impl of trait ManagedBuffer, then the high-level IO functionality would be implemented in trait BytesExt.

The ManagedBuffer design would be similar to proposal 2, except that it would be designed in such a way that the implementation of ManagedBuffer is 100% responsible for itself, e.g. clone, drop, resize, into_parts, from_parts. It would be completely self contained.

In order to maintain compatibility with the existing API, Bytes and BytesMut would become thin wrappers that store a dyn ManagedBuffer and then provide a facade over the functions provided by BytesExt.

The implementor of ManagedBuffer could be any 3rd party type.

So it'd look something like:

pub trait ManagedBuffer {
        fn from_parts() -> Self; 
        fn into_parts(this: Self) -> (...);
        fn clone() -> Self;
        fn drop() -> Self
        fn resize(usize) -> Self
}

pub trait ByteExt: ManagedBuffer {
    fn slice() -> Self {
        ...
    }
    fn len() -> usize { 
        ....
    }
    ....
}

impl<T> ByteExt for T
where
    T: ManagedBuffer + ?Sized


struct StaticBuffer {
    buf: &'static [u8]
} 

// this is where the STATIC_VTABLE impl would go.. more or less
impl ManagedBuffer for StaticBuffer {
 ....
}

... 

[rrichardson]

pinging @ipoupaille for comment, since they made a PR (#558) for VTable exposure as well.

[chenyuanrun]

I need this to avoid memory copy between rust and c++ code.

[djkoloski]

A couple people are asking for ways to convert rkyv's AlignedVec into an axum::body::Body without copying the bytes. It looks like custom Bytes constructors are required to do this.

[maxburke]

Hi! I'd also like to throw an up-vote in on this issue. I've got some memory mapped files I would like to expose a zero-copy Bytes interface on, and right now the only way I've been able to do that is by using Bytes::with_vtable. So, if it could be made public, perhaps behind a feature gate, it would make things easier.

[scottlamb]

I've got some memory mapped files I would like to expose a zero-copy Bytes interface on,

YMMV depending on whether your files are disk-backed or SSD-backed, whether you can mlock them (in terms of affording the RAM and the syscall cost), and whether you're accessing the Bytes from an async thread, but...my experience has been this combo is a bad idea because it causes unpredictable slow major faults that stall the event-driven threads for a long time. More here.