beve transported on sockets

[jbbjarnason]

I am thinking of changing to beve for interprocess communications via unix domain sockets. Currently using manual encode/decode methods.
What I am not able to wrap my head around is the size of the message, if I receive partial packet I would need to concatenate the binary buffer before trying to convert it.
What I was thinking is having a header before the payload in the binary buffer,

template <type_e type_enum>
struct header_t {
  static constexpr auto type_v{ type_enum }; // type_e is an enum
  version_e version{ version_e::v0 }; // version_e is an enum
  type_e type{ type_v };
  std::size_t payload_size{};
};

When I would write header_t will the resulting binary output be of fixed size? I would like to expect on the receiving end some fixed size before conversion back to header_t.

[stephenberry]

Ah, interesting! I've actually been writing an RPC specification for BEVE. I just made it public so that I can get your thoughts on it: https://github.com/stephenberry/repe

I've been designing it with insights into performance issues with the JSON RPC specification, but still quite similar.

One important design consideration was that the header must not be enclosed in the same object as the body, because we want to be able to efficiently parse this separately and not require double parsing or the entire message to be received. This pertains to the issue you raised. I don't think it is a good idea to have a fixed size header, because I think using method names is extremely helpful for generic code and RPC frameworks. However, this specification solves that problem by using a BEVE array, which lets you know how many parameters to parse, and each parameter has known sizes. But, to make this even simpler, we separate the header, parameters, and body with BEVE delimiters, so that users can write a bit cleaner code and switch on value tags. This way you know you've read your entire header when you've hit the first BEVE delimiter.

I see that you included the payload size in your header. I'm really torn about this. I don't like the idea of needing to have my entire payload defined prior to sending my header. And, the BEVE specification sends sizes of all values, so you can still reserve memory on individual elements. And, in networking typically you are using a fixed size input buffer anyway. I'd love to hear your thoughts on this.

I haven't made much progress on implementing this spec yet in Glaze, but I have plans to do much more effort soon. There's been a branch: binary_rpc for a little while now.

[stephenberry]

Thanks for the example code. I think this fits really well with my goals.

[justend29]

Please ignore this if it's irrelevant to the thoughts here, but one thing we currently do at work to make reading network data a bit faster is to store the network payload in the data object and refer to it with members. This only works with data that doesn't need to be decoded, but alleviates reallocating and copying data into each member. The basic process is:

1. read network data into buffer type as usual.
2. move buffer into custom data
3. use viewing members (std::string_view) instead of owning members (std::string) in data object, refer to held buffer.

Each network read then needs to allocate a new buffer, but we usually found that this is faster than an allocation per member, since it's one large vs. many small.

Another point is that when decoding the members is needed, you can do it lazily since you have the network data. This has been massively helpful when reading endpoints that return entire structures with complex data, but only some of the fields are used without knowing which ones will be used before-hand.

Just spewing some findings from work in case they're inspiring here!

[jbbjarnason]

@justend29 This is interesting aspect, I did think of something like this before with my manual implementation but didn't implement it. I would like to add to your suggestion.

1. read network data into buffer type as usual.

Maybe it could be achievable to not copy the data into a buffer, with a special type of buffer, something like memory_map_buffer. This buffer type would use glz::object tuple to generate array of iterators, I will try write an example:

struct foo {
  int a{};
  int space_between_memory_addresses{};
  std::string b{};
  struct glaze { static constexpr auto value{ glz::object(&foo::a, &foo::b) }; };
};
template <typename T>
struct memory_map_buffer;
void do_stuff() {
  memory_map_buffer<foo> c{ foo{} };
  // c would have an iterator which would iterate value of `a` firstly then `b` secondly according to their sizes 
  // so this buffer type would make it look like the memory is contiguous even though it can be non-contiguous
}

I am not sure on how to propagate the memory_map_buffer to underlying socket mechanism, but in terms of asio something like buffer sequence would be needed, https://www.boost.org/doc/libs/1_83_0/doc/html/boost_asio/reference/ConstBufferSequence.html.

2. move buffer into custom data

I do not understand this point fully.

3. use viewing members (std::string_view) instead of owning members (std::string) in data object, refer to held buffer.

Could this be implemented by declaring a view alternative, something like:

struct foo {
  int a{};
  view<int> a_view{};
  int space_between_memory_addresses{};
  std::string b{};
  std::string_view b_view{};
  struct glaze { 
    static constexpr auto value{ glz::object(&foo::a, &foo::b) }; 
    static constexpr auto view{ glz::object(&foo::a_view, &foo::b_view) }; 
  };
};

Maybe this should be a separate struct instance, something like foo_view.

Great discussions!

[justend29]

@jbbjarnason I'm glad this wasn't a waste. I may have been confusing because buffer sequences from asio are always non-owning, but that doesn't have to be the case. By "buffer" I was referring to the backing data of the buffer sequence, which is possibly the sequence itself.

Let's say we have some custom data type, Foo, with members whose data matches the read network data. This custom data type holds a buffer sequence as a (private) member, into which its public members point. Each public member does not own data, it merely points to the held buffer sequence which owns all the data. Foo owns the buffer sequence, though, so it is not a view, it still has the same ownership semantics.

1. read into a buffer, as usual, -> just normally read from the IO device into a buffer sequence as all asio read operations do. I wasn't even going as far as to say the buffer sequence could be fancy and write directly to the members of the decoded data, as you describe. Maybe that's better? I'm not sure, but it wasn't what I was describing.

2. move the buffer into custom data. Once the network data has been read into the buffer, the custom data type, Foo, must now own it. Should the buffer not be owning, the underlying data could be moved. So, move that into Foo.

3. Realistically, this isn't a "step" but it's just ensuring that the public members are assigned into the network buffer appropriately. This should coincide with the process of moving the network data into Foo such that the members are built correctly.

// here, an owning buffer sequence from Beast is held, but the underlying std::vector<std::byte>` or whatever is used as the backing for a non-owning buffer sequence could be held

class Foo {
  std::string_view get_name();
  std::string_view get_title();

  static std::expected<Foo, glz::parse_error> from_network(beast::flat_buffer&& net_data) {
    Foo result{};
    result._net_data = std::move(net_data);
    if(const auto err = glz::read_json(result._fields, result._net_data); err) {
      return std::unexpected(err);
    }
    return result;
  }

 private:
  struct Fields {
     std::string_view name{};
     std::string_view title{};
     struct glaze { /* as usual */ };
  };
  
  beast::flat_buffer _net_data{};
  Fields _fields{};
}

int main() {
   best::flat_buffer net_data{};
   read_data_from_socket_with_asio(net_data); // please, nobody read IO sync. like this
   const auto network_foo = Foo::from_network(net_data);
}

Comparing this with using std::strings as members, performance gains were found because allocation & copying was not necessary for each member. It does mean a new buffer needs to be allocated for each network read, as the buffer cannot be reused due to step 2.

[justend29]

Then, the part about lazily reading members, is if you have complex data, Foo could have a member that decodes it on the fly instead of having it decoded on construction (i.e. with glz::read_json/binary/...). So,

struct Foo {
  // ...
  std::chrono::time_point make_timestamp() {
    // parse timestamp from bytes into time_point
  }
  struct Fields {
    //...
    std::span<std::byte>timestamp{}; // assign to bytes on `from_network`
  };