Flatten Out an unordered_map in a Struct

[dwaydwaydway]

Hi,

I have a C++ application which is sending&receiving json messages that have the following structure

1. It has unknown numbers of string key/value pairs
2. It might have a member "errors" which is a nested json capturing some system errors

For example:

{
    "a": "1", 
    "b": "2", 
    "c": "3",
    ...
    "errors": {
        "x": "y",
        "w": "z"
    }
}

I tried creating a struct like the following:

struct Feature {
    std::unordered_map<std::string_view, std::string_view> attributes;
    std::optional<std::unordered_map<std::string_view, std::string_view>> errors;
};

However, this will result in a key "attributes" capturing all the key/value pairs, which is not what I want:

{
    "attribute": {
        "a": "1", 
        "b": "2", 
        "c": "3",
        ...
    },
    "errors": {
        "x": "y",
        "w": "z"
    }
}

I also tried using glz::merge:

std::unordered_map<std::string_view, std::string_view> attributes;
struct Errors{
    std::optional<std::unordered_map<std::string_view, std::string_view>> errors;
};
auto merged = glz::merge{attributes, errors};
std::string s{};
glz::write_json(merged, s); 

Which does write the json structure I want, but then I'm not sure I should read such message

Is there a way to get around this?
Thanks!

[stephenberry]

Glaze currently doesn't have a clean solution for merging during reads for dynamic types. This is tricky to do in a generic way because it means combining two separate hashing systems for efficiency.

The solution I typically take for these cases is to utilize glz::raw_json or glz::raw_json_view.

Consider this solution:

#include <iostream>

#include "glaze/glaze.hpp"

using map_t = std::unordered_map<std::string_view, std::string_view>;

struct Feature {
    map_t attributes{};
    std::optional<map_t> errors{};
};

namespace glz {
template <>
struct detail::to<JSON, Feature> {
    template <auto Opts, class Value>
    static void op(Value&& value, auto&& ctx, auto&& b, auto&& ix) {
        if (value.errors) {
            auto merged = glz::merge{value.attributes,
                                     glz::obj{"errors", value.errors.value()}};
            detail::write<JSON>::template op<Opts>(merged, ctx, b, ix);
        } else {
            detail::write<JSON>::template op<Opts>(value.attributes, ctx, b,
                                                   ix);
        }
    }
};
}  // namespace glz

int main() {
    Feature obj{.attributes = {{"a", "1"}, {"b", "2"}, {"c", "3"}},
                .errors = map_t{{"x", "y"}, {"w", "z"}}};

    auto buffer = glz::write_json(obj).value_or("error");
    std::cout << buffer << '\n';

    std::unordered_map<std::string_view, glz::raw_json_view> input;
    auto ec = glz::read_json(input, buffer);
    if (ec) {
        std::cerr << glz::format_error(ec, buffer) << '\n';
    }

    if (input.contains("errors")) {
        ec = glz::read_json(obj.errors, input["errors"].str);
        if (ec) {
            std::cerr << glz::format_error(ec, input["errors"].str) << '\n';
        }

        // errors have now been populated
        if (obj.errors) {
            std::cout << "Errors:\n";
            for (auto& [key, value] : obj.errors.value()) {
                std::cout << key << ", " << value << '\n';
            }
        }
    }

    return 0;
}

Note that I added custom serialization for Feature so I could make the memory exactly match your original struct. If you don't care so much for the memory layout then you can use the merge approach you proposed and avoid specializing detail::to.

glz::raw_json_view will parse the JSON value into a std::string_view underneath. Note that I used views in this example because your code uses views, but this means that buffers must be kept alive. You can use std::string and glz::raw_json if buffers will be destroyed.

Making custom to/from specializations will become easier in the future and should allow you to avoid using glz::raw_json, but for now I think this is the most flexible/cleanest approach.

[stephenberry]

Here's a compiler explorer link to the example: https://gcc.godbolt.org/z/K9KGbnh35

[dwaydwaydway]

Thanks for the details and the example!
However, the issue I have with approach is that, when reading such json this way, the values of attributes would come with double quotes

std::unordered_map<std::string_view, glz::raw_json_view> input;
auto ec = glz::read_json(input, buffer);
if (ec) {
    std::cerr << glz::format_error(ec, buffer) << '\n';
}
for (auto& [key, value] : input) {
    std::cout << key << ", " << value.str << '\n';
}

a, "1"
b, "2"
c, "3"

Is there a way to get around that?

[stephenberry]

Yes, you're delaying your parse for the raw_json values. So, you need to do a read_json call to decode these values (and remove quotes in the case of strings). This does mean you are doing two passes on the input buffer, but the first is just skipping the value and quite fast.