Got syntax_error/unknown_key when reading files with carriage return

[Nukoooo]

I kinda got annoyed by this error so I decided to figure out what happened with it, because the files can be read by Libreoffice, I saw the code checks for '\r' at here

glaze/include/glaze/csv/read.hpp

Line 128 in feabaaf

case '\r':

so I wondered if it was because of that so I made a minimum example with the following code (data is from issue #666)

    std::ifstream file("./fish_record.csv", std::ios::binary);

    std::vector<std::uint8_t> file_content{ std::istreambuf_iterator(file), {} };
    auto has_char = [&](char c)
    {
        return std::ranges::find(file_content, c) != file_content.end();
    };

    std::cout << has_char('\\') << std::endl;
    std::cout << has_char('\b') << std::endl;
    std::cout << has_char('\f') << std::endl;
    std::cout << has_char('\r') << std::endl;
    std::cout << has_char('\t') << std::endl;

    std::string buffer_with_r{ file_content.begin(), file_content.end() };

    FishRecord rec{};
    auto error = glz::read_csv<glz::colwise>(rec, buffer_with_r);
    std::cout << "Read error code: " << glz::format_error(error, std::string {}) << std::endl;

    auto it = std::ranges::find(file_content, '\r');
    std::string remaining { std::ranges::find(it, file_content.end(), ','), file_content.end() };
    std::cout << remaining << std::endl;

    for (const auto& caught_fish_name : rec.CaughtFishName)
    {
        std::cout << caught_fish_name << std::endl;
    }

    std::erase_if(file_content,
                  [&](const auto& i)
                  {
                      return i == '\r';
                  });

    std::string buffer_without_r { file_content.begin(), file_content.end() };
    FishRecord rec2 {};
    error = glz::read_csv<glz::colwise>(rec2, buffer_without_r);
    
    std::cout << "Read error code: " << glz::format_error(error, std::string {}) << std::endl;

    for (const auto& caught_fish_name : rec2.CaughtFishName)
    {
        std::cout << caught_fish_name << std::endl;
    }

and gave the result:

0
0
0
1
0
Read error code: unknown_key
,8.365,12.5,1,万能拟饵,无,无,撒饵,利姆萨·罗敏萨下层甲板,重杆,提钩,否,否
海港鲱,7.586,11.1,1,万能拟饵,无,无,撒饵,利姆萨·罗敏萨下层甲板,重杆,提钩,否,否
猛犸章鱼,26.044,55.6,1,万能拟饵,无,海港鲱,撒饵,利姆萨·罗敏萨下层甲板,鱼王杆,提钩,否,否

珊瑚蝶
Read error code :none
珊瑚蝶
海港鲱
猛犸章鱼

After removing \r from the byte array it can parse the content properly into the struct without any weird behavior. So I'm curious about if checking \r intended or not, if it is, what are the reasons behind this 👀

My code in the project:

[stephenberry]

Ah, as you noticed the CSV reader doesn't currently support \r. However, this should be simple to add. Do you expect \r within your strings? Is it supposed to be part of the key? Or, does it just appear when you have new lines in your file? I think handling it at new lines makes the most sense.

[Nukoooo]

I don't expect \r to be in the string or part of the key honestly, it seems to be a Windows-only thing

Or, does it just appear when you have new lines in your file? I think handling it at new lines makes the most sense.

Yes, and I agree with this

[stephenberry]

Excellent. I'll make an issue for this and hopefully fix it soon, it shouldn't be very difficult.