Statistics::total_byte_size
does not account for projection in FileScanConfig::with_projection
#14936
Labels
bug
Something isn't working
Describe the bug
As @blaginin found in #14685, the statistics when a File is projected (aka only a subset of the columns are present) is incorrect
Specifically, the projected statistics have the same
total_byte_size
as the input. However, given only a subset of columns are selected this will mean that thetotal_byte_size
should actually be lowerTo Reproduce
See tests referenced in #14685
Expected behavior
total_byte_size
should take into account the subset of columnsAdditional context
No response
The text was updated successfully, but these errors were encountered: