Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Statistics::total_byte_size does not account for projection in FileScanConfig::with_projection #14936

Open
Tracked by #8227
alamb opened this issue Feb 28, 2025 · 1 comment
Labels
bug Something isn't working

Comments

@alamb
Copy link
Contributor

alamb commented Feb 28, 2025

Describe the bug

As @blaginin found in #14685, the statistics when a File is projected (aka only a subset of the columns are present) is incorrect

Specifically, the projected statistics have the same total_byte_size as the input. However, given only a subset of columns are selected this will mean that the total_byte_size should actually be lower

To Reproduce

See tests referenced in #14685

Expected behavior

total_byte_size should take into account the subset of columns

Additional context

No response

@blaginin
Copy link
Contributor

Should be easier to fix after #15352

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants