Add hooks to SchemaAdapter to add custom column generators
#15261
+562
−45
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
core
adriangb
commented
Mar 16, 2025
Comment on lines
279
to
280
| let err = adapter.map_schema(&file_schema).unwrap_err().to_string(); | ||
| assert_eq!(err, "Error during planning: Column a is missing from the file schema, cannot be generated, and is non-nullable"); |
There was a problem hiding this comment.
It's nice that this fails earlier 😄
|
I've moved the complex bit over to #15263. I'll let that settle first then resume work here. |
|
Noting that in #15263 (comment) I realized that it might be good to have a system to report stats for columns that will be generated before they are generated (is it all nulls? is it a constant?) to be used with stats pruning. |
|
Now that #15263 is merged I'll come back here and:
|
adriangb
force-pushed
the
defaults-schema-adapter
branch
from
896eb8b to
74b884f
Compare
adriangb
force-pushed
the
defaults-schema-adapter
branch
from
de02a9a to
0bdebc8
Compare
|
Marking as ready for review. The main TODO is an API for transmitting statistics information for generated columns before they get generated, but that can even be a followup PR. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #15220
A lot of the work of this PR is meant to resolve #15220 (comment). I think I'll move that into a standalone PR.