Skip to content

Avoid redundant Schema clones #17643

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 8 commits into from
Sep 22, 2025
Merged

Avoid redundant Schema clones #17643

merged 8 commits into from
Sep 22, 2025

Conversation

findepi
Copy link
Member

@findepi findepi commented Sep 18, 2025

Avoid cloning Schema structs.

@findepi findepi added the performance Make DataFusion faster label Sep 18, 2025
@github-actions github-actions bot added sql SQL Planner logical-expr Logical plan and expressions physical-expr Changes to the physical-expr crates core Core DataFusion crate substrait Changes to the substrait crate catalog Related to the catalog crate common Related to common crate proto Related to proto crate physical-plan Changes to the physical-plan crate labels Sep 18, 2025
@mbutrovich
Copy link
Contributor

Do you have a programmatic way to find these? I'd be curious to give Comet a pass, if so.

alamb
alamb approved these changes Sep 18, 2025
View reviewed changes
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @findepi -- I had a suggestion to make the syntax more clear, but this looks like a major improvement to me already

@findepi
Copy link
Member Author

findepi commented Sep 18, 2025

Do you have a programmatic way to find these? I'd be curious to give Comet a pass, if so.

I commented out those expensive From variants and iterated over compile errors.
Those I didn't care about I was replacing with todo!() as DFSchema/Schema to keep the compiler going.
There must be a better way to do this.

@findepi findepi requested a review from alamb September 18, 2025 18:39
@findepi findepi added this pull request to the merge queue Sep 22, 2025
Merged via the queue into apache:main with commit 4ea7601 Sep 22, 2025
28 checks passed
joroKr21
joroKr21 reviewed Sep 26, 2025
View reviewed changes
datafusion/expr/src/logical_plan/builder.rs
}

pub fn table_source(table_schema: &Schema) -> Arc<dyn TableSource> {
// TODO should we take SchemaRef and avoid cloning?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do this in #17675

@findepi findepi deleted the findepi/clones branch September 26, 2025 15:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Dandandan Dandandan Dandandan left review comments

@alamb alamb alamb approved these changes

@xudong963 xudong963 xudong963 approved these changes

+1 more reviewer

@joroKr21 joroKr21 joroKr21 left review comments

Reviewers whose approvals may not affect merge requirements
Assignees
No one assigned
Labels
catalog Related to the catalog crate common Related to common crate core Core DataFusion crate logical-expr Logical plan and expressions performance Make DataFusion faster physical-expr Changes to the physical-expr crates physical-plan Changes to the physical-plan crate proto Related to proto crate sql SQL Planner substrait Changes to the substrait crate
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@findepi @mbutrovich @Dandandan @alamb @joroKr21 @xudong963