Skip to content

Replace concat_elements_utf8_many BufferBuilders with Vec - #10633

Open
cakeni wants to merge 1 commit into
apache:mainfrom
cakeni:perf/arrow-string-concat-many-vecs
Open

Replace concat_elements_utf8_many BufferBuilders with Vec#10633
cakeni wants to merge 1 commit into
apache:mainfrom
cakeni:perf/arrow-string-concat-many-vecs

Conversation

@cakeni

@cakeni cakeni commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

Using Vec instead of BufferBuilder can benefit from Rust's optimized vector implementation. This updates the value and offset builders in concat_elements_utf8_many.

What changes are included in this PR?

  • Replace the many-array output value and offset builders with capacity-matched vectors.
  • Use extend_from_slice and push while preserving the existing offset iteration.
  • Convert the vectors directly into the buffers used by ArrayDataBuilder.
  • Leave the separate two-array concatenation path unchanged.

Are these changes tested?

Yes. The following checks pass:

  • cargo +stable-x86_64-pc-windows-gnu fmt --all -- --check
  • cargo +stable-x86_64-pc-windows-gnu clippy -p arrow-string --all-targets --all-features --no-deps -- -D warnings
  • cargo +stable-x86_64-pc-windows-gnu test -p arrow-string --all-features (182 unit tests and 10 doctests passed)

Are there any user-facing changes?

No.

@github-actions github-actions Bot added arrow Changes to the arrow crate arrow-string labels Aug 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

arrow Changes to the arrow crate arrow-string

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants