Skip to content

[C++] Fix vtable deduplication for 64-bit buffers >2GB #8591

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

gowsiany
Copy link

@gowsiany gowsiany commented May 2, 2025

Fixes #8590

Summary

This PR corrects vtable deduplication for FlatBuffers using 64-bit offsets. Previously, when serializing buffers larger than 2GB, vtable deduplication failed, resulting in much slower serialization and larger buffer sizes.

Root Cause

The issue was caused by 64-bit offset calculations not accounting for length_of_64_bit_region_. As a result, the addresses of vtable candidates stored in scratch_data were miscalculated, preventing proper deduplication.

Solution

This fix includes length_of_64_bit_region_ in the offset calculation for vtables in 64-bit mode. This ensures correct vtable candidate addresses, enabling deduplication and restoring efficient serialization for large buffers.

Impact

  • Restores vtable deduplication for buffers >2GB with 64-bit offsets.
  • Significantly improves serialization performance and reduces buffer size for large data sets.
  • Maintains full backward compatibility with 32-bit buffers.

Verification

  • Serialization benchmarks with large (2GB+) buffers now show linear scaling and reduced buffer size.

@github-actions github-actions bot added the c++ label May 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[C++] Vtable deduplication fails for 64-bit buffers >2GB, causing slow serialization and larger buffers
1 participant