Move column vector and host memory helpers to the columnar module#15031
Move column vector and host memory helpers to the columnar module#15031gerashegalov wants to merge 2 commits into
Conversation
0c5a1fb to
ac60c19
Compare
204a929 to
6fbb0ad
Compare
Signed-off-by: Gera Shegalov <gshegalov@nvidia.com>
Signed-off-by: Gera Shegalov <gshegalov@nvidia.com>
ac60c19 to
8288b83
Compare
6fbb0ad to
03678ac
Compare
Greptile SummaryThis PR moves column-vector Java classes and host-memory stream helpers from
Confidence Score: 4/5Safe to merge; this is a pure structural relocation with faithful Scala-to-Java translations and complete call-site updates for the removed convenience overload. All Scala-to-Java translations match their originals exactly, including the pre-existing mark <= 0 boundary in HostMemoryInputStreamMixIn. The only new finding is a redundant limit() override in HostByteBufferIterator. The removal of from(CompressedTable) is complete — the two callers not touched by this diff (SpillFramework.scala, ShuffleBufferCatalog.scala) already used the 2-arg form. HostMemoryInputStream intentionally drops the HostMemoryInputStreamMixIn relationship as part of the stated migration plan. No files require special attention beyond the minor redundant override in HostByteBufferIterator.java. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
subgraph OLD["sql-plugin (before)"]
A1[GpuCompressedColumnVector.java]
A2[HostMemoryInputStream/OutputStream.scala]
A3[AbstractHostByteBufferIterator.scala]
A4[RapidsHostColumnVector classes]
end
subgraph NEW["sql-plugin-columnar (after)"]
B1[GpuCompressedColumnVector.java]
B2[HostMemoryInputStream.java]
B3[HostMemoryOutputStream.java]
B4[NullHostMemoryOutputStream.java]
B5[AbstractHostByteBufferIterator.java]
B6[HostByteBufferIterator.java]
B7[MemoryBufferToHostByteBufferIterator.java]
B8[RapidsHostColumnVector classes]
end
subgraph KEPT["sql-plugin (retained)"]
C1[HostMemoryInputStreamMixIn trait]
C2[GpuPartitioning.scala updated callers]
end
OLD -->|moved/translated| NEW
C2 -->|from buffer meta| B1
Reviews (1): Last reviewed commit: "Move host memory stream helpers to colum..." | Re-trigger Greptile |
Related to #14834.
Description
This PR is one reviewable layer in the unshim stack introduced by #15025. It moves column-vector Java classes and host-memory stream helpers into the columnar helper module. Later caller updates can then depend on the shared columnar module instead of mixed shim-local copies.
Stack context
Testing and validation notes
Checklists
Documentation
Testing
(Covered by the validation notes in the PR description.)
Performance