[VL]Use Velox's HashTableCache to cache the BHJ's HashTable by JkSelf · Pull Request #12163 · apache/gluten

JkSelf · 2026-05-28T06:12:44Z

What changes are proposed in this pull request?

After facebookincubator/velox#17662 merged, Gluten can directly use the Velox's HashTableCache to cache the pre-built hash table in Velox's HashBuild operator. And then we can remove this commit IBM/velox@40d2554.

How was this patch tested?

Existing unit tests.

Was this patch authored or co-authored using generative AI tooling?

no

github-actions · 2026-05-28T06:13:17Z

Run Gluten Clickhouse CI on x86

github-actions · 2026-06-05T08:40:02Z

Run Gluten Clickhouse CI on x86

Copilot

Pull request overview

This PR wires Velox’s HashTableCache into Gluten’s Velox broadcast hash join (BHJ) path by stabilizing the build-side hash table identifier across AQE wrappers and propagating Spark execution metadata down into the native runtime to support cache scoping/reuse.

Changes:

Canonicalize BHJ build hash table IDs across BroadcastQueryStageExec / ReusedExchangeExec so reuse paths share a stable cache key.
Add Spark SQL execution id propagation (Java → JNI → native) and use it in Velox task/query identifiers.
Integrate Velox HashTableCache injection/drop in native BHJ build/cleanup, and update Velox-side build relation/cache call sites accordingly.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
gluten-ut/spark35/src/test/scala/org/apache/spark/sql/execution/adaptive/velox/VeloxAdaptiveQueryExecSuite.scala	Adjusts AQE test conf to avoid an optimizer rule impacting reuse scenarios.
gluten-substrait/src/main/scala/org/apache/gluten/execution/JoinExecTransformer.scala	Introduces canonical build hash table ID derivation across AQE wrappers and uses it in join parameters.
gluten-arrow/src/main/java/org/apache/gluten/vectorized/PlanEvaluatorJniWrapper.java	Extends JNI kernel creation signature to include Spark execution id.
gluten-arrow/src/main/java/org/apache/gluten/vectorized/NativePlanEvaluator.java	Extracts Spark execution id from task local properties and passes it to JNI.
ep/build-velox/src/get-velox.sh	Changes default Velox repo/branch selection for builds.
cpp/velox/substrait/SubstraitToVeloxPlan.cc	Updates BHJ plan construction to align with Velox-side hash table caching behavior.
cpp/velox/jni/VeloxJniWrapper.cc	Injects built hash tables into Velox `HashTableCache` and updates clone/clear JNI APIs to use cache keys.
cpp/velox/compute/WholeStageResultIterator.cc	Uses Spark execution id for Velox task/query identification.
cpp/core/jni/JniWrapper.cc	Propagates execution id into native `SparkTaskInfo`.
cpp/core/compute/Runtime.h	Extends `SparkTaskInfo` with execution id and updates formatting.
backends-velox/src/main/scala/org/apache/spark/sql/execution/unsafe/UnsafeColumnarBuildSideRelation.scala	Updates hash table “clone” call to pass cache key.
backends-velox/src/main/scala/org/apache/spark/sql/execution/ColumnarBuildSideRelation.scala	Updates hash table “clone” call to pass cache key.
backends-velox/src/main/scala/org/apache/gluten/execution/VeloxBroadcastBuildSideCache.scala	Updates hash table clear to drop by cache key.
backends-velox/src/main/scala/org/apache/gluten/execution/HashJoinExecTransformer.scala	Uses canonical build hash table ID for broadcast-table resource tracking and reuse.
backends-velox/src/main/java/org/apache/gluten/vectorized/HashJoinBuilder.java	Updates native API signatures to include cache key for clone/clear.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@@ -111,11 +124,7 @@ WholeStageResultIterator::WholeStageResultIterator(
  velox::core::PlanFragment planFragment{planNode, velox::core::ExecutionStrategy::kUngrouped, 1, emptySet};
  std::shared_ptr<velox::core::QueryCtx> queryCtx = createNewVeloxQueryCtx();
  task_ = velox::exec::Task::create(
-      fmt::format(
-          "Gluten_Stage_{}_TID_{}_VTID_{}",
-          std::to_string(taskInfo_.stageId),
-          std::to_string(taskInfo_.taskId),
-          std::to_string(taskInfo.vId)),
+      getVeloxTaskId(taskInfo_),
      std::move(planFragment),
      0,


 CURRENT_DIR=$(cd "$(dirname "$BASH_SOURCE")"; pwd)
-VELOX_REPO=https://github.com/IBM/velox.git
-VELOX_BRANCH=dft-2026_06_04
-VELOX_ENHANCED_BRANCH=ibm-2026_06_04
+VELOX_REPO=https://github.com/JkSelf/velox.git
+VELOX_BRANCH=dft-2026_06_04-hashtable-cache
+VELOX_ENHANCED_BRANCH=ibm-2026_06_04-hashtable-cache


github-actions · 2026-06-05T08:57:42Z

Run Gluten Clickhouse CI on x86

github-actions · 2026-06-08T06:48:25Z

Run Gluten Clickhouse CI on x86

Copilot

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 5 comments.

+VELOX_REPO=https://github.com/JkSelf/velox.git
+VELOX_BRANCH=dft-2026_06_05-hashtable-cache
+VELOX_ENHANCED_BRANCH=ibm-2026_06_05-hashtable-cache


+std::string getVeloxTaskId(const SparkTaskInfo& taskInfo) {
+  if (taskInfo.executionId != -1) {
+    return fmt::format("Gluten_Execution_{}", std::to_string(taskInfo.executionId));
+  }
+  return fmt::format(
+      "Gluten_Stage_{}_TID_{}_VTID_{}",
+      std::to_string(taskInfo.stageId),
+      std::to_string(taskInfo.taskId),
+      std::to_string(taskInfo.vId));
+}


+    const auto& hashTableId = sJoin.hashtableid();
+    const auto joinNodeId = hashTableId.empty() ? nextPlanNodeId() : hashTableId;
    // Create HashJoinNode node
    return std::make_shared<core::HashJoinNode>(
-        nextPlanNodeId(),
+        joinNodeId,


  // Unique ID for builded hash table
-  lazy val buildHashTableId: String = "BuiltHashTable-" + buildPlan.id
+  lazy val buildHashTableId: String = canonicalBuildHashTableId(buildPlan)



  // Unique ID for built table
-  lazy val buildBroadcastTableId: String = buildPlan.id.toString
+  lazy val buildBroadcastTableId: String = canonicalBuildHashTableId(buildPlan)



github-actions · 2026-06-09T03:09:15Z

Run Gluten Clickhouse CI on x86

github-actions · 2026-06-09T04:29:23Z

Run Gluten Clickhouse CI on x86

Copilot

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 3 comments.

+std::string getVeloxTaskId(const SparkTaskInfo& taskInfo) {
+  if (taskInfo.executionId != -1) {
+    return fmt::format("Gluten_Execution_{}", std::to_string(taskInfo.executionId));
+  }
+  return fmt::format("Gluten_Execution_{}", "");
+}


    val executionId = sparkContext.getLocalProperty(SQLExecution.EXECUTION_ID_KEY)
+    var cacheKey = ""
    if (executionId != null) {
+      cacheKey = "Gluten_Execution_" + executionId + ":" + buildHashTableId
      GlutenDriverEndpoint.collectResources(executionId, buildBroadcastTableId)


+VELOX_REPO=https://github.com/JkSelf/velox.git
+VELOX_BRANCH=dft-2026_06_05-hashtable-cache
+VELOX_ENHANCED_BRANCH=ibm-2026_06_05-hashtable-cache


github-actions · 2026-06-09T06:40:10Z

Run Gluten Clickhouse CI on x86

github-actions · 2026-06-10T06:52:46Z

Run Gluten Clickhouse CI on x86

Copilot

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 6 comments.

+std::string getVeloxTaskId(const SparkTaskInfo& taskInfo) {
+  if (taskInfo.executionId != -1) {
+    return fmt::format("Gluten_Execution_{}", std::to_string(taskInfo.executionId));
+  }
+  return fmt::format("Gluten_Execution_{}", "");
+}



+    size_t pos = hashTableId.find(':');
+    auto planNodeId = hashTableId.substr(pos + 1);
+    std::cout << "the cacheKey is " << hashTableId << " the planNode id is " << planNodeId << std::endl;


+    size_t pos = hashTableId.find(':');
+    auto planNodeId = hashTableId.substr(pos + 1);
+    std::cout << "the cacheKey is " << hashTableId << " the planNode id is " << planNodeId << std::endl;
+    const auto joinNodeId = hashTableId.empty() ? nextPlanNodeId() : planNodeId;
+


  const auto abandonHashBuildDedupMinPct =
      queryConf.get<uint32_t>(kAbandonDedupHashMapMinPct, kAbandonDedupHashMapMinPctDefault);
  const auto hashTableId = jStringToCString(env, tableId);
+  std::cout << "the cacheKey in nativeBuild is " << hashTableId << std::endl;


+VELOX_REPO=https://github.com/JkSelf/velox.git
+VELOX_BRANCH=dft-2026_06_05-hashtable-cache
+VELOX_ENHANCED_BRANCH=ibm-2026_06_05-hashtable-cache


  enableSuite[GlutenTableScanSuite]
+    .includeByPrefix("Caching")


github-actions · 2026-06-10T11:40:46Z

Run Gluten Clickhouse CI on x86

github-actions · 2026-07-06T03:28:56Z

Run Gluten Clickhouse CI on x86

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

cpp/velox/substrait/SubstraitToVeloxPlan.cc:466

joinNodeId is computed but never used, and it calls nextPlanNodeId() when hashtableid is empty. That consumes an extra plan node id and can shift IDs for subsequent nodes, which can break plan determinism (and also triggers unused-variable warnings). Use joinNodeId as the HashJoinNode id (or remove it) to avoid the extra nextPlanNodeId() call.

    const auto& hashTableId = sJoin.hashtableid();
    const auto joinNodeId = hashTableId.empty() ? nextPlanNodeId() : hashTableId;
    // Create HashJoinNode node
    return std::make_shared<core::HashJoinNode>(
        nextPlanNodeId(),
        joinType,

        getJoinOutputType(leftNode, rightNode, joinType),
+        true,
        false,
        false,
-        joinHasNullKeys,
-        opaqueSharedHashTable);
+        nullptr,
+        sJoin.hashtableid());


  override def hashJoinType: JoinType = joinType

-  lazy val buildHashTableId: String = buildPlan.id.toString
+  // Unique ID for builded hash table


github-actions · 2026-07-08T03:25:59Z

Run Gluten Clickhouse CI on x86

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

+    const auto& hashTableId = sJoin.hashtableid();
+    const auto joinNodeId = hashTableId.empty() ? nextPlanNodeId() : hashTableId;
    // Create HashJoinNode node


  override def hashJoinType: JoinType = joinType

-  lazy val buildHashTableId: String = buildPlan.id.toString
+  // Unique ID for builded hash table


github-actions · 2026-07-08T05:22:30Z

Run Gluten Clickhouse CI on x86

github-actions · 2026-07-08T05:25:18Z

Run Gluten Clickhouse CI on x86

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

+    const auto& hashTableId = sJoin.hashtableid();
+    const auto joinNodeId = hashTableId.empty() ? nextPlanNodeId() : hashTableId;
    // Create HashJoinNode node
    return std::make_shared<core::HashJoinNode>(
        nextPlanNodeId(),


  override def hashJoinType: JoinType = joinType

-  lazy val buildHashTableId: String = buildPlan.id.toString
+  // Unique ID for builded hash table


Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

@@ -489,10 +471,11 @@ core::PlanNodePtr SubstraitToVeloxPlanConverter::toVeloxPlan(const ::substrait::
        leftNode,
        rightNode,
        getJoinOutputType(leftNode, rightNode, joinType),
+        true,
        false,
        false,
-        joinHasNullKeys,
-        opaqueSharedHashTable);
+        nullptr,
+        sJoin.hashtableid());


  override def hashJoinType: JoinType = joinType

-  lazy val buildHashTableId: String = buildPlan.id.toString
+  // Unique ID for builded hash table


github-actions · 2026-07-09T03:01:08Z

Run Gluten Clickhouse CI on x86

Copilot

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.

github-actions Bot added CORE works for Gluten Core BUILD VELOX labels May 28, 2026

Copilot AI review requested due to automatic review settings June 5, 2026 08:39

JkSelf force-pushed the gluten-hashtable-cache branch from 63a6382 to 10aec74 Compare June 5, 2026 08:39

Copilot started reviewing on behalf of JkSelf June 5, 2026 08:39 View session

Copilot AI reviewed Jun 5, 2026

View reviewed changes

JkSelf force-pushed the gluten-hashtable-cache branch from 10aec74 to 581f7e5 Compare June 5, 2026 08:56

JkSelf force-pushed the gluten-hashtable-cache branch from 581f7e5 to 824da36 Compare June 8, 2026 06:41

Copilot AI review requested due to automatic review settings June 8, 2026 06:41

Copilot started reviewing on behalf of JkSelf June 8, 2026 06:41 View session

Copilot AI reviewed Jun 8, 2026

View reviewed changes

JkSelf force-pushed the gluten-hashtable-cache branch from 824da36 to 2f34754 Compare June 9, 2026 03:08

Copilot AI review requested due to automatic review settings June 9, 2026 04:28

JkSelf force-pushed the gluten-hashtable-cache branch from 2f34754 to 677658b Compare June 9, 2026 04:28

Copilot started reviewing on behalf of JkSelf June 9, 2026 04:29 View session

Copilot AI reviewed Jun 9, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings June 10, 2026 06:52

Copilot started reviewing on behalf of JkSelf June 10, 2026 06:52 View session

Copilot AI reviewed Jun 10, 2026

View reviewed changes

JkSelf force-pushed the gluten-hashtable-cache branch from 96e888a to 67b5711 Compare June 11, 2026 04:35

Copilot AI review requested due to automatic review settings July 6, 2026 03:28

Copilot started reviewing on behalf of JkSelf July 6, 2026 03:28 View session

Copilot AI reviewed Jul 6, 2026

View reviewed changes

JkSelf added 2 commits July 8, 2026 04:25

Use Velox's HashTableCache to cache the BHJ's HashTable

eb2cfce

fix

db2fac4

Copilot AI review requested due to automatic review settings July 8, 2026 03:25

JkSelf force-pushed the gluten-hashtable-cache branch from 05e2388 to fe50505 Compare July 8, 2026 03:25

Copilot started reviewing on behalf of JkSelf July 8, 2026 03:25 View session

Copilot AI reviewed Jul 8, 2026

View reviewed changes

fix

f902109

Copilot AI review requested due to automatic review settings July 8, 2026 05:22

JkSelf force-pushed the gluten-hashtable-cache branch from fe50505 to f902109 Compare July 8, 2026 05:22

Copilot started reviewing on behalf of JkSelf July 8, 2026 05:22 View session

format

fe2423b

Copilot AI reviewed Jul 8, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings July 8, 2026 05:28

Copilot started reviewing on behalf of JkSelf July 8, 2026 05:28 View session

Copilot AI reviewed Jul 8, 2026

View reviewed changes

JkSelf changed the title ~~[VL][TEST]Use Velox's HashTableCache to cache the BHJ's HashTable~~ [VL]Use Velox's HashTableCache to cache the BHJ's HashTable Jul 8, 2026

refine

48cd516

Copilot AI review requested due to automatic review settings July 9, 2026 03:00

Copilot started reviewing on behalf of JkSelf July 9, 2026 03:01 View session

Copilot AI reviewed Jul 9, 2026

View reviewed changes

zhztheplayer approved these changes Jul 9, 2026

View reviewed changes

JkSelf merged commit 0f8074c into apache:main Jul 9, 2026
101 of 102 checks passed

		enableSuite[GlutenTableScanSuite]
		.includeByPrefix("Caching")

Uh oh!

Conversation

JkSelf commented May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes are proposed in this pull request?

How was this patch tested?

Was this patch authored or co-authored using generative AI tooling?

Uh oh!

github-actions Bot commented May 28, 2026

Uh oh!

github-actions Bot commented Jun 5, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

github-actions Bot commented Jun 5, 2026

Uh oh!

github-actions Bot commented Jun 8, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

github-actions Bot commented Jun 9, 2026

Uh oh!

github-actions Bot commented Jun 9, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

github-actions Bot commented Jun 9, 2026

Uh oh!

github-actions Bot commented Jun 10, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

github-actions Bot commented Jun 10, 2026

Uh oh!

github-actions Bot commented Jul 6, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

github-actions Bot commented Jul 8, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

github-actions Bot commented Jul 8, 2026

Uh oh!

github-actions Bot commented Jul 8, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

github-actions Bot commented Jul 9, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

JkSelf commented May 28, 2026 •

edited

Loading