Skip to content

feat: aes encrypt support #3211

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Shekharrajak wants to merge 14 commits into
base: main
Choose a base branch
from
Open

feat: aes encrypt support #3211

Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
ca01860
add crypto dependencies and implement crypto_utils with tests
Shekharrajak Jan 17, 2026
7748b8a
implement ECB, CBC, and GCM cipher modes with tests
Shekharrajak Jan 17, 2026
20a7bbb
implement aes_encrypt with vectorized processing and tests
Shekharrajak Jan 17, 2026
92ce310
register aes_encrypt function in scalar function registry
Shekharrajak Jan 17, 2026
8147ebc
add scala serde layer for aes_encrypt expression
Shekharrajak Jan 17, 2026
d4d53d7
add encryption benchmark for aes_encrypt performance testing
Shekharrajak Jan 17, 2026
6c658ff
fix aes_encrypt serde to handle StaticInvoke properly
Shekharrajak Jan 18, 2026
6a9d2f8
fix: pass return type in scalar function serialization
Shekharrajak Jan 18, 2026
bd897f9
test: add CometStaticInvokeSuite for aes_encrypt
Shekharrajak Jan 18, 2026
b0e52a8
fix: address clippy warnings in encryption code
Shekharrajak Jan 18, 2026
ec407ff
Delete benchmarks/CometDPPBenchmark-jdk17-results.txt
Shekharrajak Jan 19, 2026
52dea17
fix: address clippy warnings in encryption code
Shekharrajak Jan 18, 2026
da932ca
fix: remove invalid AesEncrypt.enabled config entry
Shekharrajak Jan 20, 2026
f3ca5f2
fix: update auto-generated compatibility docs formatting
Shekharrajak Jan 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions spark/src/main/scala/org/apache/comet/serde/CometScalarFunction.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,15 @@ package org.apache.comet.serde
import org.apache.spark.sql.catalyst.expressions.{Attribute, Expression}

import org.apache.comet.serde.ExprOuterClass.Expr
import org.apache.comet.serde.QueryPlanSerde.{exprToProtoInternal, optExprWithInfo, scalarFunctionExprToProto}
import org.apache.comet.serde.QueryPlanSerde.{exprToProtoInternal, optExprWithInfo, scalarFunctionExprToProtoWithReturnType}

/** Serde for scalar function. */
case class CometScalarFunction[T <: Expression](name: String) extends CometExpressionSerde[T] {
override def convert(expr: T, inputs: Seq[Attribute], binding: Boolean): Option[Expr] = {
val childExpr = expr.children.map(exprToProtoInternal(_, inputs, binding))
val optExpr = scalarFunctionExprToProto(name, childExpr: _*)
// Pass return type to avoid native lookup in DataFusion registry
Copy link
Contributor Author

@Shekharrajak Shekharrajak Jan 18, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When expr.return_type is None, it tries to look up the function in DataFusion's built-in UDF registry , which doesn't have aes_encrypt because it's a Comet-specific function registered later via create_comet_physical_fun

val optExpr =
scalarFunctionExprToProtoWithReturnType(name, expr.dataType, false, childExpr: _*)
optExprWithInfo(optExpr, expr, expr.children: _*)
}
}