List view

Clean up stage fallback rules

enhancementNew feature or requestNew feature or request

Status: Closed (completed).

#3984 In apache/datafusion-comet;

· by andygrove was closed on Apr 20, 2026

Avoid JVM shuffle when parent stage will just convert back to rows

enhancementNew feature or requestNew feature or request

performance

requires-triage

Status: Closed (completed).

#4004 In apache/datafusion-comet;

· by andygrove was closed on Apr 25, 2026

[EPIC] Add support for Spark 4.0.1

enhancementNew feature or requestNew feature or request

help wantedExtra attention is neededExtra attention is needed

priority:highCrashes, panics, segfaults, major functional breakageCrashes, panics, segfaults, major functional breakage

spark 4.0

Status: Closed (completed).

#1637 In apache/datafusion-comet;

· by andygrove was closed on Apr 28, 2026

Change Comet default Spark version to 4.0

enhancementNew feature or requestNew feature or request

priority:lowMinor issues, test failures, tooling, cosmeticMinor issues, test failures, tooling, cosmetic

spark 4.0

Status: Closed (completed).

#4054 In apache/datafusion-comet;

· by andygrove was closed on May 7, 2026

Add scripts to automate release process

enhancementNew feature or requestNew feature or request

Status: Closed (completed).

#1419 In apache/datafusion-comet;

· by andygrove was closed on Apr 25, 2026

Reinstate macOS CI builds of Comet with Spark 4.0.0

enhancementNew feature or requestNew feature or request

spark 4.0

Status: Closed (completed).

#1949 In apache/datafusion-comet;

· by andygrove was closed on Apr 28, 2026

[EPIC] Iceberg compatibility fixes

enhancementNew feature or requestNew feature or request

EPIC

Status: Closed (completed).

#2060 In apache/datafusion-comet;

· by andygrove was closed on May 6, 2026

Investigate OpenDAL features

enhancementNew feature or requestNew feature or request

Status: Closed (completed).

#2367 In apache/datafusion-comet;

· by andygrove was closed on Apr 25, 2026

Add support for Spark 4.0.x + Iceberg 1.9.x

enhancementNew feature or requestNew feature or request

spark 4.0

Status: Closed (completed).

#2380 In apache/datafusion-comet;

· by andygrove was closed on May 7, 2026

Improve documentation for configs related to explain/fallback

documentationImprovements or additions to documentationImprovements or additions to documentation

enhancementNew feature or requestNew feature or request

good first issueGood for newcomersGood for newcomers

Status: Closed (completed).

#2413 In apache/datafusion-comet;

· by andygrove was closed on Apr 27, 2026

Include GLIBC version with provided jars

enhancementNew feature or requestNew feature or request

Status: Not planned (skipped).

#2504 In apache/datafusion-comet;

· by lfdversluis was closed on Apr 25, 2026

Use older version of GLIBC when building official Comet jars

enhancementNew feature or requestNew feature or request

Status: Closed (completed).

#2511 In apache/datafusion-comet;

· by andygrove was closed on Apr 25, 2026

Remove mutable Parquet buffer use from native_iceberg_compat

enhancementNew feature or requestNew feature or request

Status: Closed (completed).

#2152 In apache/datafusion-comet;

· by andygrove was closed on Apr 25, 2026

Shuffle memory pool accounting: peak RSS significantly exceeds configured memory limit

area:shuffleShuffle (JVM and native)Shuffle (JVM and native)

Status: Closed (completed).

#3821 In apache/datafusion-comet;

· by andygrove was closed on May 6, 2026

Explore zero-copy batch coalescing during shuffle write

area:shuffleShuffle (JVM and native)Shuffle (JVM and native)

performance

Status: Not planned (skipped).

#3836 In apache/datafusion-comet;

· by andygrove was closed on May 6, 2026

Experiment: immediate-mode shuffle

enhancementNew feature or requestNew feature or request

priority:lowMinor issues, test failures, tooling, cosmeticMinor issues, test failures, tooling, cosmetic

Status: Closed (completed).

#3855 In apache/datafusion-comet;

· by andygrove was closed on Apr 25, 2026

[native_iceberg_compat] VariantShreddingSuite test failures with Spark 4.0.0

bugSomething isn't workingSomething isn't working

native_iceberg_compatSpecific to native_iceberg_compat scan typeSpecific to native_iceberg_compat scan type

priority:criticalData corruption, silent wrong results, security issuesData corruption, silent wrong results, security issues

spark 4.0

Status: Closed (completed).

#2209 In apache/datafusion-comet;

· by andygrove was closed on Apr 26, 2026

native_datafusion: STRING column read as INT silently returns garbage values

correctness

priority:criticalData corruption, silent wrong results, security issuesData corruption, silent wrong results, security issues

Status: Closed (completed).

#4088 In apache/datafusion-comet;

· by andygrove was closed on Apr 27, 2026

native_datafusion: incompatible decimal precision/scale read silently succeeds without value validation

correctness

priority:criticalData corruption, silent wrong results, security issuesData corruption, silent wrong results, security issues

Status: Closed (completed).

#4089 In apache/datafusion-comet;

· by andygrove was closed on Apr 27, 2026

Add Java 21 to the supported version matrix (particularly for Spark 4.0)

area:ciCI/CD, GitHub Actions, build toolingCI/CD, GitHub Actions, build tooling

good first issueGood for newcomersGood for newcomers

priority:lowMinor issues, test failures, tooling, cosmeticMinor issues, test failures, tooling, cosmetic

spark 4.0

Status: Closed (completed).

#4059 In apache/datafusion-comet;

· by james-willis was closed on Apr 29, 2026

fix: [Spark 4.1] preserve union output partitioning in CometUnionExec

correctness

spark 4.1

Status: Merged (completed).

apache/datafusion-cometnumber 4207

#4207 In apache/datafusion-comet;

· by andygrove was closed on May 6, 2026

feat: support AQE DPP broadcast reuse for Iceberg native scans

Status: Merged (completed).

apache/datafusion-cometnumber 4215

#4215 In apache/datafusion-comet;

· by mbutrovich was closed on May 6, 2026

fix(spark-expr): preserve scalar tag in WideDecimalBinaryExpr when both inputs are scalars

Status: Merged (completed).

apache/datafusion-cometnumber 4187

#4187 In apache/datafusion-comet;

· by tomz was closed on May 8, 2026

fix: prevent native sort crash for Struct(Map(...)) keys

Status: Merged (completed).

apache/datafusion-cometnumber 4157

#4157 In apache/datafusion-comet;

· by 0lai0 was closed on May 7, 2026

build: change default Maven profile to Spark 4.1 / Scala 2.13

Status: Merged (completed).

apache/datafusion-cometnumber 4140

#4140 In apache/datafusion-comet;

· by andygrove was closed on May 7, 2026