Skip to content

[BUILD] iceberg and iceberg-test profiles collide on one build-helper execution id, silently dropping src-iceberg/test sources #12933

Description

@LuciferYang

What happens

Both profiles in the root pom declare a build-helper-maven-plugin execution with the id add-iceberg-test-sources: pom.xml:1833 under iceberg, pom.xml:1884 under iceberg-test. Maven merges plugin executions by id, so when both profiles are active only one <sources> list survives.

help:effective-pom with -Pbackends-clickhouse -Pspark-3.5 -Pspark-ut -Pdelta -Piceberg -Piceberg-test resolves that execution to:

add-iceberg-test-sources
    src-iceberg-spark35/test/scala
    src-iceberg-spark35/test/java

The iceberg profile's four entries (src-iceberg/test/{scala,java} and src-iceberg${iceberg.binary.version}/test/{scala,java}) are gone. No warning is printed; the sources are simply not compiled.

Effect on ClickHouse: the combination cannot build

ClickHouseIcebergHiveTableSupport lives in backends-clickhouse/src-iceberg/test/scala, and both Java tests under src-iceberg-spark35/test/java instantiate it. The class therefore leaves the source path at exactly the moment the tests join it:

$ ./build/mvn -Pbackends-clickhouse -Pspark-3.5 -Pscala-2.13 -Pspark-ut -Pdelta \
      -Piceberg -Piceberg-test -DskipTests test-compile -pl backends-clickhouse
[ERROR] .../src-iceberg-spark35/test/java/.../TestPositionDeletesTableGluten.java:[41,18] cannot find symbol
[ERROR]   symbol:   class ClickHouseIcebergHiveTableSupport

Giving the iceberg-test execution a distinct id (add-iceberg-spark-test-sources) and changing nothing else makes the same command exit 0 and emit both ClickHouseIcebergHiveTableSupport.class and TestPositionDeletesTableGluten.class.

Effect on Velox: harmless today, which is why nobody noticed

Nothing in backends-velox needs both directories at once. src-iceberg/test holds VeloxIcebergSuite and VeloxTPCHIcebergSuite, which run in the jobs that pass -Piceberg without -Piceberg-test (velox_backend_x86.yml:941 and :1013, whose wildcardSuites includes org.apache.gluten), plus a @EnhancedFeaturesTest variant that velox_backend_enhanced.yml covers, a JUnit test surefire picks up, and one helper object. The 52 files under src-iceberg-spark34/test run in the three groups at :817, :821, :825 that pass both profiles.

Suggested fix

Rename the iceberg-test profile's execution so the two lists no longer overwrite each other. Both profiles are always passed together in every in-repo invocation, so the merged list is what the plugin was meant to produce.

Two things a reviewer will want to know about the blast radius. The three x86 groups and the 4.0 test-compile at :1277 would newly compile src-iceberg/test/**, so those suites run twice across the matrix; expect longer jobs rather than new failures. And src-iceberg${iceberg.binary.version}/test is empty in every module, so nothing else appears on the source path.

Only backends-velox and backends-clickhouse hold files under src-iceberg*/test, so the change is confined to those two modules.

This blocks the ClickHouse Iceberg Java test work in #12934.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions