Skip to content

Conversation

kabo87777
Copy link

Fix Non-Deterministic Behavior in ColumnInjectionPushDownTest

Problem

Two tests were failing non-deterministically under NonDex with ~50% failure rate:

  • testCannotPushDownTimeJoinAlignByTime
  • testPartialPushDownTimeJoinAlignByDevice

Reproduce

cd iotdb-core/datanode
mvn edu.illinois:nondex-maven-plugin:2.1.1:nondex \
  -Dtest=ColumnInjectionPushDownTest#testCannotPushDownTimeJoinAlignByTime \
  -DnondexRuns=4

Failure example:
java.lang.AssertionError: expected:ColumnInjectNode@79d58e5b but was:ColumnInjectNode@d7938b9b

Root Cause

  1. HashMap in test utilities - Non-deterministic iteration order caused plan nodes to be constructed differently across runs
  2. Order-sensitive comparison in FullOuterTimeJoinNode - Children were compared as ordered lists, but join order doesn't affect semantic meaning

Solution

  1. Test utilities: Replaced HashMap with LinkedHashMap to preserve insertion order

    • OptimizationTestUtil.schemaMap
    • OptimizationTestUtil.getAggregationDescriptor()
    • All deviceToMeasurementIndexesMap in tests
  2. Production code: Implemented order-independent equals() in FullOuterTimeJoinNode

    • Children are now compared as sets rather than ordered lists
    • Semantically correct since full outer time join order doesn't matter

Key Changed Classes

  • OptimizationTestUtil - HashMap → LinkedHashMap
  • ColumnInjectionPushDownTest - HashMap → LinkedHashMap
  • TestPlanBuilder - HashMap → LinkedHashMap
  • FullOuterTimeJoinNode - Order-independent equals()

This PR has:

  • been self-reviewed.
    • concurrent read
    • concurrent write
    • concurrent read and write
  • added documentation for new or modified features or behaviors.
  • added Javadocs for most classes and all non-trivial methods.
  • added or updated version, license, or notice information
  • added comments explaining the "why" and the intent of the code wherever would not be obvious
    for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold
    for code coverage.
  • added integration tests.
  • been tested in a test IoTDB cluster.

Key changed/added classes (or packages if there are too many classes) in this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant