Skip to content

Conversation

VGalaxies
Copy link
Contributor

reopen #16435

@VGalaxies VGalaxies requested a review from Copilot October 13, 2025 08:31
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Enable Pipe to accept multiple tree path patterns by introducing a union pattern representation and updating parsing, selection, and tests.

  • Add UnionTreePattern to represent a union of multiple TreePattern instances and extend parser to build unions from comma-separated inputs (backtick-aware).
  • Update parsing semantics to unionize source.path and source.pattern when both are provided; adjust logic to avoid scan parsers for unions.
  • Add integration and unit tests covering multiple prefix/IoTDB/hybrid patterns for historical and realtime data; minor test name fixes.

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/pipe/source/IoTDBNonDataRegionSource.java Temporarily unwraps UnionTreePattern to first pattern, affecting multi-pattern support.
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/pipe/datastructure/pattern/UnionTreePattern.java New union pattern type and utilities (toString, matching, etc.).
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/pipe/datastructure/pattern/TreePattern.java Refactor to parse multiple patterns, introduce helper methods, and move single-pattern state to SingleTreePattern.
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/pipe/datastructure/pattern/SingleTreePattern.java New base class encapsulating single-pattern behavior and defaults.
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/pipe/datastructure/pattern/PrefixTreePattern.java Switch to extend SingleTreePattern.
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/pipe/datastructure/pattern/IoTDBTreePattern.java Switch to extend SingleTreePattern.
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/pipe/source/IoTDBDataRegionSourceTest.java Add assertions for comma-separated patterns and backtick-escaped commas.
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/event/common/tsfile/parser/TsFileInsertionEventParserProvider.java Avoid scan parser paths when a UnionTreePattern is used.
integration-test/src/test/java/org/apache/iotdb/pipe/it/dual/treemodel/auto/basic/IoTDBTreePatternFormatIT.java Add integration tests for multiple patterns; fix method names and semantics where both path and pattern are provided.

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@VGalaxies VGalaxies requested a review from Copilot October 14, 2025 08:27
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 4 comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.


// The statements do not contain AlterLogicalViewStatements
// Here we apply the statements as many as possible
// Even if there are failed statements
Copy link

Copilot AI Oct 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This unchecked cast assumes the built TreePattern is always a UnionIoTDBTreePattern. To make this robust, either construct a UnionIoTDBTreePattern directly (e.g., map to List and new UnionIoTDBTreePattern(...)) or guard with an instanceof check and fail fast with a clear error if a non-IoTDB union is received.

Suggested change
// Even if there are failed statements
// Even if there are failed statements
if (!(treePattern instanceof UnionIoTDBTreePattern)) {
throw new IllegalArgumentException(
"Expected treePattern to be UnionIoTDBTreePattern, but got: "
+ treePattern.getClass().getName());
}

Copilot uses AI. Check for mistakes.

@VGalaxies VGalaxies requested a review from Caideyipi October 14, 2025 10:24
/ PipeDataNodeResourceManager.memory().getTotalNonFloatingMemorySizeInBytes()
> PipeTsFilePublicResource.MEMORY_SUFFICIENT_THRESHOLD
&& treePattern.isSingle()) {
return new TsFileInsertionEventScanParser(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why "ScanParser" must use single?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure whether the implementation of scan parser will conflict with multi patterns?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Surely not. Just check whether the device or measurement fits the pattern one by one.

Copy link
Collaborator

@Caideyipi Caideyipi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants