Skip to content

feat: Support watermark status propagation#62

Merged
lgbo-ustc merged 7 commits into
gluten-0530from
feature/watermark-status-propagation
Jul 3, 2026
Merged

feat: Support watermark status propagation#62
lgbo-ustc merged 7 commits into
gluten-0530from
feature/watermark-status-propagation

Conversation

@lgbo-ustc

@lgbo-ustc lgbo-ustc commented Jun 30, 2026

Copy link
Copy Markdown

Summary

  • add native WatermarkStatus stream element propagation
  • track combined idle/active watermark status across inputs
  • expose StatefulTask/StatefulOperator APIs for watermark status notifications
  • add CombinedWatermarkStatus unit coverage for active/idle transitions

releated PRs

Validation

  • git diff --check
  • cmake --build build --target velox_stateful_combined_watermark_status_test -j 8
  • ./build/velox/experimental/stateful/tests/velox_stateful_combined_watermark_status_test
  • velox4j C++/Java compile passed after updating velox4j to this Velox change

Note

  • standalone velox_stateful_exec build was previously blocked because the environment was missing xsimd/xsimd.hpp through unrelated stateful dependencies; the new CombinedWatermarkStatus test target avoids those unrelated dependencies and passed.

@lgbo-ustc lgbo-ustc changed the title Support watermark status propagation feat: Support watermark status propagation Jun 30, 2026
@lgbo-ustc
lgbo-ustc force-pushed the feature/watermark-status-propagation branch from e93063e to 83d1771 Compare July 2, 2026 06:19
}

// If the watermark is already set, we do not update it.
bool CombinedWatermarkStatus::updateStatus(int index, bool idle) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

int -> int32_t

bool updateWatermark(int index, int64_t timestamp);

// idle == true means WatermarkStatus.IDLE; false means ACTIVE.
bool updateStatus(int index, bool idle);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

@lgbo-ustc

Copy link
Copy Markdown
Author

Addressed review feedback: changed int to int32_t for the watermark index parameters across the call chain (CombinedWatermarkStatus, StatefulOperator, StatefulTask) in commit bb19102.

@lgbo-ustc
lgbo-ustc force-pushed the feature/watermark-status-propagation branch from 7a312db to d267ca9 Compare July 3, 2026 01:44
@lgbo-ustc
lgbo-ustc merged commit 21023e0 into gluten-0530 Jul 3, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants