feat(optimizer): Support predicate stitching in MaterializedViewRewrite #26728

tdcmeehan · 2025-12-02T03:48:05Z

Note to reviewers:

This PR depends on feat(optimizer): Add support for configurable freshness thresholds for materialized views #26764. Please only review the last commit, and do not merge this PR until the dependent PR has been merged.
While this PR is large, a substantial portion of this PR is in test coverage. Please focus the review on MaterializedViewRewrite, PlanClonerWithVariableMapping and MaterializedViewStitchingUtils, which is more manageable (~`1500 LOCs).

Description

Implement predicate stitching for materialized views in MaterializedViewRewrite. When a materialized view is partially stale, the optimizer can now generate a UNION query that reads fresh data from storage and recomputes only the stale portions from base tables.

Notable changes:

Add USE_STITCHING mode to MaterializedViewStaleReadBehavior
Extend MaterializedViewRewrite to generate UNION plans combining storage reads with recompute branches
Add PlanClonerWithVariableMapping utility for cloning plan trees with variable remapping
Add MaterializedViewStitchingUtils for predicate propagation through join equivalences
Add Iceberg support for tracking changed partitions and generating stale predicates
Add session properties: materialized_view_staleness_window, materialized_view_force_stale (hidden, for testing)
Add Iceberg config: iceberg.materialized-view-max-changed-partitions

Depends on: #26764

Motivation and Context

Fixes #26756

Large materialized views are expensive to fully recompute. When only some base table partitions have changed since the last refresh, this change enables the optimizer to selectively recompute only the stale data rather than either serving stale results or reprocessing terabytes of unchanged data.

Impact

New USE_STITCHING value for materialized_view_stale_read_behavior session property (for default behavior when no table property value is present)
New session properties: materialized_view_staleness_window, materialized_view_force_stale
New Iceberg config: iceberg.materialized-view-max-changed-partitions (default: 100)
Iceberg connector now tracks changed partitions for staleness detection

Test Plan

Added unit tests in TestMaterializedViewStitchingUtils for predicate propagation
Added unit tests in TestPlanClonerWithVariableMapping for plan cloning
Extended TestMaterializedViewRewrite with stitching scenarios
Extended TestIcebergMaterializedViews with end-to-end stitching tests
Extended TestIcebergMaterializedViewOptimizer with many partition tracking tests

Contributor checklist

Please make sure your submission complies with our contributing guide, in particular code style and commit standards.
PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
Documented new properties (with its default value), SQL syntax, functions, or other functionality.
If release notes are required, they follow the release notes guidelines.
Adequate tests were added if applicable.
CI passed.
If adding new dependencies, verified they have an OpenSSF Scorecard score of 5.0 or higher (or obtained explicit TSC approval for lower scores).

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== RELEASE NOTES ==

General Changes
* Add ``USE_STITCHING`` mode for ``materialized_view_stale_read_behavior`` session property to selectively recompute stale data instead of full recomputation.
* Add ``materialized_view_staleness_window`` session property to configure acceptable staleness duration.
* Add ``materialized_view_force_stale`` session property for testing stale read behavior.

Iceberg Connector Changes
* Add ``iceberg.materialized-view-max-changed-partitions`` config property (default: 100) to limit partition tracking for predicate stitching.
* Add support for tracking changed partitions in materialized views to enable predicate stitching optimization.

sourcery-ai

Sorry @tdcmeehan, your pull request is larger than the review limit of 150000 diff characters

steveburnett

Nice work on the documentation! A few nits and suggestions, nothing major.

steveburnett · 2025-12-02T15:56:00Z

presto-docs/src/main/sphinx/admin/materialized-views.rst

+In this example:
+
+* ``orders.order_date`` and ``customers.reg_date`` are equivalent due to the equality join condition
+* Even though ``reg_date`` is not in the MV's SELECT list, staleness can be tracked through the equivalence to ``order_date``


Suggested change

* Even though ``reg_date`` is not in the MV's SELECT list, staleness can be tracked through the equivalence to ``order_date``

* Even though ``reg_date`` is not in the SELECT list, staleness can be tracked through the equivalence to ``order_date``

The all caps MV was jarring. Suggest removing it as not needed because of context supporting the meaning, or maybe "the materialized view's SELECT list".

steveburnett · 2025-12-02T15:56:52Z

presto-docs/src/main/sphinx/admin/materialized-views.rst

+
+**How Passthrough Mapping Works**
+
+1. **Equivalence Extraction**: During MV creation, Presto analyzes JOIN conditions to identify


Same comment as above about MV. Suggest deleting, or spelling it out.

steveburnett · 2025-12-02T15:57:03Z

presto-docs/src/main/sphinx/admin/materialized-views.rst

+* Join must be an INNER JOIN (not LEFT, RIGHT, or FULL OUTER)
+* Equality must be direct (``col1 = col2``), not through expressions like ``col1 = col2 + 1``
+* Both columns must be partition columns in their respective tables
+* At least one column in the equivalence class must be in the MV's output


Same comment as above about MV. Suggest deleting, or spelling it out.

steveburnett · 2025-12-02T16:04:40Z

presto-docs/src/main/sphinx/connector/iceberg.rst


- All refreshes recompute the entire result set
- REFRESH does not provide snapshot isolation across multiple base tables
+- All refreshes recompute the entire result set (incremental refresh not yet supported)


Suggested change

- All refreshes recompute the entire result set (incremental refresh not yet supported)

- All refreshes recompute the entire result set (incremental refresh not supported)

"yet" is an implied promise that should be avoided in documentation.

I thought about suggesting deleting the entire parenthetical "(incremental refresh not yet supported)" as it's arguably implied in "All refreshes recompute the entire result set", but I also find value in the explicit declaration of "incremental refresh not supported" so I could go either way on it and be fine with it staying.

steveburnett · 2025-12-03T15:48:17Z

presto-docs/src/main/sphinx/admin/materialized-views.rst

+3. Partition constraints are built that identify exactly which data is stale
+
+See the connector-specific documentation for details on how staleness is tracked.
+For Iceberg tables, see :doc:`/connector/iceberg` (Materialized Views section).


Suggested change

For Iceberg tables, see :doc:`/connector/iceberg` (Materialized Views section).

For Iceberg tables, see :ref:`connector/iceberg:materialized views`.

Tested improved link in local doc build.

…m view and table properties

sourcery-ai

Sorry @tdcmeehan, your pull request is larger than the review limit of 150000 diff characters

prestodb-ci added the from:IBM PR from IBM label Dec 2, 2025

sourcery-ai bot reviewed Dec 2, 2025

View reviewed changes

steveburnett requested changes Dec 2, 2025

View reviewed changes

steveburnett reviewed Dec 3, 2025

View reviewed changes

tdcmeehan force-pushed the mv-iceberg-disjuncts branch 9 times, most recently from fae20f5 to 188b139 Compare December 10, 2025 21:12

Add support for configurable freshness thresholds for materialized views

aefcddf

tdcmeehan linked an issue Dec 11, 2025 that may be closed by this pull request

Disjunctive Predicate Stitching for Materialized Views and Partition Stitching for Iceberg #26756

Open

Address Sourcery comments

20869b1

tdcmeehan force-pushed the mv-iceberg-disjuncts branch from 188b139 to dc5813d Compare December 11, 2025 18:54

tdcmeehan added 2 commits December 12, 2025 10:56

Use shorter MV property names now that MV properties are distinct fro…

195b482

…m view and table properties

Support predicate stitching in MaterializedViewRewrite

b8e71ad

tdcmeehan force-pushed the mv-iceberg-disjuncts branch from dc5813d to b8e71ad Compare December 12, 2025 18:19

tdcmeehan requested a review from aaneja December 12, 2025 18:28

tdcmeehan marked this pull request as ready for review December 12, 2025 18:28

tdcmeehan requested review from a team, ClarenceThreepwood, ZacBlanco, elharo, feilong-liu, hantangwangd and jaystarshot as code owners December 12, 2025 18:28

prestodb-ci requested a review from a team December 12, 2025 18:28

prestodb-ci requested review from NivinCS and removed request for a team December 12, 2025 18:28

sourcery-ai bot reviewed Dec 12, 2025

View reviewed changes

tdcmeehan marked this pull request as draft December 13, 2025 04:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(optimizer): Support predicate stitching in MaterializedViewRewrite #26728

feat(optimizer): Support predicate stitching in MaterializedViewRewrite #26728

Uh oh!

tdcmeehan commented Dec 2, 2025 •

edited

Loading

Uh oh!

sourcery-ai bot left a comment

Uh oh!

steveburnett left a comment

Uh oh!

steveburnett Dec 2, 2025

Uh oh!

steveburnett Dec 2, 2025

Uh oh!

steveburnett Dec 2, 2025

Uh oh!

steveburnett Dec 2, 2025

Uh oh!

steveburnett Dec 3, 2025

Uh oh!

sourcery-ai bot left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	* Even though ``reg_date`` is not in the MV's SELECT list, staleness can be tracked through the equivalence to ``order_date``
	* Even though ``reg_date`` is not in the SELECT list, staleness can be tracked through the equivalence to ``order_date``


		How Passthrough Mapping Works

		1. Equivalence Extraction: During MV creation, Presto analyzes JOIN conditions to identify

	- All refreshes recompute the entire result set (incremental refresh not yet supported)
	- All refreshes recompute the entire result set (incremental refresh not supported)

	For Iceberg tables, see :doc:`/connector/iceberg` (Materialized Views section).
	For Iceberg tables, see :ref:`connector/iceberg:materialized views`.

feat(optimizer): Support predicate stitching in MaterializedViewRewrite #26728

Are you sure you want to change the base?

feat(optimizer): Support predicate stitching in MaterializedViewRewrite #26728

Uh oh!

Conversation

tdcmeehan commented Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation and Context

Impact

Test Plan

Contributor checklist

Release Notes

Uh oh!

sourcery-ai bot left a comment

Choose a reason for hiding this comment

Uh oh!

steveburnett left a comment

Choose a reason for hiding this comment

Uh oh!

steveburnett Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

steveburnett Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

steveburnett Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

steveburnett Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

steveburnett Dec 3, 2025

Choose a reason for hiding this comment

Uh oh!

sourcery-ai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

tdcmeehan commented Dec 2, 2025 •

edited

Loading