feat: Derive pg_partman partition width from ttl unit and partition-divisor setting

[WellMafra]

Reworked per review feedback from @ferenc-csaky and @mbroecheler: instead of adding a PG-specific second argument to the engine-agnostic ttl hint, the pg_partman partition width is now derived automatically from the TTL duration and a postgres-specific partition-divisor engine setting.

How the partition width is picked

For range-partitioned tables with ttl(X unit) and the postgres setting partition-divisor (P, default 100):

X_min     = TTL normalized to minutes
Floor_min = 1 unit (as declared in the ttl hint) in minutes
menu      = [15 min, 30 min, 1h, 2h, 4h, 6h, 8h, 12h, 1d, 2d, 4d, 1w, 2w, 4w, 8w, 12w]  # calendar-aligned
W_target  = max(X_min / P, Floor_min)
W         = largest menu value <= W_target, else smallest menu value

The divisor caps the partition count for long TTLs, while the unit the user declares the TTL with sets the floor: ttl(14 days) produces 1-day partitions, ttl(48 hours) produces hourly partitions. Retention is still always the TTL duration.

Key Changes

• TtlHint stays a single-argument hint and now captures the declared unit. Units are validated to be between minute and week (min, hour, day, week + short/plural forms); seconds and below or months and above raise StatementParserException with the source location. The cache(...) hint is unchanged.
• New postgres engine setting partition-divisor (default 100), read in PostgresJdbcEngine and validated as a positive integer.
• AbstractJdbcStatementFactory gains an overridable derivePartitionInterval(partitionType, ttl, ttlUnit) hook (null by default); PostgresStatementFactory implements the formula above for RANGE tables.
• CreateTableJdbcStatement.partitionInterval now holds the computed width (still omitted from JSON when null); PgPartmanExtension uses it directly and the old TTL-bucketing deriveInterval fallback is removed.
• Documented the unit constraint in sqrl-language.md and the partition-divisor setting + partitioning behavior in configuration-engine/postgres.md.
• Tests: TtlHintTest rewritten for unit parsing/validation, new PostgresStatementFactoryTest covering the formula (floor dominance, divisor scaling, menu snapping, smallest-entry fallback, config parsing), updated PgPartmanExtensionTest, and partitionTest.sqrl extended with a ttl(48 hours) table (snapshot regenerated).

Behavior change

Derived partition widths change for existing TTLs: e.g. ttl(30 days) previously produced 1 week partitions and now produces 1 day (floor-driven), per the agreed default handling.

[codecov]

Codecov Report

❌ Patch coverage is 96.42857% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 17.93%. Comparing base (98a02a8) to head (23afed9).
⚠️ Report is 1 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
...abase/relational/AbstractJdbcStatementFactory.java	80.00%	1 Missing ⚠️
...engine/database/relational/PostgresJdbcEngine.java	0.00%	1 Missing ⚠️
...c/main/java/com/datasqrl/planner/hint/TtlHint.java	97.77%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #2236      +/-   ##
============================================
+ Coverage     16.91%   17.93%   +1.01%     
- Complexity     1040     1105      +65     
============================================
  Files           613      613              
  Lines         17738    17802      +64     
  Branches       2155     2161       +6     
============================================
+ Hits           3001     3193     +192     
+ Misses        14425    14291     -134     
- Partials        312      318       +6     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

[ferenc-csaky]

I don't think this solution is viable as is. The ttl hint is orthogonal to the external system, and now we are introducing a PG-specific parameter to it, which violates the contract of this hint, i.e. it works regardless what external data system connection that table has. Cause TTL is also meaningful for Kafka for example.

It makes sense to control this on the table level, but i may add a new pg_partman hint for this maybe?

Or another idea to extend the engine hint to engine-specific configs, which is a more general construct, but would solve this problem too.

cc @mbroecheler

[mbroecheler]

@WellMafra I agree with @ferenc-csaky reasoning. We don't want this engine specific logic in the generic TTL hint. Moreover, I feel like it is too much to ask our users to fine-tune this variable. Instead, we should have better default handling that works for 99% of cases. Here is what I propose:

We have a postgres specific partition-divisor setting. Let's call that number P. When a user configured ttl(X unit) -meaning a TTL of X with the given time unit - then we set the partition width W with the same time unit using the following formula:

X_min = normalize(X, unit)
Floor_min = normalize(1, unit)
menu = [15 min, 30min, 1h, 2h, 4h, 6h, 8h, 12h, 1d, 2d, 4d, 1w, 2w, 4w, 8w, 12w]        # ensure calendar-alignment
W_target = max(X_min / P, Floor_min)
W = largest menu value (in min) <= W_target, else smallest menu value

This allows the user to control the number of partitions (via config option) and the floor (via the unit they use).
We should validate that the unit used for the TTL is at least min and at most week.

The default value for partition-divisor should be 100.

That means, a ttl(14 days) would result in a partition width of 1 day (driven by the Floor_min).