Skip to content

feat(#217): add regionless / region-pair sync for inter-region data transfer - #219

Merged
elecnix merged 2 commits into
mainfrom
feat/data-transfer-regionless-sync
Jul 3, 2026
Merged

feat(#217): add regionless / region-pair sync for inter-region data transfer#219
elecnix merged 2 commits into
mainfrom
feat/data-transfer-regionless-sync

Conversation

@elecnix

@elecnix elecnix commented Jul 3, 2026

Copy link
Copy Markdown
Owner

Summary

Follow-up to #218. Wires up DataTransfer-InterRegion-GB — the one #217 metric left seed-only because it didn't fit the region-scoped Infracost sync.

The problem

AWSDataTransfer products are catalogued globally (region="") with a distinct usagetype per source/destination region pair (e.g. USE1-APS4-AWS-Out-Bytes, transferType="InterRegion Outbound", $0.02/GB). The sync queried region="us-east-1" (→ 0 rows) and stored under the product's region (→ ""), so the metric never resolved.

The mechanism (two optional, additive descriptor keys)

  • query_region — query a service's global catalogue (region="") while still storing under the caller's region.
  • region_pair_source (+ usagetype_suffix) — collapse the many region-pair rows to one representative rate via _upsert_region_pair_representative:
    1. keep only rows leaving the sync region (usagetype starts with its REGION_PREFIX short code) matching the suffix (-AWS-Out-Bytes),
    2. drop $0 rows (Local Zones / Wavelength / same-metro pairs),
    3. pick the modal price — the region's standard published rate — and store it once, flat, under the sync region.

For us-east-1 this resolves to the standard $0.02/GB inter-region rate, matching the seed. Existing descriptors are untouched (both keys are optional; default behavior unchanged).

Scope note

Only DataTransfer-InterRegion-GB is wired (its grouping was validated against the live API). DataTransfer-Internet-Out-GB and DataTransfer-InterAZ-GB remain seed-only — they use different transferType/usagetype groupings I haven't live-confirmed; they can adopt the same mechanism in a follow-up.

Test plan (HTTP layer mocked)

  • query_region: "" actually queries the global catalogue (asserts posted region == "").
  • Pair collapse to the modal rate, with source-prefix / suffix / $0 exclusions verified.
  • Graceful no-match → 0 upserts.
  • Descriptor presence.
  • Full suite: 888 passed locally.

Note: the AWSDataTransfer grouping (regionless, transferType="InterRegion Outbound", <PREFIX>-<dst>-AWS-Out-Bytes) was confirmed against the live Cloud Pricing API during development; CI runs fully mocked.

elecnix and others added 2 commits July 3, 2026 11:20
…ransfer

Wire up DataTransfer-InterRegion-GB, the one #217 metric left seed-only because
it didn't fit the region-scoped sync. AWSDataTransfer products are catalogued
globally (region="") with a distinct usagetype per source/destination region
pair (e.g. USE1-APS4-AWS-Out-Bytes, transferType "InterRegion Outbound").

Two small, additive descriptor capabilities:
- `query_region`: query a service's global catalogue (region="") while still
  storing the result under the caller's region.
- `region_pair_source` (+ `usagetype_suffix`): collapse the many region-pair
  rows to one representative rate — keep only rows leaving the sync region
  (usagetype starts with its REGION_PREFIX short code) matching the suffix, drop
  $0 rows (Local Zones / Wavelength / same-metro), pick the modal (standard)
  rate, and store it once, flat, under the sync region.

For us-east-1 this resolves to the standard $0.02/GB inter-region rate, matching
the seed. Existing descriptors are unaffected (both keys are optional).

Tests (HTTP mocked): global-region query, pair collapse to the modal rate with
source-prefix / suffix / $0 exclusions, graceful no-match (0 upserts), and
descriptor presence.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant