-
Notifications
You must be signed in to change notification settings - Fork 1.2k
feat(billing): time-based cutover for HTTP↔S3 event-name suffix swap #6705
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 2 commits
21abfe7
93d90b1
290b323
8f7c551
9140129
a1cc334
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,11 +2,21 @@ import { uuidv7 } from 'uuidv7'; | |
|
|
||
| import { Err, Ok } from '@nangohq/utils'; | ||
|
|
||
| import { envs } from '../../envs.js'; | ||
| import { putOrbCustomerSchema } from './types.js'; | ||
|
|
||
| import type { BillingAddress, BillingCustomer, BillingEvent, BillingInvoicingDetails, Result, UsageMetric } from '@nangohq/types'; | ||
| import type Orb from 'orb-billing'; | ||
|
|
||
| // True once the S3-fed pipeline is authoritative for billing (see | ||
| // BILLING_EVENTS_CUTOVER_AT in packages/utils). While this returns false, | ||
| // HTTP-emitted events stay unsuffixed so customer billable metrics keep | ||
| // matching them. Once it flips true, HTTP events pick up the "_http" | ||
| // suffix and become the defensive shadow. | ||
| function cutoverActive(): boolean { | ||
| return !!envs.BILLING_EVENTS_CUTOVER_AT && new Date() >= new Date(envs.BILLING_EVENTS_CUTOVER_AT); | ||
| } | ||
|
|
||
| export function toOrbEvent(event: BillingEvent): Orb.Events.EventIngestParams.Event { | ||
| const { idempotencyKey, timestamp, accountId, ...rest } = event.properties; | ||
|
|
||
|
|
@@ -23,8 +33,9 @@ export function toOrbEvent(event: BillingEvent): Orb.Events.EventIngestParams.Ev | |
| } | ||
| } | ||
|
|
||
| const suffix = cutoverActive() ? (envs.BILLING_EVENTS_HTTP_EVENT_NAME_SUFFIX ?? '') : ''; | ||
| return { | ||
| event_name: event.type, | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. can we make the cut-over time-based? Then you don't have anything to do, just be in front of your computer and confirm correct events are being injested
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. good call, but still I might want to have a flag for disabling it just in case .... let me work on this |
||
| event_name: `${event.type}${suffix}`, | ||
| idempotency_key: idempotencyKey || uuidv7(), | ||
| external_customer_id: accountId.toString(), | ||
| timestamp: timestamp.toISOString(), | ||
|
|
||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -349,7 +349,23 @@ export const ENVS = z.object({ | |||||
| BILLING_INGEST_MAX_RETRY: z.coerce.number().optional().default(3), | ||||||
| BILLING_EVENTS_S3_BUCKET: z.string().optional(), | ||||||
| BILLING_EVENTS_S3_WRITER_ROLE_ARN: z.string().optional(), | ||||||
| // Temporary. Applied to S3-emitted Orb event_name only BEFORE the | ||||||
| // S3-fed cutover (i.e. while BILLING_EVENTS_CUTOVER_AT is unset or in | ||||||
| // the future). After the cutover, S3 events land unsuffixed under the | ||||||
| // canonical event names. | ||||||
| BILLING_EVENTS_S3_EVENT_NAME_SUFFIX: z.string().optional(), | ||||||
| // Temporary. Applied to HTTP-fed Orb event_name only AFTER the | ||||||
| // S3-fed cutover (i.e. once BILLING_EVENTS_CUTOVER_AT has passed) so | ||||||
| // HTTP emissions remain queryable as a defensive backup. Remove once | ||||||
| // the HTTP emission path is retired. | ||||||
| BILLING_EVENTS_HTTP_EVENT_NAME_SUFFIX: z.string().optional(), | ||||||
| // Temporary. ISO 8601 timestamp at which the S3-fed pipeline becomes | ||||||
| // authoritative for billing. Before this instant, S3 events are | ||||||
| // shadow (suffixed) and HTTP events are canonical (unsuffixed). At | ||||||
| // and after this instant, the two swap roles. Unset (or set to a | ||||||
| // future date) to defer or roll back the cutover. Remove once the | ||||||
| // HTTP emission path is retired. | ||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. not sure you need environment variables for the suffix. they are never gonna be dynamic and always be
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. yes, good point. Ill keep the S3 one bc was already there and there was some divergence btw dev an prod, but the HTTP one can be removed and hardcoded |
||||||
| BILLING_EVENTS_CUTOVER_AT: z.string().datetime().optional(), | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P2: Valid ISO 8601 cutover timestamps with explicit offsets can make the service fail env validation, because Prompt for AI agents
Suggested change
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We have this config which if Im not mistaken is a correct value
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That value looks valid as a Z timestamp. The parent comment was about explicit-offset ISO 8601 values, so it still applies to that case. |
||||||
| BILLING_EVENTS_S3_REGION: z.string().optional().default('us-west-2'), | ||||||
|
|
||||||
| // ClickHouse | ||||||
|
|
||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P3: The
cutoverActive()cutover-active check is duplicated across two packages (billingandmetering). If the cutover comparison ever needs to change — logging, timezone handling, grace-period logic, or date format — both copies must be updated in sync. Consider extracting a shared helper into@nangohq/utils(or a billing-adjacent shared module) so the two packages share one source of truth.Prompt for AI agents