Repository navigation
Conversation
The invoices page recounts workflow_executions and direct_executions live on every load, and the only durable copy of a billed figure is written in one place that returns early for a period inside its plan and earlier still for a plan with overage disabled. A free-plan organization therefore never gets a row, and production holds exactly one. Once the run-row retention pass starts deleting executions past its flat window, a past invoice shows a smaller number than we charged -- and not zero, because that pass never touches direct_executions. Adds execution_usage_periods: one row per organization per closed period with the plan, tier and execution limit as they were, the two execution halves kept apart, and the charge. It is written when a period closes, before any of the gates that decide not to raise a charge, and read back for closed periods so only the open one is still counted live. Organizations with a provider period are recorded when that cycle closes. Everyone else is billed on the UTC calendar month and closed by a sweep on the existing twice-daily overage scan. That sweep drives from a single usage aggregate rather than the subscriptions table: on production 1,088 of 1,525 organizations have no subscription row and 397 of those run executions, so enumerating subscriptions would skip 490 of the 892 org-months that should exist. It also makes the close four queries instead of two per organization. A backfill writes what is still reconstructible: provider invoice periods where there is a customer, calendar months otherwise. KEEP-1338
Staging now owns migrations 0157-0161, so the Pyth checkpoint migration moves to 0162_pyth_trigger_checkpoints with a journal entry after 0161_workflow_state and a snapshot regenerated against current staging. The SQL body is unchanged.
checkDispatchAdmission reads plan and usage through the default db pool. Called inside the observation transaction, it took a second pooled connection while the transaction held one plus the FOR UPDATE lock on the workflow row, so concurrent crossings could exhaust the pool and the row lock was held across those reads. Admission now resolves before the transaction opens, from an unlocked read of the workflow, and only for observe commands. A decision that goes stale before the lock is taken is harmless: the executor re-runs the execution-limit and plan-feature checks before it claims the row.
The registry minted a new sessionId on every reconnect attempt, and observePythPrice refuses any session other than the stored one while the 45s lease is live. After every stream drop the process was locked out of its own workflows for up to 45s: every price in that window came back leased, a transient crossing inside it was lost, and a sustained one fired about 45s late. A feed quiet long enough to trip the 20s read timeout could settle into roughly one accepted observation per lease. The session identified two different things: which replica owns the workflow, and whether to rebaseline. The session is now per registry instance, so a reconnect keeps the lease, and rebaselining is an explicit flag on the observation. The registry sends it for each workflow until the server records a baseline on the new connection, so an out_of_order or stale first price after a reconnect does not use up the request. A different session or a lapsed lease still rebaselines as before.
The 20s abort timer was armed before reader.read() and not cleared until the next iteration, so it also covered the onPrice fan-out, which ran one observation after another. Around 600 workflows on one feed the fan-out alone exceeded 20s, the connection aborted and reconnected with nothing identifying the cause, and the socket was not read while the fan-out ran. The timer now covers reader.read() only. Reading no longer waits on delivery: prices that arrive while one is being delivered replace each other, and delivery resumes on the newest, so a slow fan-out neither backs up the socket nor ages a queue of prices past maxAgeSeconds. The connection does not return while a delivery is in flight, so two connections never interleave observations for one workflow. Fan-out runs at most four observations at a time per feed. Two related fixes in the same loop: a malformed frame is logged and skipped rather than tearing down the connection for every workflow on the feed, and the reconnect backoff resets only after a connection has stayed up for 60s, not on any delivered price.
…execution-usage feat(billing): persist per-period execution usage
Staging took 0162 for 0162_keep_1338_execution_usage_periods, so the Pyth checkpoint migration moves to 0163_pyth_trigger_checkpoints with a journal entry after it and a snapshot regenerated against current staging. The SQL body is unchanged.
6b30c10 converted CodeEditor to a next/dynamic boundary and made Monaco configuration an awaited, idempotent promise. None of the Pyth trigger, its settings panel or the /pyth page imports the code editor or Monaco, and the change alters how every code editor in the app loads, including a new loading placeholder on first open. It should be reviewed as its own pull request. This restores components/ui/code-editor.tsx and lib/monaco-loader-config.ts to staging and removes code-editor-implementation.tsx, its test and the two showcase fixtures recorded as evidence for it. The change stays available in 6b30c10.
Only the internal observation route had a test for its disabled behaviour. The gates that keep the trigger out of reach without the key - POST /api/workflows/create, the enabling PATCH, and the trigger list in the published action schemas - were untested, and the existing PATCH case sets the key so it can reach config validation. Each new case removes the key and fails if the gate is bypassed.
The tracker polled getSignaturesForAddress every 2s per watched program (code default). Set SOLANA_SIGNATURES_POLL_INTERVAL_MS=10000 in the staging and prod values to cut that RPC volume 5x.
Registration read the first trigger node, the feature gate matched any node whose config said Pyth Price, and MCP validation ran its own loop over trigger nodes. A Pyth config on a node registration never reads was blocked by the gate and validated by MCP but could never fire. findPythTriggerNode is now the single definition - the first trigger node, when its triggerType is Pyth Price - and findPythConfig, hasPythPriceTrigger and validateWorkflow all resolve through it.
buildExecutorInput threw whenever triggerInput.triggerType was "upstream". That field is caller-writable: the MCP call route passes the request body straight through (validateInputSchema allows extra fields), and the executor's manual and webhook paths spread message.input over triggerType. The call route builds the input synchronously after the execution row and payment are committed, outside start()'s catch, so a crafted body stranded rows that count against the owner's concurrency limit. The same check also ran after the executor had claimed the row and, for PAYG-overflow orgs, charged it, so a genuine signal that expired while its runner pod started became a failed run rather than a skipped one. Expiry and configuration are still enforced where they are authoritative: the executor checks signed upstream messages before claim and records a refusal as skipped, and the internal execute route checks upstream dispatches before starting them.
The create route checked only whether Pyth is available, but it accepts enabled: true. A workflow created enabled with an invalid config (a rearm threshold on the wrong side, a non-decimal threshold) got a 201, and discovery then skipped it with only a log line, so the user had an enabled trigger that would never fire. PATCH already rejects the same config with INVALID_PYTH_TRIGGER when enabling. Create now applies the same rule: an incomplete draft is still saved, but a workflow created enabled must carry a registrable config.
Two rules rebaselined a workflow that had simply been sampled slowly: evaluatePythPrice treated a gap of maxAgeSeconds or more between updates as a baseline, and observePythPrice treated its own lapsed 45s lease the same way. Once a feed's fan-out took that long, every observation was a baseline and the trigger silently never fired. Neither rule protected anything the stale check does not: the price being evaluated is still required to be fresh. A baseline now comes only from a first observation, an explicit reconnect rebaseline, or a different session taking over. The description agents read and the settings copy said crossings missed during an outage are not replayed. Since the armed state is carried through a rebaseline so a reconnect cannot re-fire a trigger, an armed trigger whose price crossed during the gap fires on the update after the baseline. That is the intended behaviour - a routine reconnect lasts seconds, and disarming there would drop the whole excursion - so the copy now says so, and the test that claimed otherwise now asserts the update after the baseline.
Admission moved out of the observation transaction by reading the workflow and running checkDispatchAdmission before every observe, but its answer is only used when a price crosses. Every tick paid an extra read of the workflow's nodes for a value it almost always discarded. The transaction now runs without it. When a crossing needs a decision, it throws PythAdmissionRequired, the rollback releases the connection and the row lock, admission resolves outside any transaction, and the observation is replayed once with the answer. Ticks that do not cross pay nothing, and admission still never runs under the lock. The checkpoint update before evaluation ran on every observe, pending recovery and ack, almost always writing back what it had just read. It now runs only when an expired or reconfigured pending signal, a config change or an ack actually changed the checkpoint.
joelorzet
added
the
metrics-db-reviewed
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.