feat(workers): implement outbox processor for ClickHouse sync

[koxudaxi]

TL;DR

Added an outbox processor for ClickHouse synchronization that handles span data extraction, transformation, and insertion.

What changed?

Created a new outboxProcessor.ts file that implements a robust system for synchronizing span data from PostgreSQL to ClickHouse. The processor:

• Defines types for outbox messages and ClickHouse analytics rows
• Implements utility functions for backoff calculation, error sanitization, and attribute extraction
• Provides a transformation function to convert spans to ClickHouse format
• Implements a main processing function that:
  • Locks outbox rows to prevent duplicate processing
  • Extracts span and trace data from PostgreSQL
  • Transforms data to ClickHouse format
  • Handles batch insertion with retry logic
  • Updates outbox status based on processing results

How to test?

1. Create outbox messages for spans that need to be synchronized
2. Verify the processor correctly extracts and transforms span data
3. Confirm successful insertion into ClickHouse's spans_analytics table
4. Check error handling by simulating failures and verifying retry behavior
5. Verify outbox status updates correctly after processing

Why make this change?

This implementation provides a reliable mechanism for synchronizing span data to ClickHouse for analytics purposes. The outbox pattern ensures data consistency between PostgreSQL and ClickHouse, with features like:

• Exponential backoff for retries
• Error tracking and limiting
• Proper locking to prevent duplicate processing
• Efficient batch processing
• Shared logic that can be used by both queue consumers and polling workers

[koxudaxi]

• feat(workers): implement outbox processor for ClickHouse sync #1811 : 2 dependent PRs (#1812 , #1813 ) 👈 (View in Graphite)
• feat(db): write to outbox on span insertion #1810
• feat(db): add spans_outbox table schema #1809
• feat(clickhouse): implement ClickHouse client service #1808
• feat(docker): add ClickHouse service for local development #1807 : 1 other dependent PR (#1819 )
• feat(clickhouse): add clickhouse migrations and migrate tooling #1849
• v2

This stack of pull requests is managed by Graphite. Learn more about stacking.

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Updated (UTC)
✅ Deployment successful! View logs	v2-docs	a296314	Jan 09 2026, 03:57 AM

[willbakst]

Merge activity

• Jan 9, 1:29 AM UTC: A user started a stack merge that includes this pull request via Graphite.
• Jan 9, 1:29 AM UTC: Graphite couldn't merge this pull request because a downstack PR feat(clickhouse): add clickhouse migrations and migrate tooling #1849 failed to merge.
• Jan 9, 1:40 AM UTC: A user started a stack merge that includes this pull request via Graphite.
• Jan 9, 1:41 AM UTC: Graphite couldn't merge this pull request because a downstack PR feat(clickhouse): add clickhouse migrations and migrate tooling #1849 failed to merge.
• Jan 9, 3:50 AM UTC: @willbakst merged this pull request with Graphite.