Skip to content

Promote edge to stable: Delphi on Postgres behind a switch - #3009

Merged
colinmegill merged 4 commits into
stablefrom
promote-dynamo-switch
Oct 11, 2026
Merged

colinmegill merged 4 commits into
stablefrom
promote-dynamo-switch

Conversation

@colinmegill

@colinmegill colinmegill commented Oct 11, 2026 •

Copy link
Copy Markdown
Member

Ships #3006, #3007 and #3008.

Observable changes: none by default. DELPHI_RESULT_BACKEND defaults to DynamoDB; the Postgres queue path, Postgres readers and writers, and the importer command are present but inactive.

SCHEMA: this release applies three new migrations on deploy (through the migration runner, before old services stop): 000028_create_delphi_results, 000029_extend_delphi_graph_stages, 000030_create_delphi_writers. They create new tables for Delphi results and writers and extend the job-graph stage list; nothing existing is altered beyond that extension.

Rollback: redeploy the previous stable; the added tables are unused by it.

🤖 Generated with Claude Code

https://claude.ai/code/session_018NVGBuYCk4EZmiz4csUv9s

colinmegill and others added 4 commits October 10, 2026 13:45
Lands on its own, as the routing guard requires, before the change that
adds the reader.


Claude-Session: https://claude.ai/code/session_018NVGBuYCk4EZmiz4csUv9s

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Lands on its own, as the routing guard requires, before the change that
adds them.


Claude-Session: https://claude.ai/code/session_018NVGBuYCk4EZmiz4csUv9s

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…witch (#3007)

* Delphi on Postgres: queue-driven runs, run-bound results, readers behind a switch

Delphi runs through the job queue as dependent per-step jobs; each step's
result is stored in Postgres tied to its run. Readers can read from
Postgres behind a switch that defaults to DynamoDB, so merging changes
nothing until the switch is turned on. An importer command copies existing
DynamoDB results; it is not run automatically.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018NVGBuYCk4EZmiz4csUv9s

* Delphi on Postgres: writers follow the switch too

With DELPHI_RESULT_BACKEND=postgres, job submission is admitted to the
Postgres queue and narrative/statement writes go to Postgres, so nothing
writes DynamoDB once the switch is on. The default stays DynamoDB.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018NVGBuYCk4EZmiz4csUv9s

* Delphi on Postgres: fix lint, clippy and replay checks for the writers change

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018NVGBuYCk4EZmiz4csUv9s

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Delphi Coverage Report

File Stmts Miss Cover
init.py 2 0 100%
benchmarks/bench_pca.py 128 107 16%
benchmarks/bench_repness.py 81 65 20%
benchmarks/bench_update_votes.py 38 28 26%
benchmarks/benchmark_utils.py 34 0 100%
components/init.py 1 0 100%
components/config.py 165 70 58%
conversation/init.py 2 0 100%
conversation/conversation.py 1291 264 80%
conversation/group_vote_counts.py 38 2 95%
conversation/manager.py 131 42 68%
database/init.py 1 0 100%
database/dynamodb.py 396 178 55%
database/postgres.py 414 95 77%
delphi_storage/init.py 1 0 100%
delphi_storage/codec.py 295 23 92%
delphi_storage/golden_corpus.py 60 9 85%
delphi_storage/legacy_import.py 168 14 92%
delphi_storage/postgres.py 49 22 55%
delphi_storage/resource.py 176 29 84%
delphi_storage/writer.py 158 17 89%
empty_output.py 29 2 93%
engine_adapter.py 317 265 16%
job_child/init.py 331 50 85%
job_child/census.py 101 8 92%
pca_kmeans_rep/init.py 5 0 100%
pca_kmeans_rep/clusters.py 257 38 85%
pca_kmeans_rep/corr.py 98 17 83%
pca_kmeans_rep/group_k_smoother.py 28 0 100%
pca_kmeans_rep/legacy_kmeans.py 197 8 96%
pca_kmeans_rep/pca.py 162 19 88%
pca_kmeans_rep/repness.py 226 5 98%
poller/init.py 4 0 100%
poller/admission.py 318 15 95%
poller/backfill.py 936 32 97%
poller/capacity.py 485 27 94%
poller/capacity_queue.py 248 13 95%
poller/coordinator_bridge.py 317 317 0%
poller/math_writer.py 113 3 97%
poller/promotion.py 112 4 96%
poller/readiness.py 368 13 96%
poller/rebuild_child.py 133 5 96%
poller/service.py 665 52 92%
poller/worker_pool.py 140 0 100%
queue/executor.py 223 111 50%
regression/init.py 4 0 100%
regression/clojure_comparer.py 188 20 89%
regression/comparer.py 891 640 28%
regression/datasets.py 135 27 80%
regression/recorder.py 36 27 25%
regression/utils.py 130 85 35%
replay/init.py 2 0 100%
replay/attribution_capture.py 49 0 100%
replay/axis_continuity.py 347 24 93%
replay/certify.py 999 59 94%
replay/crosslang.py 128 2 98%
replay/decision_ties.py 55 8 85%
replay/diagnostics.py 50 7 86%
replay/driver.py 73 1 99%
replay/dsn_admission.py 36 1 97%
replay/equiv_query.py 10 0 100%
replay/event_ingress.py 95 1 99%
replay/fixture_bundle.py 815 62 92%
replay/fixture_config.py 266 11 96%
replay/fixture_extract.py 285 12 96%
replay/fixture_generate.py 103 1 99%
replay/fixture_samples.py 157 14 91%
replay/fixture_selection.py 119 0 100%
replay/fixture_survey.py 147 11 93%
replay/legacy_pca.py 29 17 41%
replay/polarity.py 319 25 92%
replay/poller_equiv.py 776 99 87%
replay/prodclone.py 193 2 99%
replay/real_data.py 146 10 93%
replay/schedule.py 205 10 95%
replay/served_math.py 159 4 97%
replay/shard_bench.py 189 65 66%
replay/stagecompare.py 890 161 82%
replay/stages.py 256 40 84%
replay/stepcompare.py 86 1 99%
replay/store.py 120 11 91%
replay/tie_capture.py 107 19 82%
replay/types.py 71 6 92%
run_math_pipeline.py 284 93 67%
tools/engine_rebuild.py 230 11 95%
types.py 22 0 100%
umap_narrative/501_calculate_comment_extremity.py 113 38 66%
umap_narrative/502_calculate_priorities.py 136 136 0%
umap_narrative/700_datamapplot_for_layer.py 502 502 0%
umap_narrative/701_static_datamapplot_for_layer.py 311 311 0%
umap_narrative/702_consensus_divisive_datamapplot.py 435 275 37%
umap_narrative/801_narrative_report_batch.py 666 455 32%
umap_narrative/803_check_batch_status.py 268 148 45%
umap_narrative/llm_factory_constructor/init.py 2 0 100%
umap_narrative/llm_factory_constructor/model_provider.py 234 107 54%
umap_narrative/narrative_data.py 178 65 63%
umap_narrative/numerical_stages.py 54 25 54%
umap_narrative/polismath_commentgraph/init.py 1 0 100%
umap_narrative/polismath_commentgraph/cli.py 270 270 0%
umap_narrative/polismath_commentgraph/core/init.py 3 3 0%
umap_narrative/polismath_commentgraph/core/clustering.py 108 108 0%
umap_narrative/polismath_commentgraph/core/embedding.py 104 104 0%
umap_narrative/polismath_commentgraph/lambda_handler.py 219 219 0%
umap_narrative/polismath_commentgraph/schemas/init.py 2 0 100%
umap_narrative/polismath_commentgraph/schemas/dynamo_models.py 160 9 94%
umap_narrative/polismath_commentgraph/tests/conftest.py 17 17 0%
umap_narrative/polismath_commentgraph/tests/test_clustering.py 74 74 0%
umap_narrative/polismath_commentgraph/tests/test_embedding.py 55 55 0%
umap_narrative/polismath_commentgraph/tests/test_storage.py 87 87 0%
umap_narrative/polismath_commentgraph/utils/init.py 9 7 22%
umap_narrative/polismath_commentgraph/utils/converter.py 283 237 16%
umap_narrative/polismath_commentgraph/utils/group_data.py 332 289 13%
umap_narrative/polismath_commentgraph/utils/storage.py 581 466 20%
umap_narrative/reset_conversation.py 162 53 67%
umap_narrative/run_pipeline.py 337 207 39%
umap_narrative/topic_naming.py 148 11 93%
utils/cli_flags.py 15 0 100%
utils/clj_hash.py 44 0 100%
utils/env_flags.py 13 0 100%
utils/general.py 63 39 38%
utils/output_profile.py 66 2 97%
utils/serialization.py 10 5 50%
utils/vote_convention.py 159 5 97%
Total 24865 7885 68%

@colinmegill
colinmegill merged commit c14a3ec into stable Oct 11, 2026
16 of 18 checks passed
@colinmegill
colinmegill deleted the promote-dynamo-switch branch October 11, 2026 01:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant