Add kagent db migrate CLI for out-of-band migrations

[iplay88keys]

Description

Adds a kagent db migrate command group for running and inspecting database migrations out-of-band from server startup.

kagent db migrate up                    # apply all pending migrations, every registered source
kagent db migrate status [--output json] # aggregate + per-source applied/pending breakdown
kagent db migrate version               # per-source current versions (--source filters)
kagent db migrate down N  --source core # roll back the N most-recent migrations on one source
kagent db migrate goto V  --source core # move one source to version V (0 = empty schema)
kagent db migrate force V --source core # assert a version without running SQL (recovery)

• New exported packages go/core/pkg/cli/db and go/core/pkg/cli/db/migrate; NewCommand(sources ...migrations.Source) takes the source list explicitly so downstream consumers can wire their own sources.
• The connection comes from --db-url, falling back to the controller's POSTGRES_DATABASE_URL env var. The vector track is gated on the controller's DATABASE_VECTOR_ENABLED env var (default enabled).
• New migrations.WithMigrator export: the CLI opens migrators through the orchestrator's own schema handling, tracking-table config, and advisory-lock identity, so a CLI invocation racing a booting server serializes.
• up, down, and goto refuse a dirty source and report the force invocation that clears it; the CLI never auto-recovers (startup remains the automatic tier). status/version report dirty state instead of refusing.
• force 0 clears the version record entirely (recovery when a track's first migration fails dirty); any other value must be a shipped migration version.
• down on an empty track reports "no migrations to roll back" instead of golang-migrate's raw "file does not exist".
• status --output json has a frozen field shape locked by TestStatusJSONShape.
• Help output for the db subtree hides the root's server-oriented persistent flags (--kagent-url, -n, -o, --timeout, etc.); db commands talk to Postgres directly.

Testing

Version:

❯ ./go/core/bin/kagent-local db migrate version
core: 6
vector: 3

Fresh install:

❯ ./go/core/bin/kagent-local db migrate status
9 migration(s) applied, 0 pending
  core: 6 applied (at v6), 0 pending
  vector: 3 applied (at v3), 0 pending

kagent=# \dt
                   List of tables
 Schema |           Name           | Type  | Owner
--------+--------------------------+-------+--------
⋮
(17 rows)

Down (Rollback):

kagent=# \dt
                   List of tables
 Schema |           Name           | Type  | Owner
--------+--------------------------+-------+--------
⋮
(17 rows)

❯ ./go/core/bin/kagent-local db migrate down 1 --source core 
rolled back 1 migration(s)

❯ ./go/core/bin/kagent-local db migrate status              
8 migration(s) applied, 1 pending
  core: 5 applied (at v5), 1 pending
  vector: 3 applied (at v3), 0 pending

kagent=# \dt
                   List of tables
 Schema |           Name           | Type  | Owner
--------+--------------------------+-------+--------
⋮
(15 rows)

Goto:

❯ ./go/core/bin/kagent-local db migrate goto 1 --source core
schema is at version 1

kagent=# \dt
                   List of tables
 Schema |           Name           | Type  | Owner
--------+--------------------------+-------+--------
 public | memory                   | table | kagent
 public | schema_migrations        | table | kagent
 public | vector_schema_migrations | table | kagent
(3 rows)

Force:

kagent=# select * from schema_migrations;
 version | dirty
---------+-------
       6 | f
(1 row)

❯ ./go/core/bin/kagent-local db migrate force 1 --source core 
version 1 marked as applied

kagent=# select * from schema_migrations;
 version | dirty
---------+-------
       1 | f
(1 row)

Goto + Up:

❯ ./go/core/bin/kagent-local db migrate goto 1 --source core
schema is at version 1

❯ ./go/core/bin/kagent-local db migrate goto 1 --source vector
schema is at version 1

❯ ./go/core/bin/kagent-local db migrate status                
2 migration(s) applied, 7 pending
  core: 1 applied (at v1), 5 pending
  vector: 1 applied (at v1), 2 pending

❯ ./go/core/bin/kagent-local db migrate up                    
applied 7 migration(s); schema is up to date

[Copilot]

Pull request overview

Adds an out-of-band database migration CLI (kagent db migrate) that uses the same migration orchestration and locking behavior as server startup, enabling operators to apply/inspect/rollback migrations without booting the controller.

Changes:

• Export migrations.WithMigrator to open/close a migrator with the orchestrator’s schema/tracking-table/lock configuration.
• Introduce go/core/pkg/cli/db and go/core/pkg/cli/db/migrate implementing up/down/goto/force/status/version, including JSON status output and dirty-state refusal semantics.
• Wire the db command subtree into the main kagent CLI and add unit + container-backed tests for the new migrate commands.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
go/core/pkg/migrations/runner.go	Adds exported WithMigrator helper to share orchestrator migrator setup with CLI tooling.
go/core/pkg/cli/db/migrate/migrate.go	Implements the kagent db migrate command group and subcommands, output, and validation logic.
go/core/pkg/cli/db/migrate/migrate_test.go	Adds unit + Postgres container-backed tests covering the CLI behavior and JSON shape.
go/core/pkg/cli/db/db.go	Adds kagent db parent command and hides root persistent flags in help output for the subtree.
go/core/cli/cmd/kagent/main.go	Wires the new db command into the root CLI and selects built-in migration sources.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[EItanya]

Can we somehow read this from the live cluster rather than requiring information from the user. We can use this as a fallback of course

[iplay88keys]

Good question, I will look into that.

[iplay88keys]

I have a way to look up the kagent's config. The only issue is that if the kubeconfig is pointing to a Kagent which is attached to a different postgres than you are connecting to via --db-url, the looked-up value of VECTOR_ENABLED could silently differ from what's in postgres. I can't detect a mismatch, but I can print it in the cli output as stderr:

❯ go run ./core/cli/cmd/kagent db migrate status   
resolved vector track from cluster context "kind-cluster": configmap kagent/kagent-controller has DATABASE_VECTOR_ENABLED=false (set DATABASE_VECTOR_ENABLED to override)

9 migration(s) applied, 0 pending
  core: 6 applied (at v6), 0 pending
  vector: 3 applied (at v3), 0 pending

[EItanya]

Why do you have this and the check inside of clusterVectorEnabled

[iplay88keys]

They are technically different sources (env var override for the cli vs the deployed controller's configmap key). The env var wins when both are set. I've made that more clear in the code.