feat(indexer): test SQLite + Postgres backend matrix and document selection

.github/workflows/ci.yml

@@ -282,6 +282,56 @@ jobs:
       - name: Run issuer tests (incl. cross-boundary circuit test)
         run: pnpm --filter @stellarcred/issuer test
 
+  indexer:
+    # The indexer's DB layer supports BOTH SQLite and Postgres, so the same
+    # test matrix must run against both backends or one silently rots. This job
+    # spins up a Postgres 16 service container so the parameterized DB suite
+    # (services/indexer/src/db.test.ts) exercises migrations, upserts, revokes,
+    # cursor updates, etc. on Postgres while also still running the SQLite leg.
+    name: Indexer tests (SQLite + Postgres)
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: services/indexer
+    services:
+      postgres:
+        image: postgres:16
+        env:
+          POSTGRES_USER: indexer
+          POSTGRES_PASSWORD: indexer
+          POSTGRES_DB: indexer
+        ports:
+          - 5432:5432
+        # Gate job steps until Postgres is accepting connections.
+        options: >-
+          --health-cmd "pg_isready -U indexer -d indexer"
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 5
+    steps:
+      - uses: actions/checkout@v7
+
+      - uses: actions/setup-node@v7
+        with:
+          node-version: 20
+          cache: npm
+          cache-dependency-path: services/indexer/package-lock.json
+
+      - name: Install dependencies
+        run: npm ci
+
+      - name: Build
+        run: npm run build
+
+      - name: Run DB test matrix (SQLite + Postgres) and unit tests
+        run: npm test
+        env:
+          # Dummy contract ID / RPC so the ingester tests run headlessly.
+          PROOF_REGISTRY_CONTRACT_ID: C000000000000000000000000000000000000000000000000000000000000001
+          RPC_URL: https://soroban-testnet.stellar.org
+          # Point the Postgres leg of the DB matrix at the service container.
+          TEST_POSTGRES_URL: postgres://indexer:indexer@localhost:5432/indexer
+
   sdk-integration:
     name: SDK integration tests (testnet)
     runs-on: ubuntu-latest

services/indexer/.env.example

@@ -1,6 +1,13 @@
+# Backend selection: "sqlite" (default; embedded, zero-infra, single-writer) or
+# "postgres" (standalone DB, required for multi-instance production deployments).
+# See README "Database Backend Selection & Tradeoffs".
 DB_DRIVER=sqlite
 SQLITE_PATH=./data/indexer.db
+# Required when DB_DRIVER=postgres, otherwise ignored.
 DATABASE_URL=postgres://indexer:indexer@localhost:5432/indexer
+# Used by the DB test matrix (src/db.test.ts) to exercise the Postgres backend
+# locally; falls back to DATABASE_URL. Ignored at runtime.
+TEST_POSTGRES_URL=postgres://indexer:indexer@localhost:5432/indexer
 
 RPC_URL=https://soroban-testnet.stellar.org
 NETWORK_PASSPHRASE=Test SDF Network ; September 2015

services/indexer/README.md

@@ -133,6 +133,69 @@ the first page; a `null` `nextCursor` means there are no more claims. `limit`
 
 ---
 
+## Database Backend Selection & Tradeoffs
+
+The indexer is a thin storage layer over one of two backends, selected at
+startup with the `DB_DRIVER` environment variable:
+
+| `DB_DRIVER` | Engine          | Connection config            | Best for |
+|---|---|---|---|
+| `sqlite` (default) | better-sqlite3 (`journal_mode=WAL`) | `SQLITE_PATH` | local dev, demos, single-instance / hobby deployments |
+| `postgres` | node-postgres pool | `DATABASE_URL` | production multi-instance deployments |
+
+**How selection works.** `loadConfig()` reads `DB_DRIVER` (defaulting to
+`sqlite`) and validates it. `createDb()` then returns the matching adapter and
+runs the schema migrations for that engine. `DATABASE_URL` **must** be set
+when `DB_DRIVER=postgres` (and is ignored by the SQLite adapter). Everything
+above the adapter — the ingester and the HTTP API — is backend-agnostic and
+talks only to the `Db` interface, so adding a new backend means implementing
+that interface, not touching the business logic.
+
+**Tradeoffs.**
+
+- **Operational scale** — SQLite is embedded in the process (zero
+  infrastructure,  single file, WAL for concurrent readers) and is perfect for
+  local development and single-instance nodes. Postgres is a standalone
+  service that supports concurrent writers and many readers, which is what a
+  multi-instance / horizontally-scaled deployment needs.
+- **Concurrency** — SQLite allows a single writer process; if you run more than
+  one indexer instance against the same SQLite file you can corrupt/resolve the
+  cursor incorrectly. Postgres serializes writes with row-level locking and a
+  shared cursor row.
+- **Operational tooling** — Postgres gives you replication, backups, managed
+  hosting, and point-in-time recovery out of the box; SQLite needs your own
+  file-backup strategy.
+- **Dependency footprint** — SQLite (via `better-sqlite3`) adds a native
+  module to `node_modules`; the Postgres driver (`pg`) is pure JS. Choose the
+  default (`sqlite`) unless you actually need Postgres's scaling and tooling.
+
+> **Recommendation:** run `sqlite` in development and single-instance
+> production; enable `postgres` only when you need multiple reader/writer
+> instances or managed database tooling.
+
+**Testing both backends.** The worker test suite runs the **same DB test
+matrix against SQLite and Postgres** (`src/db.test.ts`). Coverage includes
+schema migrations (idempotency), ledger-cursor updates, claim upserts,
+revokes, `claimsByWallet`, `stats`, paginated `recent`, `deleteClaimsAfter`
+and `getMaxClaimLedger`. The SQLite leg always runs locally; the Postgres leg
+runs in CI (via the `postgres` service container in `.github/workflows/ci.yml`)and locally whenever `TEST_POSTGRES_URL` (or `DATABASE_URL`) points at a live
+Postgres, and is skipped otherwise:
+
+```bash
+# SQLite leg only (no Postgres reachable):
+npm test
+
+# Both legs, against a local Postgres, e.g. `docker run ... -p 5432:5432 postgres`: 
+TEST_POSTGRES_URL=postgres://user:pass@localhost:5432/db npm test
+```
+
+Because the two engines use different SQL dialects (`INSERT OR IGNORE` vs
+`ON CONFLICT`, `INTEGER` vs `BIGINT`), the matrix is exactly where silent
+cross-backend divergences surface (e.g. Postgres returning `BIGINT` columns as
+strings) — running it on both is how we keep either backend from rotting.
+
+---
+
 ## Consistency, Finality & Reorg Guarantees
 
 - **Cursor Progression**: The indexer stores the last successfully processed ledger sequence in database metadata. In the event of a restart, ingestion resumes seamlessly from the saved checkpoint without skipping events.
@@ -147,7 +210,8 @@ the first page; a `null` `nextCursor` means there are no more claims. `limit`
 # Install dependencies
 npm install
 
-# Run unit and integration tests
+# Run unit and integration tests (SQLite by default; add TEST_POSTGRES_URL
+# to also exercise the Postgres backend — see "Database Backend Selection")
 npm test
 
 # Build TypeScript to dist/