feat: DMS connectors — iManage and NetDocuments behind a pluggable interface

[amal66]

Summary

Law firms keep their documents in a document management system — iManage or NetDocuments — and forcing every file through a manual upload breaks how they actually work. This PR teaches Mike to read documents where they live: a user connects their DMS once (OAuth), then browses/searches it and pulls a document straight into a project. The imported file flows through the exact same upload pipeline as a drag-and-drop (documents row + V1 version, Office→PDF rendition), is tagged source: "dms_import", and remembers where it came from so its edited version can be pushed back to the DMS as a new version.

The vendors sit behind a small pluggable DMSConnector interface with a kind-keyed registry — the DMS analog of the existing MCP connector machinery — so a new vendor is one adapter class + one registerDmsAdapter call, with no caller changes. A deterministic in-memory FakeDMSAdapter backs the test suite and doubles as an egress-free demo backend.

Changes

• backend/src/lib/dms/adapter.ts — the DMSConnector contract: authenticate / listFolders / search / fetchDocument (bytes + metadata + version) / exportDocument (with versioning) / checkReady.
• backend/src/lib/dms/index.ts — kind-keyed adapter registry (getDmsAdapter / registerDmsAdapter / listDmsAdapters), cloud-kind classification.
• backend/src/lib/dms/fake.ts — in-memory FakeDMSAdapter (deterministic, no egress).
• backend/src/lib/dms/imanage.ts, netdocuments.ts — cloud adapters (iManage Work REST /api/v2, NetDocuments /v2 cabinets), isolated behind the interface.
• backend/src/lib/dms/http.ts — shared egress helpers; every outbound request goes through the MCP guardedFetch (HTTPS-only, private-IP SSRF guard, redirect: "manual"), downloads capped at the upload size ceiling.
• backend/src/lib/dms/oauth.ts — auth-code + refresh OAuth (PKCE S256, hashed one-time state, 60s refresh skew), structurally mirroring lib/mcp/oauth.ts; credentials from IMANAGE_OAUTH_* / NETDOCS_OAUTH_* env vars.
• backend/src/lib/dms/crypto.ts — re-exports the exact MCP encryptString/decryptString; no new crypto.
• backend/src/lib/dms/servers.ts — connector CRUD, adapter resolution, sync/search/import/export orchestration, air-gap gating, project authorization via checkProjectAccess, SSRF validation of the tenant base URL on save (same as createUserMcpConnector).
• backend/src/lib/dms/import.ts — import/export wiring into the upload pipeline + dms_document_links provenance rows.
• backend/src/lib/dms/types.ts, backend/src/lib/dmsConnectors.ts — row/summary types and the stable barrel export (mirrors lib/mcpConnectors.ts).
• backend/src/lib/airgap.ts — 8-line isAirgapped() helper; cloud DMS kinds are refused when AIRGAPPED=true (the Fake stays usable).
• backend/src/routes/user.ts — /user/dms-connectors routes (list/get/create/patch/delete, oauth start/callback, sync, search, import), same auth posture and error idiom as the neighboring /user/mcp-connectors routes (requireAuth on reads, + requireMfaIfEnrolled on writes, unauthenticated state-validated OAuth callback).
• backend/src/routes/documents.ts — exported createDocumentFromUpload: the initial-upload pipeline (same steps as handleDocumentUpload) callable outside an Express handler, parameterized on the version source so DMS imports land as dms_import. Existing handlers untouched.
• backend/src/index.ts — startup alias: DMS_CONNECTORS_ENCRYPTION_SECRET → MCP_CONNECTORS_ENCRYPTION_SECRET when the latter is unset, so operators can name the secret after the feature without a second crypto path.
• backend/migrations/20260717_01_dms_connectors.sql — dms_connectors, dms_connector_oauth_tokens, dms_connector_oauth_states (column-for-column mirrors of the user_mcp_* tables), dms_document_links, 'dms_import' added to the document_versions.source check. RLS enabled, anon/authenticated revoked, guarded/re-runnable.
• backend/schema.sql — same tables + source-check value, kept in lockstep with the migration.
• backend/src/lib/dms/__tests__/ — 5 deterministic Vitest suites (36 tests) + an in-memory Supabase stand-in (fakeDb.ts).
• backend/tsconfig.json — exclude __tests__/*.test.ts from the production build (same two-line change as the other test-bearing PRs).

Why

• Pluggable, not vendor-locked: the interface/registry pattern mirrors how the codebase already isolates storage and MCP connectors, so iManage and NetDocuments are implementations, not assumptions.
• No new dependencies: runtime code uses only existing modules (lib/mcp/client.ts crypto + guarded fetch, lib/storage, lib/convert, lib/access, node crypto). backend/package.json is untouched.
• Zero behavior change until configured — stated precisely: no existing route, table, or pipeline changes behavior. The new /user/dms-connectors endpoints exist but are inert until a user creates a connector; the in-memory fake kind works without any env (it is the test/demo backend and has no egress); the cloud kinds additionally require per-vendor OAuth env credentials (IMANAGE_OAUTH_* / NETDOCS_OAUTH_*) before they can do anything. The migration is additive (new tables + one additional allowed enum value in the document_versions.source check).
• Live-tenant caveat (also in code comments): the cloud adapters' endpoint paths, response envelopes, and version semantics are best-effort from public API docs and are proven only against mocked DNS + HTTP. Real-tenant validation needs OAuth client credentials, the tenant base URL, and library/cabinet IDs — an operator acceptance step, not CI.

Testing

• cd backend && npm install && npm run build — tsc green as committed.
• With the test harness (vitest devDeps + script) merged locally: npx vitest run src/lib/dms — 5 files, 36/36 passed (Fake adapter contract + registry swap; iManage and NetDocuments against mocked dns/promises + fetch — auth, folders, search, fetch-with-metadata/version, export-new-version, redirect:"manual"/SSRF private-IP rejection; OAuth start/callback/refresh with encrypted-at-rest assertions; air-gap refusal per kind).
• Full backend suite with harness: 6 files, 48/48 passed.
• Frontend untouched (the fork ships no DMS UI surface).

Provenance

All added lines are mechanical ports of amal66/mike@origin/main (commit b3166dd): apps/api/src/lib/dms/** and apps/api/src/lib/dmsConnectors.ts → backend/src/lib/ (14 files byte-identical), fork migration 20260701000004_dms_connectors.sql → backend/migrations/20260717_01_dms_connectors.sql (statements verbatim; header comment re-pointed at this repo's migration filenames), route hunks from the fork's user.routes.ts/user.dms.ts re-expressed in this repo's monolithic routes/user.ts idiom (inline try/catch + console.error, matching the adjacent MCP routes), and the fork's createDocumentFromUpload (modules/documents/documents.upload.ts) → routes/documents.ts. Exceptions, all mechanical adaptations to what upstream lacks: fork logger calls → console.error; fork lib/env.ts zod schema → direct process.env reads + the startup alias in src/index.ts; org_id column reference and org resolution stripped (no organizations feature here); fork-only queue-deferral/embedding-enqueue calls and the documents.filename column write stripped from createDocumentFromUpload; lib/airgap.ts copied verbatim from the fork (8 lines, required by the air-gap gating and its tests); one fork test fixture (fakeEnv.ts, a stub of the fork-only lib/env) dropped along with its vi.mock. The fork's route-level Supabase integration test was not ported (no integration harness in this repo; consistent with the other feature PRs).

Credits & prior art

• @bmersereau (upstream #76) — the connector-secret encryption this PR reuses (lib/dms/crypto.ts re-exports the MCP encryptString/decryptString) descends from the HKDF + per-record-salt scheme fix: replace SHA-256 with HKDF + per-row salt for user API key encryption Open-Legal-Products/mike#76 introduced for user API keys; the fork's HKDF commit records fix: replace SHA-256 with HKDF + per-row salt for user API key encryption Open-Legal-Products/mike#76 as its precedent.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CEguyEgXa9JjCciXCcVemC