Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions docs/api-server.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ description: Daemon-served analytical Web UI and REST API for your msgvault arch
`msgvault serve` starts an HTTP server that exposes your archive through the
first-party Web UI at `/` and a REST API under `/api`. It optionally runs a
background sync scheduler to keep accounts up to date on a cron-based schedule.
The complete UI is embedded in the release binary; see [Web UI](/web-ui/) for
The complete UI is embedded in the release binary; see [Web UI](/docs/web-ui/) for
browser login, secure remote deployment, search states, and keyboard controls.

The API is registered through Huma and exposes a generated OpenAPI document at `/openapi.json`. You can also run `msgvault openapi` to print the same checked-in contract without starting a daemon or opening the archive database. The OpenAPI `info.version` is the API schema version used for client/server compatibility; the current schema is 2.4.0. The running daemon binary version is exposed separately in the generated document metadata. The API queries the same archive database and attachment store as the CLI, Web UI, and TUI. SQLite is the default archive database; PostgreSQL is supported when `[data].database_url` is a PostgreSQL DSN. Keyword search and ordinary archive reads stay local to that database. If vector search is enabled, semantic and hybrid search also call the embedding endpoint configured in `[vector.embeddings]`. The server is designed for interactive archive use, local integrations, dashboards, and automation scripts.
Expand Down Expand Up @@ -160,7 +160,7 @@ the building generation while a rebuild is in flight, otherwise the active
generation. During a rebuild the old active generation keeps serving vector
and hybrid search, but active-generation top-ups are frozen until the
building generation activates. See
[Vector Search](/usage/vector-search/) for the end-to-end workflow.
[Vector Search](/docs/usage/vector-search/) for the end-to-end workflow.

---

Expand Down Expand Up @@ -1046,8 +1046,8 @@ that signal (BM25 missed it or the ANN pool did not include it).
nothing to fuse). `subject_boosted` is true when the subject-line
boost was applied.

See [Searching](/usage/searching/) for the full query syntax
reference and [Vector Search](/usage/vector-search/) for vector /
See [Searching](/docs/usage/searching/) for the full query syntax
reference and [Vector Search](/docs/usage/vector-search/) for vector /
hybrid setup.

---
Expand Down Expand Up @@ -1634,7 +1634,7 @@ The same HTTP server backs configured remote CLI access and the local background
messages. Teams and Discord importers detect and checkpoint their own
first-run history backfills.

`msgvault serve` also runs scheduled SyncTech SMS Backup & Restore Drive sources configured under `[[synctech_sms.sources]]`; see [Configuration](/configuration/#synctech-sms-sources).
`msgvault serve` also runs scheduled SyncTech SMS Backup & Restore Drive sources configured under `[[synctech_sms.sources]]`; see [Configuration](/docs/configuration/#synctech-sms-sources).

## Security Model

Expand Down Expand Up @@ -1682,7 +1682,7 @@ requires a usable Parquet cache and keeps analytics unavailable until it is
ready; a build or open failure is fatal rather than a silent SQL fallback.
`auto_build_cache = false` leaves cache rebuilds to explicit
`msgvault build-cache` runs. These settings replace the TUI/MCP analytics flags
deprecated in 0.17.0; see [Configuration: analytics](/configuration/#analytics).
deprecated in 0.17.0; see [Configuration: analytics](/docs/configuration/#analytics).

`min_rebuild_interval` limits only automatic post-sync rebuilds. Explicit
builds, startup maintenance, query-required builds, and unusable-cache recovery
Expand All @@ -1700,4 +1700,4 @@ repeated archive-scale work on frequently synced archives. Changes under
| `schedule` | — | Cron expression for sync schedule |
| `enabled` | `true` | Whether scheduled sync is active |

See the [Configuration](/configuration/) page for the full config file reference.
See the [Configuration](/docs/configuration/) page for the full config file reference.
2 changes: 1 addition & 1 deletion docs/architecture/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ embedding endpoint configured in `[vector.embeddings]` to build/query semantic
vectors, then stores those vectors in `vectors.db` on SQLite or pgvector tables
on PostgreSQL.

<img src="/assets/static/how-it-works.svg" alt="msgvault architecture: Gmail API syncs to SQLite, then offline Parquet analytics, FTS5 search, TUI, and MCP Server" style="width: 100%; max-width: 960px; margin: 1.5rem auto; display: block;" />
<img src="/docs/assets/static/how-it-works.svg" alt="msgvault architecture: Gmail API syncs to SQLite, then offline Parquet analytics, FTS5 search, TUI, and MCP Server" style="width: 100%; max-width: 960px; margin: 1.5rem auto; display: block;" />

## Package Structure

Expand Down
4 changes: 2 additions & 2 deletions docs/architecture/postgresql.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,7 @@ workspaces; and Relationships' ranking, timeline, and identity link/unlink
cache refresh) have no PostgreSQL equivalent: those endpoints detect the
missing DuckDB/Parquet cache and return a named unavailable-cache state
rather than falling back to live SQL. If you see that state on a PostgreSQL
backend, it is expected — the [cache troubleshooting guidance](/web-ui/#cache-states)
backend, it is expected — the [cache troubleshooting guidance](/docs/web-ui/#cache-states)
applies to SQLite archives only.

## Current Scope
Expand Down Expand Up @@ -192,5 +192,5 @@ that generation's embedding rows so old vectors do not consume search candidate
budget for the active generation. Frequent full rebuilds can create dead tuples,
so monitor autovacuum on the embedding tables and run maintenance when needed.

See [Search Ranking Across Backends](/architecture/search-ranking/) for
See [Search Ranking Across Backends](/docs/architecture/search-ranking/) for
ranking differences between SQLite, PostgreSQL, sqlite-vec, and pgvector.
6 changes: 3 additions & 3 deletions docs/architecture/storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ string when checking already-imported source items.

SQLite uses an FTS5 virtual table named `messages_fts`. PostgreSQL uses a `search_fts` `tsvector` column on `messages` with a GIN index.

Both power `msgvault search`, but the rankers differ. See [Search Ranking Across Backends](/architecture/search-ranking/).
Both power `msgvault search`, but the rankers differ. See [Search Ranking Across Backends](/docs/architecture/search-ranking/).

### Relationships

Expand All @@ -146,7 +146,7 @@ PostgreSQL uses native types such as `BIGINT GENERATED ALWAYS AS IDENTITY`, `TIM

For semantic search, pgvector stores index generations, pending embedding work, and embedding vectors in the same PostgreSQL database. There is no separate `vectors.db` on PostgreSQL.

There is currently no SQLite to PostgreSQL migration command. Use PostgreSQL for a new archive or re-sync/import into an empty PostgreSQL database. See [PostgreSQL Backend](/architecture/postgresql/) for setup and operational notes.
There is currently no SQLite to PostgreSQL migration command. Use PostgreSQL for a new archive or re-sync/import into an empty PostgreSQL database. See [PostgreSQL Backend](/docs/architecture/postgresql/) for setup and operational notes.

## Parquet (Analytics Cache)

Expand Down Expand Up @@ -258,7 +258,7 @@ Use `pack-attachments` to migrate the eligible loose backlog immediately,
`unpack-attachments` to restore cataloged packed objects to loose files before
downgrading. The last command is local-only and requires the daemon to be
stopped because it removes production pack files. See the [CLI
reference](/cli-reference/#pack-attachments) and [Backup](/usage/backup/) guide
reference](/docs/cli-reference/#pack-attachments) and [Backup](/docs/usage/backup/) guide
for maintenance and restore behavior.

Set `[data].loose_attachments = true` when file-oriented backup or storage
Expand Down
12 changes: 6 additions & 6 deletions docs/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -444,7 +444,7 @@ All notable changes to msgvault, grouped by release.
the single archive writer: concurrent operations queue with a visible
`Waiting:` message, read-only commands run immediately, and scheduled
syncs yield to interactive commands. See the
[Daemon Migration Guide](/guides/daemon-migration/).
[Daemon Migration Guide](/docs/guides/daemon-migration/).
- Daemon lifecycle management via `msgvault serve start|status|stop|restart`,
with automatic restart of older local daemons on binary upgrade
(`[server].daemon_auto_restart`).
Expand Down Expand Up @@ -634,7 +634,7 @@ All notable changes to msgvault, grouped by release.

**New features**

- **Vector search (semantic and hybrid).** msgvault can now embed your archive using a configured OpenAI-compatible embedding endpoint (Ollama, llama.cpp `server`, LM Studio, etc.) and search it by meaning, not just keywords. `msgvault search --mode vector` runs pure semantic search; `--mode hybrid` fuses BM25 and vector similarity via Reciprocal Rank Fusion. Exposed through local CLI search (`msgvault search`), the HTTP API (`GET /api/v1/search?mode=vector|hybrid`), and the MCP server (`search_messages` mode argument plus a new `find_similar_messages` tool). See [Vector Search](/usage/vector-search/).
- **Vector search (semantic and hybrid).** msgvault can now embed your archive using a configured OpenAI-compatible embedding endpoint (Ollama, llama.cpp `server`, LM Studio, etc.) and search it by meaning, not just keywords. `msgvault search --mode vector` runs pure semantic search; `--mode hybrid` fuses BM25 and vector similarity via Reciprocal Rank Fusion. Exposed through local CLI search (`msgvault search`), the HTTP API (`GET /api/v1/search?mode=vector|hybrid`), and the MCP server (`search_messages` mode argument plus a new `find_similar_messages` tool). See [Vector Search](/docs/usage/vector-search/).
- `msgvault build-embeddings` command to generate and maintain the local vector index. Incremental by default; `--full-rebuild` creates a new generation and atomically activates it once coverage reaches zero. Same-model rebuilds keep answering against the previous active generation while the new one is built, with active-generation top-ups frozen until activation; model or dimension changes return `index_stale` until activation.
- Background embedding via the daemon scheduler. A new `[vector.embed.schedule]` config block drives the embed worker on cron and/or after every successful scheduled sync, so `msgvault serve` can keep the vector index current without manual intervention.
- `/api/v1/stats` gains a `vector_search` sub-object reporting the active generation, any in-flight rebuild, and the actionable missing embedding count for the generation the worker will target next.
Expand All @@ -643,7 +643,7 @@ All notable changes to msgvault, grouped by release.
**Improvements**

- `search` command gains `--mode fts|vector|hybrid` and `--explain` flags. `--explain` includes per-signal scores (RRF, BM25, vector) in table and JSON output for ranking inspection.
- Configuration gains a full `[vector]` block with sub-tables for the embedding endpoint, message preprocessing, hybrid ranking, and the embed scheduler. See [Configuration: vector](/configuration/#vector).
- Configuration gains a full `[vector]` block with sub-tables for the embedding endpoint, message preprocessing, hybrid ranking, and the embed scheduler. See [Configuration: vector](/docs/configuration/#vector).
- `remove-account` deletes attachment files from disk when they were unique to the removed account. Files shared across multiple accounts are preserved automatically, and an in-progress sync on any account skips file deletion to avoid racing new attachment writes.

**Bug fixes**
Expand Down Expand Up @@ -672,7 +672,7 @@ All notable changes to msgvault, grouped by release.

**New features**

- Structured file logging with per-run correlation IDs. Every CLI invocation gets a unique `run_id` on every log line, making it easy to trace a single run across shared log files. New `msgvault logs` command for viewing and tailing logs. File logging is opt-in; see [Configuration: Log](/configuration/#log) for setup.
- Structured file logging with per-run correlation IDs. Every CLI invocation gets a unique `run_id` on every log line, making it easy to trace a single run across shared log files. New `msgvault logs` command for viewing and tailing logs. File logging is opt-in; see [Configuration: Log](/docs/configuration/#log) for setup.

**Improvements**

Expand Down Expand Up @@ -725,9 +725,9 @@ All notable changes to msgvault, grouped by release.

**New features**

- SQL query interface via `msgvault query`. Run arbitrary SQL against DuckDB over Parquet with `--format json|csv|table`. See [SQL Queries](/usage/querying/).
- SQL query interface via `msgvault query`. Run arbitrary SQL against DuckDB over Parquet with `--format json|csv|table`. See [SQL Queries](/docs/usage/querying/).
- Microsoft 365 OAuth2 support via `msgvault add-o365` for Outlook.com and organizational accounts. Auto-detects personal vs. org IMAP hosts.
- Text message import: `import-whatsapp`, `import-imessage`, and `import-gvoice` for WhatsApp, iMessage, and Google Voice. See [Text Messages](/usage/text-messages/).
- Text message import: `import-whatsapp`, `import-imessage`, and `import-gvoice` for WhatsApp, iMessage, and Google Voice. See [Text Messages](/docs/usage/text-messages/).
- TUI text mode: press `m` to toggle between Email and Texts for browsing imported text conversations.
- `--after` and `--before` date filters for `sync-full` with IMAP accounts.
- CC and BCC recipients exposed in the message API responses.
Expand Down
Loading