diff --git a/docs/api-server.md b/docs/api-server.md index 102a5275a..6b52f15f6 100644 --- a/docs/api-server.md +++ b/docs/api-server.md @@ -10,7 +10,7 @@ description: Daemon-served analytical Web UI and REST API for your msgvault arch `msgvault serve` starts an HTTP server that exposes your archive through the first-party Web UI at `/` and a REST API under `/api`. It optionally runs a background sync scheduler to keep accounts up to date on a cron-based schedule. -The complete UI is embedded in the release binary; see [Web UI](/web-ui/) for +The complete UI is embedded in the release binary; see [Web UI](/docs/web-ui/) for browser login, secure remote deployment, search states, and keyboard controls. The API is registered through Huma and exposes a generated OpenAPI document at `/openapi.json`. You can also run `msgvault openapi` to print the same checked-in contract without starting a daemon or opening the archive database. The OpenAPI `info.version` is the API schema version used for client/server compatibility; the current schema is 2.4.0. The running daemon binary version is exposed separately in the generated document metadata. The API queries the same archive database and attachment store as the CLI, Web UI, and TUI. SQLite is the default archive database; PostgreSQL is supported when `[data].database_url` is a PostgreSQL DSN. Keyword search and ordinary archive reads stay local to that database. If vector search is enabled, semantic and hybrid search also call the embedding endpoint configured in `[vector.embeddings]`. The server is designed for interactive archive use, local integrations, dashboards, and automation scripts. @@ -160,7 +160,7 @@ the building generation while a rebuild is in flight, otherwise the active generation. During a rebuild the old active generation keeps serving vector and hybrid search, but active-generation top-ups are frozen until the building generation activates. See -[Vector Search](/usage/vector-search/) for the end-to-end workflow. +[Vector Search](/docs/usage/vector-search/) for the end-to-end workflow. --- @@ -1046,8 +1046,8 @@ that signal (BM25 missed it or the ANN pool did not include it). nothing to fuse). `subject_boosted` is true when the subject-line boost was applied. -See [Searching](/usage/searching/) for the full query syntax -reference and [Vector Search](/usage/vector-search/) for vector / +See [Searching](/docs/usage/searching/) for the full query syntax +reference and [Vector Search](/docs/usage/vector-search/) for vector / hybrid setup. --- @@ -1634,7 +1634,7 @@ The same HTTP server backs configured remote CLI access and the local background messages. Teams and Discord importers detect and checkpoint their own first-run history backfills. -`msgvault serve` also runs scheduled SyncTech SMS Backup & Restore Drive sources configured under `[[synctech_sms.sources]]`; see [Configuration](/configuration/#synctech-sms-sources). +`msgvault serve` also runs scheduled SyncTech SMS Backup & Restore Drive sources configured under `[[synctech_sms.sources]]`; see [Configuration](/docs/configuration/#synctech-sms-sources). ## Security Model @@ -1682,7 +1682,7 @@ requires a usable Parquet cache and keeps analytics unavailable until it is ready; a build or open failure is fatal rather than a silent SQL fallback. `auto_build_cache = false` leaves cache rebuilds to explicit `msgvault build-cache` runs. These settings replace the TUI/MCP analytics flags -deprecated in 0.17.0; see [Configuration: analytics](/configuration/#analytics). +deprecated in 0.17.0; see [Configuration: analytics](/docs/configuration/#analytics). `min_rebuild_interval` limits only automatic post-sync rebuilds. Explicit builds, startup maintenance, query-required builds, and unusable-cache recovery @@ -1700,4 +1700,4 @@ repeated archive-scale work on frequently synced archives. Changes under | `schedule` | — | Cron expression for sync schedule | | `enabled` | `true` | Whether scheduled sync is active | -See the [Configuration](/configuration/) page for the full config file reference. +See the [Configuration](/docs/configuration/) page for the full config file reference. diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md index 71792f76e..8c4d6e623 100644 --- a/docs/architecture/overview.md +++ b/docs/architecture/overview.md @@ -14,7 +14,7 @@ embedding endpoint configured in `[vector.embeddings]` to build/query semantic vectors, then stores those vectors in `vectors.db` on SQLite or pgvector tables on PostgreSQL. -msgvault architecture: Gmail API syncs to SQLite, then offline Parquet analytics, FTS5 search, TUI, and MCP Server +msgvault architecture: Gmail API syncs to SQLite, then offline Parquet analytics, FTS5 search, TUI, and MCP Server ## Package Structure diff --git a/docs/architecture/postgresql.md b/docs/architecture/postgresql.md index 5773c5925..7b79f9f06 100644 --- a/docs/architecture/postgresql.md +++ b/docs/architecture/postgresql.md @@ -147,7 +147,7 @@ workspaces; and Relationships' ranking, timeline, and identity link/unlink cache refresh) have no PostgreSQL equivalent: those endpoints detect the missing DuckDB/Parquet cache and return a named unavailable-cache state rather than falling back to live SQL. If you see that state on a PostgreSQL -backend, it is expected — the [cache troubleshooting guidance](/web-ui/#cache-states) +backend, it is expected — the [cache troubleshooting guidance](/docs/web-ui/#cache-states) applies to SQLite archives only. ## Current Scope @@ -192,5 +192,5 @@ that generation's embedding rows so old vectors do not consume search candidate budget for the active generation. Frequent full rebuilds can create dead tuples, so monitor autovacuum on the embedding tables and run maintenance when needed. -See [Search Ranking Across Backends](/architecture/search-ranking/) for +See [Search Ranking Across Backends](/docs/architecture/search-ranking/) for ranking differences between SQLite, PostgreSQL, sqlite-vec, and pgvector. diff --git a/docs/architecture/storage.md b/docs/architecture/storage.md index 97057a0f0..d6b9a1d03 100644 --- a/docs/architecture/storage.md +++ b/docs/architecture/storage.md @@ -128,7 +128,7 @@ string when checking already-imported source items. SQLite uses an FTS5 virtual table named `messages_fts`. PostgreSQL uses a `search_fts` `tsvector` column on `messages` with a GIN index. -Both power `msgvault search`, but the rankers differ. See [Search Ranking Across Backends](/architecture/search-ranking/). +Both power `msgvault search`, but the rankers differ. See [Search Ranking Across Backends](/docs/architecture/search-ranking/). ### Relationships @@ -146,7 +146,7 @@ PostgreSQL uses native types such as `BIGINT GENERATED ALWAYS AS IDENTITY`, `TIM For semantic search, pgvector stores index generations, pending embedding work, and embedding vectors in the same PostgreSQL database. There is no separate `vectors.db` on PostgreSQL. -There is currently no SQLite to PostgreSQL migration command. Use PostgreSQL for a new archive or re-sync/import into an empty PostgreSQL database. See [PostgreSQL Backend](/architecture/postgresql/) for setup and operational notes. +There is currently no SQLite to PostgreSQL migration command. Use PostgreSQL for a new archive or re-sync/import into an empty PostgreSQL database. See [PostgreSQL Backend](/docs/architecture/postgresql/) for setup and operational notes. ## Parquet (Analytics Cache) @@ -258,7 +258,7 @@ Use `pack-attachments` to migrate the eligible loose backlog immediately, `unpack-attachments` to restore cataloged packed objects to loose files before downgrading. The last command is local-only and requires the daemon to be stopped because it removes production pack files. See the [CLI -reference](/cli-reference/#pack-attachments) and [Backup](/usage/backup/) guide +reference](/docs/cli-reference/#pack-attachments) and [Backup](/docs/usage/backup/) guide for maintenance and restore behavior. Set `[data].loose_attachments = true` when file-oriented backup or storage diff --git a/docs/changelog.md b/docs/changelog.md index cc770f86b..4e704ffcc 100644 --- a/docs/changelog.md +++ b/docs/changelog.md @@ -444,7 +444,7 @@ All notable changes to msgvault, grouped by release. the single archive writer: concurrent operations queue with a visible `Waiting:` message, read-only commands run immediately, and scheduled syncs yield to interactive commands. See the - [Daemon Migration Guide](/guides/daemon-migration/). + [Daemon Migration Guide](/docs/guides/daemon-migration/). - Daemon lifecycle management via `msgvault serve start|status|stop|restart`, with automatic restart of older local daemons on binary upgrade (`[server].daemon_auto_restart`). @@ -634,7 +634,7 @@ All notable changes to msgvault, grouped by release. **New features** -- **Vector search (semantic and hybrid).** msgvault can now embed your archive using a configured OpenAI-compatible embedding endpoint (Ollama, llama.cpp `server`, LM Studio, etc.) and search it by meaning, not just keywords. `msgvault search --mode vector` runs pure semantic search; `--mode hybrid` fuses BM25 and vector similarity via Reciprocal Rank Fusion. Exposed through local CLI search (`msgvault search`), the HTTP API (`GET /api/v1/search?mode=vector|hybrid`), and the MCP server (`search_messages` mode argument plus a new `find_similar_messages` tool). See [Vector Search](/usage/vector-search/). +- **Vector search (semantic and hybrid).** msgvault can now embed your archive using a configured OpenAI-compatible embedding endpoint (Ollama, llama.cpp `server`, LM Studio, etc.) and search it by meaning, not just keywords. `msgvault search --mode vector` runs pure semantic search; `--mode hybrid` fuses BM25 and vector similarity via Reciprocal Rank Fusion. Exposed through local CLI search (`msgvault search`), the HTTP API (`GET /api/v1/search?mode=vector|hybrid`), and the MCP server (`search_messages` mode argument plus a new `find_similar_messages` tool). See [Vector Search](/docs/usage/vector-search/). - `msgvault build-embeddings` command to generate and maintain the local vector index. Incremental by default; `--full-rebuild` creates a new generation and atomically activates it once coverage reaches zero. Same-model rebuilds keep answering against the previous active generation while the new one is built, with active-generation top-ups frozen until activation; model or dimension changes return `index_stale` until activation. - Background embedding via the daemon scheduler. A new `[vector.embed.schedule]` config block drives the embed worker on cron and/or after every successful scheduled sync, so `msgvault serve` can keep the vector index current without manual intervention. - `/api/v1/stats` gains a `vector_search` sub-object reporting the active generation, any in-flight rebuild, and the actionable missing embedding count for the generation the worker will target next. @@ -643,7 +643,7 @@ All notable changes to msgvault, grouped by release. **Improvements** - `search` command gains `--mode fts|vector|hybrid` and `--explain` flags. `--explain` includes per-signal scores (RRF, BM25, vector) in table and JSON output for ranking inspection. -- Configuration gains a full `[vector]` block with sub-tables for the embedding endpoint, message preprocessing, hybrid ranking, and the embed scheduler. See [Configuration: vector](/configuration/#vector). +- Configuration gains a full `[vector]` block with sub-tables for the embedding endpoint, message preprocessing, hybrid ranking, and the embed scheduler. See [Configuration: vector](/docs/configuration/#vector). - `remove-account` deletes attachment files from disk when they were unique to the removed account. Files shared across multiple accounts are preserved automatically, and an in-progress sync on any account skips file deletion to avoid racing new attachment writes. **Bug fixes** @@ -672,7 +672,7 @@ All notable changes to msgvault, grouped by release. **New features** -- Structured file logging with per-run correlation IDs. Every CLI invocation gets a unique `run_id` on every log line, making it easy to trace a single run across shared log files. New `msgvault logs` command for viewing and tailing logs. File logging is opt-in; see [Configuration: Log](/configuration/#log) for setup. +- Structured file logging with per-run correlation IDs. Every CLI invocation gets a unique `run_id` on every log line, making it easy to trace a single run across shared log files. New `msgvault logs` command for viewing and tailing logs. File logging is opt-in; see [Configuration: Log](/docs/configuration/#log) for setup. **Improvements** @@ -725,9 +725,9 @@ All notable changes to msgvault, grouped by release. **New features** -- SQL query interface via `msgvault query`. Run arbitrary SQL against DuckDB over Parquet with `--format json|csv|table`. See [SQL Queries](/usage/querying/). +- SQL query interface via `msgvault query`. Run arbitrary SQL against DuckDB over Parquet with `--format json|csv|table`. See [SQL Queries](/docs/usage/querying/). - Microsoft 365 OAuth2 support via `msgvault add-o365` for Outlook.com and organizational accounts. Auto-detects personal vs. org IMAP hosts. -- Text message import: `import-whatsapp`, `import-imessage`, and `import-gvoice` for WhatsApp, iMessage, and Google Voice. See [Text Messages](/usage/text-messages/). +- Text message import: `import-whatsapp`, `import-imessage`, and `import-gvoice` for WhatsApp, iMessage, and Google Voice. See [Text Messages](/docs/usage/text-messages/). - TUI text mode: press `m` to toggle between Email and Texts for browsing imported text conversations. - `--after` and `--before` date filters for `sync-full` with IMAP accounts. - CC and BCC recipients exposed in the message API responses. diff --git a/docs/cli-reference.md b/docs/cli-reference.md index 56a7e5f0d..7e07e28a5 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -61,7 +61,7 @@ msgvault add-account --oauth-app | `--headless` | Show instructions for headless server setup | | `--oauth-app` | Use a named OAuth app from `[oauth.apps.]` in config | | `--force` | Delete existing token and re-authorize | -| `--readonly` | Request Gmail read-only access instead of read + write. Refused if the account already holds write access — see [OAuth Setup](/guides/oauth-setup/#read-only-access) | +| `--readonly` | Request Gmail read-only access instead of read + write. Refused if the account already holds write access — see [OAuth Setup](/docs/guides/oauth-setup/#read-only-access) | | `--display-name` | Set a display name for the account | | `--no-default-identity` | Do not auto-confirm the email address as this account's "me" identity | @@ -98,7 +98,7 @@ It tests the connection before saving credentials. Credentials are stored in `tokens/imap_.json` with restricted file permissions (0600). Use app-specific passwords when your provider supports them. -After adding an account, sync it with `msgvault sync-full`. IMAP accounts use the same `sync` and `sync-full` commands as Gmail. See [Setup Guide](/setup/#add-an-imap-account) for a walkthrough. +After adding an account, sync it with `msgvault sync-full`. IMAP accounts use the same `sync` and `sync-full` commands as Gmail. See [Setup Guide](/docs/setup/#add-an-imap-account) for a walkthrough. --- @@ -114,7 +114,7 @@ msgvault list-folders [account] Use the folder names in repeated `--folder` or `--skip-folder` flags on `sync-full` and `sync`. When the account argument is omitted, the command lists folders for every configured IMAP account. See -[IMAP Folder Sync](/usage/imap/) for examples and matching rules. +[IMAP Folder Sync](/docs/usage/imap/) for examples and matching rules. --- @@ -128,7 +128,7 @@ msgvault add-o365 The command opens your browser for Microsoft OAuth consent, then configures IMAP with XOAUTH2 automatically. The correct IMAP host is auto-detected: `outlook.office.com` for personal accounts (hotmail.com, outlook.com, live.com, msn.com) and `outlook.office365.com` for organizational accounts. -Requires a `[microsoft]` section with `client_id` in `config.toml`. See the [OAuth Setup guide](/guides/oauth-setup/#microsoft-365-outlook-hotmail) for Azure AD app registration. +Requires a `[microsoft]` section with `client_id` in `config.toml`. See the [OAuth Setup guide](/docs/guides/oauth-setup/#microsoft-365-outlook-hotmail) for Azure AD app registration. | Flag | Default | Description | |---|---|---| @@ -152,7 +152,7 @@ msgvault add-teams --tenant This stores a Teams Graph token under `tokens/teams_.json`, separate from the Microsoft IMAP token used by `add-o365`. Requires `[microsoft].client_id` in `config.toml` and the Graph permissions documented in -[Microsoft Teams](/usage/teams/). +[Microsoft Teams](/docs/usage/teams/). | Flag | Default | Description | |---|---|---| @@ -187,7 +187,7 @@ Intent, channel-history, and private-thread access issues. | `--oauth-app` | sole/default bot | Discord token binding label; no `[oauth.apps]` entry is required | After registering a guild, sync it with `msgvault sync-discord`. See -[Discord](/usage/discord/) for least-privilege bot setup and multi-bot binding +[Discord](/docs/usage/discord/) for least-privilege bot setup and multi-bot binding rules. --- @@ -243,7 +243,7 @@ local daemon and streams the daemon's stdout/stderr back to the terminal. The daemon serializes this work with other archive mutations. Folder filters are applied only to IMAP accounts. See -[IMAP Folder Sync](/usage/imap/) for examples and matching rules. +[IMAP Folder Sync](/docs/usage/imap/) for examples and matching rules. --- @@ -268,7 +268,7 @@ so importer upgrades can repair existing data without creating duplicates. | `--limit` | `0` | Maximum messages per conversation (`0` = unlimited) | | `--full` | `false` | Ignore stored cursor and re-fetch every message | -See [Microsoft Teams](/usage/teams/) for setup, scheduling, search, and inline +See [Microsoft Teams](/docs/usage/teams/) for setup, scheduling, search, and inline media backfill. --- @@ -296,7 +296,7 @@ aggregate error. Normal runs re-scan the configured trailing edit window, seven days by default. `--full --after` bounds both re-fetch and deletion repair and leaves -earlier rows untouched. See [Discord](/usage/discord/) for per-channel/thread +earlier rows untouched. See [Discord](/docs/usage/discord/) for per-channel/thread checkpoint and deletion semantics. --- @@ -348,7 +348,7 @@ removed from the archive and directs you to run `add-granola` again. | `--after` | — | Full-sync only notes created after this date (YYYY-MM-DD; implies `--full`) | | `--full` | `false` | Ignore stored cursor and re-fetch every note | -See [Meeting Transcripts](/usage/meetings/) for setup and what gets stored. +See [Meeting Transcripts](/docs/usage/meetings/) for setup and what gets stored. --- @@ -411,7 +411,7 @@ cancellation failures fail the sync and preserve the prior successful cursor. | `--full` | `false` | Ignore stored cursor and re-fetch every meeting | | `--probe` | `false` | Print the MCP tool inventory and a sample result instead of syncing | -See [Meeting Transcripts](/usage/meetings/) for setup and what gets stored. +See [Meeting Transcripts](/docs/usage/meetings/) for setup and what gets stored. --- @@ -457,14 +457,14 @@ run sequentially using the same aggregate-failure behavior as Backfill re-fetches each selected source message that has pending attachments to obtain fresh signed CDN URLs. An incomplete attachment is unrecoverable if the source message has since been deleted. See -[Discord](/usage/discord/#attachment-backfill-and-limits). +[Discord](/docs/usage/discord/#attachment-backfill-and-limits). --- ## add-beeper Register the chat accounts connected to a locally running -[Beeper Desktop](/usage/beeper/) as `beeper` sources, one per network. +[Beeper Desktop](/docs/usage/beeper/) as `beeper` sources, one per network. ```bash msgvault add-beeper @@ -498,7 +498,7 @@ is resumable; later runs are incremental. Per-account failures do not stop the run: remaining accounts still sync, the analytics cache is rebuilt for the successful ones, and the command exits non-zero listing the failures. Without `--account`, the `[beeper]` config `accounts`/`exclude_accounts` filters -select which registered sources sync. See [Beeper](/usage/beeper/). +select which registered sources sync. See [Beeper](/docs/usage/beeper/). ```bash msgvault sync-beeper @@ -536,7 +536,7 @@ msgvault backfill-beeper-media --account signal ## add-slack -Register a [Slack workspace](/usage/slack/) as a `slack` source. Requires a +Register a [Slack workspace](/docs/usage/slack/) as a `slack` source. Requires a user token (`xoxp-…`) from an internal Slack app you create (see the usage guide for the two-minute setup and scope list). The token is validated with `auth.test` plus a `search.messages` probe (thread-reply archiving needs the @@ -566,7 +566,7 @@ resumable; later runs are incremental and sweep for thread replies created since the last run (any thread age). Per-workspace failures do not stop the run: remaining workspaces still sync and the command exits non-zero listing the failures. The `[slack]` config `channels`/`exclude_channels` filters select which channels sync. See -[Slack](/usage/slack/). +[Slack](/docs/usage/slack/). ```bash msgvault sync-slack @@ -656,7 +656,7 @@ The export file may be a plain mbox file (any extension) or a `.zip` containing | `--no-attachments` | `false` | Skip writing attachments to disk | | `--no-default-identity` | `false` | Do not auto-confirm the identifier as this source's "me" identity | -See [Importing Local Email](/usage/importing/) for usage examples. +See [Importing Local Email](/docs/usage/importing/) for usage examples. --- @@ -693,7 +693,7 @@ reports the number of partial files imported. | `--no-attachments` | `false` | Skip writing attachments to disk | | `--no-default-identity` | `false` | Do not auto-confirm the identifier as this source's "me" identity | -See [Importing Local Email](/usage/importing/) for usage examples. +See [Importing Local Email](/docs/usage/importing/) for usage examples. --- @@ -715,7 +715,7 @@ The importer preserves PST folder structure as labels, imports email messages, a | `--checkpoint-interval` | `200` | Save progress every N messages | | `--no-attachments` | `false` | Skip writing attachments to disk | -See [Importing Local Email](/usage/importing/) for usage examples. +See [Importing Local Email](/docs/usage/importing/) for usage examples. --- @@ -738,7 +738,7 @@ The `--phone` flag is required and must be in E.164 format (e.g., `+447700900000 | `--display-name` | No | Display name for the phone owner | | `--no-default-identity` | No | Do not auto-confirm the phone number as this source's "me" identity | -See [Text Messages](/usage/text-messages/) for usage examples. +See [Text Messages](/docs/usage/text-messages/) for usage examples. --- @@ -761,7 +761,7 @@ Reads from `~/Library/Messages/chat.db` by default. This is a read-only operatio | `--me` | — | Your phone/email for recipient tracking | | `--contacts` | — | Path to contacts `.vcf` file for display-name backfill | -See [Text Messages](/usage/text-messages/) for usage examples. +See [Text Messages](/docs/usage/text-messages/) for usage examples. --- @@ -782,7 +782,7 @@ The directory must be the "Voice" folder from a Google Takeout export, containin | `--limit` | `0` | Limit number of messages (for testing) | | `--no-default-identity` | `false` | Do not auto-confirm the phone number as this source's "me" identity | -See [Text Messages](/usage/text-messages/) for usage examples. +See [Text Messages](/docs/usage/text-messages/) for usage examples. --- @@ -802,7 +802,7 @@ msgvault import-messenger --me | `--no-resume` | `false` | Start fresh, ignoring interrupted progress | | `--checkpoint-interval` | `200` | Save progress every N messages | -See [Text Messages](/usage/text-messages/) for usage examples. +See [Text Messages](/docs/usage/text-messages/) for usage examples. --- @@ -822,7 +822,7 @@ msgvault import-synctech-sms --owner-phone | `--calls` | `true` | Import call logs | | `--attachments` | `true` | Import MMS attachments | -See [Text Messages](/usage/text-messages/) for usage examples. +See [Text Messages](/docs/usage/text-messages/) for usage examples. --- @@ -954,7 +954,7 @@ target with that flag to guarantee a fully loose result. An overwritten target can retain uncataloged old pack files, and `unpack-attachments` processes only cataloged packs, so overwrite cannot currently make the same guarantee. Restoring into the live archive home of a running daemon is refused. See -[Backup](/usage/backup/) for repository format, scheduling, verification, and +[Backup](/docs/usage/backup/) for repository format, scheduling, verification, and privacy details. --- @@ -1038,7 +1038,7 @@ Ordinary aggregate views and statistics still default to email-only; use `--message-type` (or `message_type:` in the query) when you need an explicit search scope. -`--mode vector` and `--mode hybrid` require at least one free-text term in the query (filter-only queries use `--mode fts`). They do not support pagination (`--offset` is rejected) or non-active deletion scopes because the vector index covers active messages only. Bump `--limit` to retrieve a larger candidate pool instead. See [Searching](/usage/searching/) for the operator reference and [Vector Search](/usage/vector-search/) for semantic setup. +`--mode vector` and `--mode hybrid` require at least one free-text term in the query (filter-only queries use `--mode fts`). They do not support pagination (`--offset` is rejected) or non-active deletion scopes because the vector index covers active messages only. Bump `--limit` to retrieve a larger candidate pool instead. See [Searching](/docs/usage/searching/) for the operator reference and [Vector Search](/docs/usage/vector-search/) for semantic setup. --- @@ -1071,7 +1071,7 @@ review their disclosure and preflight. and bound to a stable index revision; restart pagination after a stale-cursor error. -See [Document Attachment Indexing](/usage/document-indexing/) for fixture +See [Document Attachment Indexing](/docs/usage/document-indexing/) for fixture generation, configuration, privacy boundaries, scheduling, and recovery. --- @@ -1088,7 +1088,7 @@ msgvault tui [flags] |---|---| | `--local` | Use the local daemon instead of the configured remote server | -Analytics engine and cache behavior are daemon-managed. Configure `[analytics].engine` and `[analytics].auto_build_cache` in `config.toml` to force live SQL, require DuckDB, or disable automatic cache builds. See [Configuration: analytics](/configuration/#analytics). +Analytics engine and cache behavior are daemon-managed. Configure `[analytics].engine` and `[analytics].auto_build_cache` in `config.toml` to force live SQL, require DuckDB, or disable automatic cache builds. See [Configuration: analytics](/docs/configuration/#analytics). Deprecated in 0.17.0: the older TUI-only `--force-sql`, `--no-cache-build`, and `--no-sqlite-scanner` flags are hidden and no longer control the foreground CLI. Use `[analytics].engine = "sql"` for live SQL, `[analytics].auto_build_cache = false` to skip daemon cache builds, or `msgvault build-cache` to prebuild cache files on the daemon host. @@ -1122,7 +1122,7 @@ manifest, all sources, all conversations, all messages, and one completion record with counts. A missing completion record means the stream is partial and must be rejected. Stdout contains JSONL only; diagnostics use stderr. -See [Exporting Data](/usage/exporting/) for the full record, identity, ordering, +See [Exporting Data](/docs/usage/exporting/) for the full record, identity, ordering, and validation contract. --- @@ -1157,7 +1157,7 @@ msgvault export-attachment [flags] The `--json`, `--base64`, and `--output` flags are mutually exclusive. -See [Exporting Data](/usage/exporting/) for usage examples. +See [Exporting Data](/docs/usage/exporting/) for usage examples. --- @@ -1173,7 +1173,7 @@ msgvault export-attachments [flags] |---|---| | `-o`, `--output ` | Output directory (default: current directory) | -Accepts internal numeric IDs or Gmail message IDs. See [Exporting Data](/usage/exporting/) for usage examples. +Accepts internal numeric IDs or Gmail message IDs. See [Exporting Data](/docs/usage/exporting/) for usage examples. --- @@ -1272,7 +1272,7 @@ Email sync enriches identities already confirmed for the source with strong sender evidence from trusted Sent metadata; it does not confirm first-time aliases. Review candidates with `msgvault identity discover` and apply strong candidates with `msgvault identity discover --apply`. Recipient-only evidence -stays review-only. See [People, Profiles, and Source Identities](/usage/people/) +stays review-only. See [People, Profiles, and Source Identities](/docs/usage/people/) for classifications, Fastmail inventory, and import formats. --- @@ -1329,7 +1329,7 @@ Exact splits restore the pre-merge profiles when their lineage and referenced rows remain available. For partial splits, use `--json` to inspect ambiguous or unrestored rows. Complete merge packets retain merge-time profile values even after later redaction and require the strongest profile-data options when -copied into a subset. See [People, Profiles, and Source Identities](/usage/people/#merge-duplicate-profiles-and-reverse-a-merge) +copied into a subset. See [People, Profiles, and Source Identities](/docs/usage/people/#merge-duplicate-profiles-and-reverse-a-merge) for the workflow and lifecycle boundaries. | Attribute flag | Applies to | Description | @@ -1348,7 +1348,7 @@ for the workflow and lifecycle boundaries. | `--json` | `list`, `set`, `clear` | Output structured JSON | Setting or clearing a value closes the current history row rather than deleting -it. See [People, Profiles, and Source Identities](/usage/people/) for the +it. See [People, Profiles, and Source Identities](/docs/usage/people/) for the shipped definitions and complete workflow. --- @@ -1532,7 +1532,7 @@ Use this if `verify` reports FTS5 shadow-table corruption such as a malformed in ## embeddings -Manage the vector embedding index used by `--mode vector` and `--mode hybrid` search. Requires a build with a vector backend (`sqlite_vec` for SQLite archives, `pgvector` for PostgreSQL archives) and a configured `[vector.embeddings]` endpoint. See [Vector Search](/usage/vector-search/) for prerequisites, model rotation, and troubleshooting. +Manage the vector embedding index used by `--mode vector` and `--mode hybrid` search. Requires a build with a vector backend (`sqlite_vec` for SQLite archives, `pgvector` for PostgreSQL archives) and a configured `[vector.embeddings]` endpoint. See [Vector Search](/docs/usage/vector-search/) for prerequisites, model rotation, and troubleshooting. ```bash msgvault embeddings [flags] @@ -1564,7 +1564,7 @@ Without `--full-rebuild`, the command is incremental: it resumes any in-flight r The account scope is part of the generation fingerprint, so building with a different `--account`/`--collection` set than the active generation requires `--full-rebuild`, exactly like changing the model. See -[Scoped Generations](/usage/vector-search/#scoped-generations). +[Scoped Generations](/docs/usage/vector-search/#scoped-generations). ### embeddings resume @@ -1640,7 +1640,7 @@ If the analytics cache is stale, it is automatically rebuilt before the query ru |---|---|---| | `--format` | `json` | Output format: `json`, `csv`, or `table` | -See [SQL Queries](/usage/querying/) for available views and example queries. +See [SQL Queries](/docs/usage/querying/) for available views and example queries. --- @@ -1654,12 +1654,12 @@ msgvault mcp [flags] | Flag | Default | Description | |---|---|---| -| `--force-sql` | `false` | Deprecated in 0.17.0; use `[analytics].engine = "sql"` in `config.toml` instead. See [Configuration: analytics](/configuration/#analytics). | +| `--force-sql` | `false` | Deprecated in 0.17.0; use `[analytics].engine = "sql"` in `config.toml` instead. See [Configuration: analytics](/docs/configuration/#analytics). | | `--no-sqlite-scanner` | `false` | Deprecated in 0.17.0; cache engine selection is daemon-managed. Use `[analytics].engine = "sql"` for live SQL. | | `--http` | — | Serve MCP over StreamableHTTP on this address instead of stdio. Bare ports bind to loopback, e.g. `8080` becomes `127.0.0.1:8080`. Non-loopback addresses require `[server].api_key` or `--http-allow-insecure`. | | `--http-allow-insecure` | `false` | Allow non-loopback HTTP binding without `[server].api_key`. A configured key is still enforced; without one, use only behind a trusted network boundary or authenticated reverse proxy. | -See [MCP Server](/usage/chat/) for configuration and tool reference. +See [MCP Server](/docs/usage/chat/) for configuration and tool reference. --- @@ -1686,7 +1686,7 @@ marker are preserved unless `--force` is supplied. `skills uninstall` accepts `--agent` and `--dir` with the same target semantics, and removes only generated copies that still carry the marker. See -[Agent Skills](/guides/agent-skills/) for the workflow and safety model. +[Agent Skills](/docs/guides/agent-skills/) for the workflow and safety model. --- @@ -1720,7 +1720,7 @@ msgvault daemon restart `start` launches the daemon in the background, `status` reports its recorded URL/PID/version/API schema/uptime, `stop` shuts it down, and `restart` performs a stop followed by a start. Starting a newer compatible binary replaces an older recorded daemon when `[server].daemon_auto_restart = "newer"`; incompatible running daemons are reported with a prompt to stop them first. -The lifecycle commands have no command-specific flags. All configuration (port, bind address, API key, CORS, account schedules, SyncTech SMS sources, background idle timeout, daemon restart policy, and vector embedding schedule) is read from your `config.toml`. See [Web UI & API Server](/api-server/) for endpoint documentation, run `msgvault openapi`, or fetch `/openapi.json` from a running server for the generated OpenAPI contract. See [Configuration](/configuration/#server) for config options. When vector search is enabled, the daemon can also run the embed worker on a cron and/or after every successful sync, see [Configuration: vector.embed.schedule](/configuration/#vectorembedschedule). +The lifecycle commands have no command-specific flags. All configuration (port, bind address, API key, CORS, account schedules, SyncTech SMS sources, background idle timeout, daemon restart policy, and vector embedding schedule) is read from your `config.toml`. See [Web UI & API Server](/docs/api-server/) for endpoint documentation, run `msgvault openapi`, or fetch `/openapi.json` from a running server for the generated OpenAPI contract. See [Configuration](/docs/configuration/#server) for config options. When vector search is enabled, the daemon can also run the embed worker on a cron and/or after every successful sync, see [Configuration: vector.embed.schedule](/docs/configuration/#vectorembedschedule). Background daemons started by `daemon start` or auto-started by a CLI command shut down after `[server].daemon_idle_timeout` with no requests. The default is `20m`; set it to `"0s"` to disable idle shutdown. `MSGVAULT_DAEMON_IDLE_TIMEOUT` can override the value for a lifecycle-managed background daemon. @@ -1991,7 +1991,7 @@ msgvault completion powershell | Out-String | Invoke-Expression ## logs -View and tail structured log files from the selected daemon. With `[remote].url` configured, this shows remote daemon logs; otherwise it starts or contacts the local daemon. File logging must be enabled first (see [Configuration: Log](/configuration/#log)). +View and tail structured log files from the selected daemon. With `[remote].url` configured, this shows remote daemon logs; otherwise it starts or contacts the local daemon. File logging must be enabled first (see [Configuration: Log](/docs/configuration/#log)). ```bash msgvault logs [flags] diff --git a/docs/configuration.md b/docs/configuration.md index 7f2400061..06a616f71 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -251,7 +251,7 @@ all supported standalone attachment sources. The first release requires `[attachments.documents.index].lexical = true` and `store_chunk_text = true`. Hosted document embeddings are not enabled by this configuration. -See [Document Attachment Indexing](/usage/document-indexing/) for the complete +See [Document Attachment Indexing](/docs/usage/document-indexing/) for the complete probe, consent, build, and recovery flow. ### `[oauth]` @@ -270,7 +270,7 @@ Named OAuth apps for Google Workspace organizations that require their own OAuth | `client_secrets` | — | Path to the org's `client_secret.json` | | `service_account_key` | — | Path to the org's Google service account key JSON | -See [OAuth Setup: Google Workspace Accounts](/guides/oauth-setup/#google-workspace-accounts) for when and why you need named apps. +See [OAuth Setup: Google Workspace Accounts](/docs/guides/oauth-setup/#google-workspace-accounts) for when and why you need named apps. Discord's `--oauth-app` value is only a protected bot-token binding label. It is not resolved from this section and does not require an `[oauth.apps]` entry. @@ -292,7 +292,7 @@ sync. Required only if you use `add-o365`, `add-teams`, or `sync-teams`. | `redirect_uri` | `http://localhost:8089/callback/microsoft` | OAuth redirect URI registered in the Azure AD app | | `tenant_id` | `common` | Azure AD tenant ID; `common` allows both personal and org accounts | -See [OAuth Setup: Microsoft 365](/guides/oauth-setup/#microsoft-365-outlook-hotmail) for app registration steps. Teams uses the same `client_id` but requests Microsoft Graph scopes and stores tokens under `tokens/teams_.json`; Outlook/Hotmail IMAP OAuth uses `tokens/microsoft_.json`. +See [OAuth Setup: Microsoft 365](/docs/guides/oauth-setup/#microsoft-365-outlook-hotmail) for app registration steps. Teams uses the same `client_id` but requests Microsoft Graph scopes and stores tokens under `tokens/teams_.json`; Outlook/Hotmail IMAP OAuth uses `tokens/microsoft_.json`. ### `[[fastmail]]` @@ -311,7 +311,7 @@ Exactly one source selector is required. Prefer `source_id` when two sources share an identifier or display name. With automatic confirmation disabled, `msgvault identity discover --source-id --provider` fetches the inventory for an explicit preview; add `--apply` only after reviewing it. See [People, -Profiles, and Source Identities](/usage/people/#fastmail-alias-inventory). +Profiles, and Source Identities](/docs/usage/people/#fastmail-alias-inventory). ### `[discord]` @@ -337,7 +337,7 @@ and forum post. Top-level channels match directly. A child inherits its parent's state unless its own ID appears explicitly. An explicit child include can override an excluded parent; an explicit child exclude can override an included parent. `exclude` wins when the same ID is in both lists. See -[Discord](/usage/discord/#configure-media-repairs-and-channel-filters). +[Discord](/docs/usage/discord/#configure-media-repairs-and-channel-filters). ### `[log]` @@ -355,7 +355,7 @@ Log files are named `msgvault-YYYY-MM-DD.log` (UTC date), written as newline-del When SQL logging is enabled, slow/error entries include query arguments and streaming query durations, which makes it easier to diagnose expensive reads without enabling full trace output. -Use `msgvault logs` to view and tail log files from the selected local or remote daemon. See [CLI Reference: logs](/cli-reference/#logs). +Use `msgvault logs` to view and tail log files from the selected local or remote daemon. See [CLI Reference: logs](/docs/cli-reference/#logs). ### `[sync]` @@ -365,7 +365,7 @@ Use `msgvault logs` to view and tail log files from the selected local or remote ### `[server]` -Settings for the Web UI and API server started by `msgvault serve`. The same HTTP server is used by remote CLI access and by the local background daemon for archive-access CLI commands. The `api_key` setting is also reused for inbound bearer authentication when `msgvault mcp --http` starts a separate Streamable HTTP listener; that listener's address comes from the `--http` flag. See [Web UI & API Server](/api-server/) for API endpoint documentation and [MCP Server](/usage/chat/#streamablehttp-transport) for MCP client setup, or fetch `/openapi.json` from a running server for the generated OpenAPI contract. +Settings for the Web UI and API server started by `msgvault serve`. The same HTTP server is used by remote CLI access and by the local background daemon for archive-access CLI commands. The `api_key` setting is also reused for inbound bearer authentication when `msgvault mcp --http` starts a separate Streamable HTTP listener; that listener's address comes from the `--http` flag. See [Web UI & API Server](/docs/api-server/) for API endpoint documentation and [MCP Server](/docs/usage/chat/#streamablehttp-transport) for MCP client setup, or fetch `/openapi.json` from a running server for the generated OpenAPI contract. | Key | Default | Description | |---|---|---| @@ -387,7 +387,7 @@ Settings for the Web UI and API server started by `msgvault serve`. The same HTT Browser sessions are additive to API-key authentication. Existing CLI and programmatic clients continue to send the configured key. For remote browser access, terminate TLS at a reverse proxy and list that proxy—not arbitrary -clients—in `trusted_proxies`. See [Web UI](/web-ui/) for the complete security +clients—in `trusted_proxies`. See [Web UI](/docs/web-ui/) for the complete security model and the plain-HTTP warning. For MCP Streamable HTTP, send `[server].api_key` as `Authorization: Bearer @@ -457,11 +457,11 @@ Cache build memory and temporary disk usage scale with archive size, so a minimum interval can prevent repeated archive-scale work when sources sync frequently. Changes under `[analytics]` take effect after the daemon restarts. -This setting governs the aggregate views (Senders/Domains/Labels/Time) and is ignored entirely when `[data].database_url` points at PostgreSQL — a PostgreSQL backend always uses live SQL for those views, and `build-cache` refuses to run against it. It does not affect the Web UI's Explore, Files, or People/domains workspaces, which require the SQLite + DuckDB/Parquet cache regardless of this setting and are unavailable on PostgreSQL; see [PostgreSQL Backend](/architecture/postgresql/) for the current scope. +This setting governs the aggregate views (Senders/Domains/Labels/Time) and is ignored entirely when `[data].database_url` points at PostgreSQL — a PostgreSQL backend always uses live SQL for those views, and `build-cache` refuses to run against it. It does not affect the Web UI's Explore, Files, or People/domains workspaces, which require the SQLite + DuckDB/Parquet cache regardless of this setting and are unavailable on PostgreSQL; see [PostgreSQL Backend](/docs/architecture/postgresql/) for the current scope. ### `[backup]` -Default settings for `msgvault backup`. See [Backup](/usage/backup/) for the +Default settings for `msgvault backup`. See [Backup](/docs/usage/backup/) for the capture, verify, and restore workflow. | Key | Default | Description | @@ -551,7 +551,7 @@ enabled = true ### `[beeper]` -Archive chats from a locally running [Beeper Desktop](/usage/beeper/). A single +Archive chats from a locally running [Beeper Desktop](/docs/usage/beeper/). A single block (not a list): the Beeper Desktop API is loopback-only, so there is one instance per machine and the daemon must run beside it. Authorize first with `msgvault add-beeper`. @@ -581,7 +581,7 @@ max_media_mb = 100 # per-attachment download cap (MiB) ### `[slack]` -Archive [Slack workspaces](/usage/slack/). A single block covers every +Archive [Slack workspaces](/docs/usage/slack/). A single block covers every registered workspace (tokens are per-workspace files). Authorize each workspace first with `msgvault add-slack`. @@ -611,7 +611,7 @@ Each entry is one Granola account. `identifier` is a stable source label; `account_email` is the primary identity used for organizer attribution. `msgvault serve` runs it on the given cron schedule. Register the account first with `msgvault add-granola`. See -[Meeting Transcripts](/usage/meetings/). +[Meeting Transcripts](/docs/usage/meetings/). ```toml [[granola]] @@ -643,7 +643,7 @@ the archive; removing it prevents the scheduler from silently recreating it. Circleback meeting sync is configured with top-level `[[circleback]]` entries. Authentication is browser OAuth (`msgvault add-circleback`); no secret lives in the config file. See -[Meeting Transcripts](/usage/meetings/). +[Meeting Transcripts](/docs/usage/meetings/). ```toml [[circleback]] @@ -669,7 +669,7 @@ opt-out flag. ### `[vector]` -Top-level toggle and backend marker for semantic/hybrid search. SQLite vector search requires a build with `sqlite_vec` support (default via `make build`). PostgreSQL vector search requires a build with the `pgvector` tag and a PostgreSQL `[data].database_url`. See [Vector Search](/usage/vector-search/) for prerequisites, initial embedding, and the full workflow. +Top-level toggle and backend marker for semantic/hybrid search. SQLite vector search requires a build with `sqlite_vec` support (default via `make build`). PostgreSQL vector search requires a build with the `pgvector` tag and a PostgreSQL `[data].database_url`. See [Vector Search](/docs/usage/vector-search/) for prerequisites, initial embedding, and the full workflow. | Key | Default | Description | |---|---|---| @@ -800,5 +800,5 @@ All data lives under the msgvault home directory (`~/.msgvault` on macOS/Linux, | `msgvault.db` | SQLite database (system of record when PostgreSQL is not configured) | | `attachments/` | Content-addressed attachment files | | `tokens/` | OAuth tokens per account | -| `logs/` | Structured log files (when [file logging](/configuration/#log) is enabled) | +| `logs/` | Structured log files (when [file logging](/docs/configuration/#log) is enabled) | | `analytics/` | Parquet cache files for Web UI and TUI analytical views | diff --git a/docs/faq.md b/docs/faq.md index 24ffa6557..87ff4d684 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -21,11 +21,11 @@ The MCP server's lack of sync capability offers some protection here, since an a uses. Open it to search and group across email, chat, calendar, and meeting data, inspect people and files, monitor sources, and review staged deletions. The same server supports automations, integrations, and scheduled background -sync. See [Web UI](/web-ui/) and [Web UI & API Server](/api-server/) for details. +sync. See [Web UI](/docs/web-ui/) and [Web UI & API Server](/docs/api-server/) for details.

Where is my email data stored?

-By default, everything stays on your local machine. msgvault stores messages in a SQLite database and Parquet analytics files inside your `MSGVAULT_HOME` directory (defaults to `~/.msgvault`). If you configure a remote deployment, that archive lives on your own server. See [Data Storage](/architecture/storage/) for details. +By default, everything stays on your local machine. msgvault stores messages in a SQLite database and Parquet analytics files inside your `MSGVAULT_HOME` directory (defaults to `~/.msgvault`). If you configure a remote deployment, that archive lives on your own server. See [Data Storage](/docs/architecture/storage/) for details.

Can I use msgvault with non-Gmail accounts?

@@ -34,20 +34,20 @@ Discord guilds, Slack workspaces, Beeper Desktop chats, Google Calendar, and sup note services. You can also import email from PST, MBOX, or Apple Mail and chats/texts from WhatsApp, iMessage, Google Voice, Facebook Messenger, and SMS Backup & Restore. All messages use the same Web UI, search, TUI, MCP, REST API, -and export surfaces. See [Setup Guide](/setup/#add-an-imap-account), -[Importing Local Email](/usage/importing/), [Text Messages](/usage/text-messages/), -and [Discord](/usage/discord/) or [Slack](/usage/slack/). +and export surfaces. See [Setup Guide](/docs/setup/#add-an-imap-account), +[Importing Local Email](/docs/usage/importing/), [Text Messages](/docs/usage/text-messages/), +and [Discord](/docs/usage/discord/) or [Slack](/docs/usage/slack/).

Can msgvault archive Discord direct messages?

No. Discord bot tokens expose guilds the bot has joined, not a person's direct messages. msgvault does not accept user tokens or implement selfbots. It can archive accessible guild channels, threads, forum posts, and attachments; see -[Discord](/usage/discord/). +[Discord](/docs/usage/discord/).

Does deleting email in msgvault delete it from Gmail?

-Only if you explicitly run the full deletion workflow. Staging messages for deletion in the Web UI or TUI does not touch Gmail or your IMAP provider. You must run `MSGVAULT_ENABLE_REMOTE_DELETE=1 msgvault delete-staged` to execute staged deletions. Gmail messages move to trash by default; `--permanent` opts into permanent Gmail deletion. IMAP deletion removes messages from the provider. Your local archive is always preserved. See [Deleting Email](/usage/deletion/) for the complete process. +Only if you explicitly run the full deletion workflow. Staging messages for deletion in the Web UI or TUI does not touch Gmail or your IMAP provider. You must run `MSGVAULT_ENABLE_REMOTE_DELETE=1 msgvault delete-staged` to execute staged deletions. Gmail messages move to trash by default; `--permanent` opts into permanent Gmail deletion. IMAP deletion removes messages from the provider. Your local archive is always preserved. See [Deleting Email](/docs/usage/deletion/) for the complete process. --- diff --git a/docs/guides/daemon-migration.md b/docs/guides/daemon-migration.md index 485a4363d..d3e3eb555 100644 --- a/docs/guides/daemon-migration.md +++ b/docs/guides/daemon-migration.md @@ -122,7 +122,7 @@ With a `[remote]` server configured, tokens live on the remote host, so authorization happens there — same as pre-daemon remote behavior. For headless remote setups, keep using `add-account --headless` or `msgvault export-token` to push a locally minted token to the server (see -[Remote Deployment](/guides/remote-deployment/)). +[Remote Deployment](/docs/guides/remote-deployment/)). ## Gotcha: `--local` means "local daemon" @@ -198,4 +198,4 @@ shows version, uptime, and vector-search state. **Something is off after upgrading. What is the first thing to try?** `msgvault daemon restart`. It re-reads config, picks up the current binary and environment, and re-registers the runtime record. See -[Troubleshooting](/troubleshooting/) for more. +[Troubleshooting](/docs/troubleshooting/) for more. diff --git a/docs/guides/oauth-setup.md b/docs/guides/oauth-setup.md index 41c2b2b51..151d7ead8 100644 --- a/docs/guides/oauth-setup.md +++ b/docs/guides/oauth-setup.md @@ -183,7 +183,7 @@ msgvault add-account personal@gmail.com # uses default The binding is stored per account, so `sync`, `verify`, and `serve` automatically use the correct credentials. You only need `--oauth-app` when first adding or rebinding an account.
- Two OAuth apps and the token files they create. A default app (config block [oauth]) authorizes personal Gmail accounts personal@gmail.com and other@gmail.com; a named app ([oauth.apps.acme]) authorizes the Workspace account you@acme.com. Each add-account run writes its own token file under ~/.msgvault/tokens/, color-matched to its account. + Two OAuth apps and the token files they create. A default app (config block [oauth]) authorizes personal Gmail accounts personal@gmail.com and other@gmail.com; a named app ([oauth.apps.acme]) authorizes the Workspace account you@acme.com. Each add-account run writes its own token file under ~/.msgvault/tokens/, color-matched to its account.
To switch an existing account to a different OAuth app: @@ -400,7 +400,7 @@ msgvault sync-teams you@example.com ``` Some organizations require administrator consent before delegated channel -message permissions can be used. See [Microsoft Teams](/usage/teams/) for the +message permissions can be used. See [Microsoft Teams](/docs/usage/teams/) for the full Teams workflow. ### Sync Your Email diff --git a/docs/guides/remote-deployment.md b/docs/guides/remote-deployment.md index d87063862..6494991c1 100644 --- a/docs/guides/remote-deployment.md +++ b/docs/guides/remote-deployment.md @@ -225,7 +225,7 @@ curl -H "X-API-Key: YOUR_API_KEY" http://remote-host:8080/api/v1/scheduler/statu `msgvault add-calendar you@gmail.com` on a machine with a browser, copy the token to the server (it now carries Gmail + Calendar), then add a `[[gcal]]` entry with a cron `schedule` so the daemon syncs it. See - [Google Calendar](/usage/calendar/). + [Google Calendar](/docs/usage/calendar/). After setup, your data directory contains: @@ -248,7 +248,7 @@ regular remote use, terminate HTTPS at a reverse proxy and add only that proxy's address or CIDR to `server.trusted_proxies`; the daemon uses trusted forwarding information to mark its browser session cookie `Secure`. Plain HTTP on a private network is supported as an explicit tradeoff and produces a UI -warning because the cookie is not encrypted in transit. See [Web UI](/web-ui/) +warning because the cookie is not encrypted in transit. See [Web UI](/docs/web-ui/) for the complete session and proxy model. ## Using the Local CLI Against Remote diff --git a/docs/index.md b/docs/index.md index 7af622c1c..cc425b859 100644 --- a/docs/index.md +++ b/docs/index.md @@ -9,13 +9,13 @@ Archive a lifetime of email and chat. Fast keyword search, opt-in semantic search, and local AI workflows.

- Quick Start + Quick Start GitHub Discord

- msgvault TUI showing the Senders view + msgvault TUI showing the Senders view
Supports Gmail, Google Calendar, Microsoft Teams, Discord, Slack, Granola, Circleback, Beeper @@ -23,7 +23,7 @@ Desktop, IMAP, and Microsoft 365 mail sync; verifiable backup snapshots; PST, MBOX, and Apple Mail import; and chat/text import from WhatsApp, iMessage, Google Voice, Facebook Messenger, and SMS Backup & Restore. -Read the [Introduction](/introduction/) to learn more about why this project +Read the [Introduction](/docs/introduction/) to learn more about why this project was created. ## Install @@ -38,14 +38,14 @@ curl -fsSL https://msgvault.io/install.sh | bash powershell -ExecutionPolicy ByPass -c "irm https://msgvault.io/install.ps1 | iex" ``` -Then [set up OAuth credentials](/guides/oauth-setup/) and [start -syncing](/setup/). You can also [build from source](/setup/#build-from-source). +Then [set up OAuth credentials](/docs/guides/oauth-setup/) and [start +syncing](/docs/setup/). You can also [build from source](/docs/setup/#build-from-source). !!! note "New in 0.19.0" Explore relationships in the new Web UI; archive Slack workspaces and Discord guilds; ingest meetings through the API; export bounded message windows; discover source identities; and manage durable person profiles. - See the [Changelog](/changelog/) for the full release notes. + See the [Changelog](/docs/changelog/) for the full release notes. ## Why msgvault? @@ -138,4 +138,4 @@ it lives in an archive on disk that you own and control. ## How It Works -msgvault architecture: Gmail API syncs to SQLite, then offline Parquet analytics, FTS5 search, TUI, and MCP Server +msgvault architecture: Gmail API syncs to SQLite, then offline Parquet analytics, FTS5 search, TUI, and MCP Server diff --git a/docs/overrides/main.html b/docs/overrides/main.html index 29f2b74a9..4579b4b8d 100644 --- a/docs/overrides/main.html +++ b/docs/overrides/main.html @@ -2,11 +2,11 @@ {% block extrahead %} {{ super() }} - + - + {% endblock %} diff --git a/docs/overrides/sitemap.xml b/docs/overrides/sitemap.xml index 6095a6707..a5df4f7d6 100644 --- a/docs/overrides/sitemap.xml +++ b/docs/overrides/sitemap.xml @@ -1,11 +1,17 @@ {#- - Zensical only includes nav pages in the sitemap; keep the homepage indexed too. + Zensical only includes nav pages in the sitemap; keep the site tiers indexed too. -#} https://msgvault.io/ + + https://msgvault.io/guide/ + + + https://msgvault.io/docs/ + {%- for page in pages -%} {%- if page.canonical_url %} diff --git a/docs/scripts/check_built_site.py b/docs/scripts/check_built_site.py index 3a11bcc7c..7851eb1d2 100755 --- a/docs/scripts/check_built_site.py +++ b/docs/scripts/check_built_site.py @@ -11,75 +11,90 @@ ROOT = pathlib.Path(__file__).resolve().parents[1] SITE = ROOT / "site" -ROUTES = [ +WEBSITE_ROUTES = [ "/", - "/api-server/", - "/architecture/overview/", - "/architecture/postgresql/", - "/architecture/search-ranking/", - "/architecture/storage/", - "/changelog/", - "/cli-reference/", - "/configuration/", - "/development/", - "/faq/", - "/guides/oauth-setup/", - "/guides/remote-deployment/", - "/guides/verification/", - "/introduction/", - "/setup/", - "/troubleshooting/", - "/usage/analytics/", - "/usage/chat/", - "/usage/deduplication/", - "/usage/deletion/", - "/usage/exporting/", - "/usage/importing/", - "/usage/multi-account/", - "/usage/querying/", - "/usage/searching/", - "/usage/text-messages/", - "/usage/tui/", - "/usage/vector-search/", - "/web-ui/", + "/guide/", +] + +WEBSITE_FILES = [ + "index.md", + "guide.md", + "llms.txt", + "favicon.svg", + "styles/site.css", + "scripts/site.js", +] + +DOCS_ROUTES = [ + "/docs/", + "/docs/api-server/", + "/docs/architecture/overview/", + "/docs/architecture/postgresql/", + "/docs/architecture/search-ranking/", + "/docs/architecture/storage/", + "/docs/changelog/", + "/docs/cli-reference/", + "/docs/configuration/", + "/docs/development/", + "/docs/faq/", + "/docs/guides/oauth-setup/", + "/docs/guides/remote-deployment/", + "/docs/guides/verification/", + "/docs/introduction/", + "/docs/setup/", + "/docs/troubleshooting/", + "/docs/usage/analytics/", + "/docs/usage/chat/", + "/docs/usage/deduplication/", + "/docs/usage/deletion/", + "/docs/usage/exporting/", + "/docs/usage/importing/", + "/docs/usage/multi-account/", + "/docs/usage/querying/", + "/docs/usage/searching/", + "/docs/usage/text-messages/", + "/docs/usage/tui/", + "/docs/usage/vector-search/", + "/docs/web-ui/", ] REQUIRED_SITEMAP_URLS = [ "https://msgvault.io/", - "https://msgvault.io/api-server/", - "https://msgvault.io/architecture/overview/", - "https://msgvault.io/architecture/postgresql/", - "https://msgvault.io/architecture/search-ranking/", - "https://msgvault.io/architecture/storage/", - "https://msgvault.io/changelog/", - "https://msgvault.io/cli-reference/", - "https://msgvault.io/configuration/", - "https://msgvault.io/development/", - "https://msgvault.io/faq/", - "https://msgvault.io/guides/oauth-setup/", - "https://msgvault.io/guides/remote-deployment/", - "https://msgvault.io/guides/verification/", - "https://msgvault.io/introduction/", - "https://msgvault.io/setup/", - "https://msgvault.io/troubleshooting/", - "https://msgvault.io/usage/analytics/", - "https://msgvault.io/usage/chat/", - "https://msgvault.io/usage/deduplication/", - "https://msgvault.io/usage/deletion/", - "https://msgvault.io/usage/exporting/", - "https://msgvault.io/usage/importing/", - "https://msgvault.io/usage/multi-account/", - "https://msgvault.io/usage/querying/", - "https://msgvault.io/usage/searching/", - "https://msgvault.io/usage/text-messages/", - "https://msgvault.io/usage/tui/", - "https://msgvault.io/usage/vector-search/", - "https://msgvault.io/web-ui/", + "https://msgvault.io/docs/", + "https://msgvault.io/docs/api-server/", + "https://msgvault.io/docs/architecture/overview/", + "https://msgvault.io/docs/architecture/postgresql/", + "https://msgvault.io/docs/architecture/search-ranking/", + "https://msgvault.io/docs/architecture/storage/", + "https://msgvault.io/docs/changelog/", + "https://msgvault.io/docs/cli-reference/", + "https://msgvault.io/docs/configuration/", + "https://msgvault.io/docs/development/", + "https://msgvault.io/docs/faq/", + "https://msgvault.io/docs/guides/oauth-setup/", + "https://msgvault.io/docs/guides/remote-deployment/", + "https://msgvault.io/docs/guides/verification/", + "https://msgvault.io/docs/introduction/", + "https://msgvault.io/docs/setup/", + "https://msgvault.io/docs/troubleshooting/", + "https://msgvault.io/docs/usage/analytics/", + "https://msgvault.io/docs/usage/chat/", + "https://msgvault.io/docs/usage/deduplication/", + "https://msgvault.io/docs/usage/deletion/", + "https://msgvault.io/docs/usage/exporting/", + "https://msgvault.io/docs/usage/importing/", + "https://msgvault.io/docs/usage/multi-account/", + "https://msgvault.io/docs/usage/querying/", + "https://msgvault.io/docs/usage/searching/", + "https://msgvault.io/docs/usage/text-messages/", + "https://msgvault.io/docs/usage/tui/", + "https://msgvault.io/docs/usage/vector-search/", + "https://msgvault.io/docs/web-ui/", ] REQUIRED_METADATA = [ - '', - '', + '', + '', '', '', ] @@ -372,11 +387,11 @@ def fragment_id(fragment: str) -> str: def check_expected_asset_files() -> None: for asset in STATIC_ASSETS: - path = SITE / "assets" / "static" / asset + path = SITE / "docs" / "assets" / "static" / asset if not path.is_file(): fail(f"missing built static asset {path.relative_to(SITE)}") for asset in GENERATED_ASSETS: - path = SITE / "assets" / "generated" / asset + path = SITE / "docs" / "assets" / "generated" / asset if not path.is_file(): fail(f"missing built generated asset {path.relative_to(SITE)}") if path.suffix == ".svg": @@ -417,15 +432,21 @@ def main() -> None: check_public_site_file_inventory() - for route in ROUTES: + for route in WEBSITE_ROUTES + DOCS_ROUTES: path = route_to_file(route) if not path.exists(): fail(f"missing route {route}: {path}") - if not (SITE / "404.html").exists(): - fail("missing 404.html") + for relative in WEBSITE_FILES: + if not (SITE / relative).is_file(): + fail(f"missing website file {relative}") + if not any((SITE / "fonts").glob("*.woff2")): + fail("missing website fonts") + + if not (SITE / "docs" / "404.html").exists(): + fail("missing docs/404.html") if not (SITE / "sitemap.xml").exists(): - fail("missing sitemap.xml") + fail("missing sitemap.xml at the site root") sitemap_text = (SITE / "sitemap.xml").read_text(encoding="utf-8", errors="ignore") for url in REQUIRED_SITEMAP_URLS: if f"{url}" not in sitemap_text: @@ -434,8 +455,10 @@ def main() -> None: check_expected_asset_files() html_files = list(SITE.rglob("*.html")) - index_text = (SITE / "index.html").read_text(encoding="utf-8", errors="ignore") - if "msgvault-logo-text" in index_text: + docs_index_text = (SITE / "docs" / "index.html").read_text( + encoding="utf-8", errors="ignore" + ) + if "msgvault-logo-text" in docs_index_text: fail("header logo must be icon-only; remove msgvault-logo-text") leaked_overrides = [path for path in html_files if "overrides" in path.relative_to(SITE).parts] @@ -455,11 +478,11 @@ def main() -> None: fail(f"forbidden generated marker found: {pattern}") parsed_by_file = {path.resolve(): parse_html(path) for path in html_files} - index_parser = parsed_by_file[(SITE / "index.html").resolve()] - web_ui_route = route_to_file("/web-ui/").resolve() + docs_index = SITE / "docs" / "index.html" + index_parser = parsed_by_file[docs_index.resolve()] + web_ui_route = route_to_file("/docs/web-ui/").resolve() if not any( - (target_file(SITE / "index.html", href) or pathlib.Path()).resolve() - == web_ui_route + (target_file(docs_index, href) or pathlib.Path()).resolve() == web_ui_route for href in index_parser.nav_links ): fail("Web UI is missing from the rendered primary navigation") diff --git a/docs/scripts/check_vercel_redirects.py b/docs/scripts/check_vercel_redirects.py index 9d4d6b8e0..25aeed247 100755 --- a/docs/scripts/check_vercel_redirects.py +++ b/docs/scripts/check_vercel_redirects.py @@ -15,6 +15,27 @@ "/install.ps1": "https://raw.githubusercontent.com/kenn-io/msgvault/main/scripts/install.ps1", } +# Legacy root docs URLs permanently redirect into the /docs/ tier so links +# published before the tiered site keep resolving. +PERMANENT = { + "/introduction/:path*": "/docs/introduction/:path*", + "/setup/:path*": "/docs/setup/:path*", + "/web-ui/:path*": "/docs/web-ui/:path*", + "/configuration/:path*": "/docs/configuration/:path*", + "/cli-reference/:path*": "/docs/cli-reference/:path*", + "/api-server/:path*": "/docs/api-server/:path*", + "/changelog/:path*": "/docs/changelog/:path*", + "/troubleshooting/:path*": "/docs/troubleshooting/:path*", + "/development/:path*": "/docs/development/:path*", + "/faq/:path*": "/docs/faq/:path*", + "/usage/:path*": "/docs/usage/:path*", + "/guides/:path*": "/docs/guides/:path*", + "/architecture/:path*": "/docs/architecture/:path*", + "/assets/static/:path*": "/docs/assets/static/:path*", + "/assets/generated/:path*": "/docs/assets/generated/:path*", + "/search/:path*": "/docs/search/:path*", +} + def fail(message: str) -> None: print(f"FAIL: {message}", file=sys.stderr) @@ -63,8 +84,9 @@ def collect_redirects(data: dict[str, object]) -> dict[str, dict[str, object]]: raw_redirects = data.get("redirects", []) if not isinstance(raw_redirects, list): fail("vercel redirects must be a list") - if len(raw_redirects) != len(TEMPORARY): - fail(f"vercel redirects must contain exactly {len(TEMPORARY)} entries") + expected = len(TEMPORARY) + len(PERMANENT) + if len(raw_redirects) != expected: + fail(f"vercel redirects must contain exactly {expected} entries") redirects: dict[str, dict[str, object]] = {} for index, item in enumerate(raw_redirects): @@ -108,6 +130,13 @@ def main() -> None: if item.get("destination") != destination or item.get("permanent") is not False: fail(f"incorrect temporary redirect {source}") + for source, destination in PERMANENT.items(): + item = redirects.get(source) + if not item: + fail(f"missing permanent redirect {source}") + if item.get("destination") != destination or item.get("permanent") is not True: + fail(f"incorrect permanent redirect {source}") + print("vercel redirect checks passed") diff --git a/docs/setup.md b/docs/setup.md index c9bb5aacd..a3621a0cd 100644 --- a/docs/setup.md +++ b/docs/setup.md @@ -23,7 +23,7 @@ the PowerShell installer selects the native package when the release provides one and falls back to the AMD64 package under emulation for older releases. !!! tip "Running on a headless server?" - msgvault works on headless machines (SSH, VPS, NAS, Docker), but OAuth requires a browser for the initial authorization. You'll authorize on your local machine and copy the token file to the server. See [Headless Server Setup](/guides/oauth-setup/#headless-server-setup) for the copy-token workflow, or jump to the [Remote Deployment](/guides/remote-deployment/) guide for a full NAS/server setup with Docker Compose. + msgvault works on headless machines (SSH, VPS, NAS, Docker), but OAuth requires a browser for the initial authorization. You'll authorize on your local machine and copy the token file to the server. See [Headless Server Setup](/docs/guides/oauth-setup/#headless-server-setup) for the copy-token workflow, or jump to the [Remote Deployment](/docs/guides/remote-deployment/) guide for a full NAS/server setup with Docker Compose. Verify the installation: @@ -68,7 +68,7 @@ On Windows, use the native PowerShell helper: ``` It detects AMD64 or ARM64 automatically and writes `msgvault.exe` in the -repository root. See [Development and Roadmap](/development/#windows) for the +repository root. See [Development and Roadmap](/docs/development/#windows) for the one-time MSYS2 compiler prerequisites. Verify the installation: @@ -79,7 +79,7 @@ msgvault --help ## Configure OAuth -Create a Google Cloud project, enable the Gmail API, and download your `client_secret.json`. If you plan to archive Google Calendar, enable the Google Calendar API too. See the full [OAuth Setup Guide](/guides/oauth-setup/). +Create a Google Cloud project, enable the Gmail API, and download your `client_secret.json`. If you plan to archive Google Calendar, enable the Google Calendar API too. See the full [OAuth Setup Guide](/docs/guides/oauth-setup/). ### Where to put config.toml @@ -112,7 +112,7 @@ $env:MSGVAULT_HOME = "E:\msgvault" export MSGVAULT_HOME=/mnt/data/msgvault ``` -The `--home` flag takes priority over `MSGVAULT_HOME`. See [Configuration](/configuration/) for all options. +The `--home` flag takes priority over `MSGVAULT_HOME`. See [Configuration](/docs/configuration/) for all options. ### Create the config file @@ -134,9 +134,9 @@ client_secrets = "C:/Users/you/Downloads/client_secret.json" msgvault add-account you@gmail.com ``` -This opens your browser for OAuth consent. For headless servers, see the [copy-token workflow](/guides/oauth-setup/#headless-server-setup). +This opens your browser for OAuth consent. For headless servers, see the [copy-token workflow](/docs/guides/oauth-setup/#headless-server-setup). -If you plan to deploy to a remote host (NAS, cloud VM, etc.), run `msgvault setup` after this step to generate a ready-to-run deployment bundle with Docker Compose and remote configuration. See the [Remote Deployment](/guides/remote-deployment/) guide. +If you plan to deploy to a remote host (NAS, cloud VM, etc.), run `msgvault setup` after this step to generate a ready-to-run deployment bundle with Docker Compose and remote configuration. See the [Remote Deployment](/docs/guides/remote-deployment/) guide. ## Add an IMAP Account @@ -153,7 +153,7 @@ Common IMAP servers: | Provider | Host | Port | Notes | |---|---|---|---| | Fastmail | `imap.fastmail.com` | 993 | App password recommended | -| Outlook / Hotmail | `outlook.office365.com` | 993 | Use [`add-o365`](/guides/oauth-setup/#microsoft-365-outlook-hotmail) for OAuth (recommended); or app password with 2FA | +| Outlook / Hotmail | `outlook.office365.com` | 993 | Use [`add-o365`](/docs/guides/oauth-setup/#microsoft-365-outlook-hotmail) for OAuth (recommended); or app password with 2FA | | Yahoo | `imap.mail.yahoo.com` | 993 | [App password](#yahoo-app-passwords) required | | iCloud | `imap.mail.me.com` | 993 | App-specific password required | | Gmail (IMAP) | `imap.gmail.com` | 993 | Use `add-account` for Gmail API instead | @@ -175,10 +175,10 @@ msgvault sync-full you@fastmail.com IMAP accounts are stored in the same database as Gmail accounts. All tools (Web UI, TUI, search, MCP, and REST API) work with IMAP messages the same way. To start with only part of a large account, see -[IMAP Folder Sync](/usage/imap/) for `--folder` and `--skip-folder` examples. +[IMAP Folder Sync](/docs/usage/imap/) for `--folder` and `--skip-folder` examples. !!! tip "Microsoft 365 / Outlook.com" - For Outlook, Hotmail, Live.com, and Microsoft 365 accounts, `add-o365` provides OAuth-based access without app passwords. It auto-detects the correct IMAP host and configures XOAUTH2 authentication. See the [OAuth Setup guide](/guides/oauth-setup/#microsoft-365-outlook-hotmail) for details. + For Outlook, Hotmail, Live.com, and Microsoft 365 accounts, `add-o365` provides OAuth-based access without app passwords. It auto-detects the correct IMAP host and configures XOAUTH2 authentication. See the [OAuth Setup guide](/docs/guides/oauth-setup/#microsoft-365-outlook-hotmail) for details. @@ -228,7 +228,7 @@ Gmail's "storage used" number includes attachments at full size. Your on-disk fo - **Attachments** are extracted and stored as-is (PDFs, images, etc. are already compressed). Identical attachments across messages are deduplicated by content hash. - **Parquet analytics cache** is a lightweight projection for the Web UI and TUI — typically a few MB even for large archives. -Use `--limit` or a date range for your first sync to gauge the ratio for your mailbox before committing to a full sync. After syncing, `msgvault stats` shows the actual sizes. See [Data Storage](/architecture/storage/) for details on compression and storage layers. +Use `--limit` or a date range for your first sync to gauge the ratio for your mailbox before committing to a full sync. After syncing, `msgvault stats` shows the actual sizes. See [Data Storage](/docs/architecture/storage/) for details on compression and storage layers. ### Full Sync Flags @@ -294,10 +294,10 @@ msgvault tui msgvault stats ```
- msgvault TUI showing the Senders view + msgvault TUI showing the Senders view
-See [Web UI](/web-ui/), [Searching](/usage/searching/), and [Interactive -TUI](/usage/tui/) for more. +See [Web UI](/docs/web-ui/), [Searching](/docs/usage/searching/), and [Interactive +TUI](/docs/usage/tui/) for more. ## Optional: Sync Google Calendar @@ -310,7 +310,7 @@ msgvault sync-calendar you@gmail.com ``` Calendar sync is read-only. Events become searchable with -`--message-type calendar_event`; see [Google Calendar](/usage/calendar/) for the +`--message-type calendar_event`; see [Google Calendar](/docs/usage/calendar/) for the full workflow, scheduled sync, and headless-server setup. ## Open the Web UI @@ -330,7 +330,7 @@ runtime or separately installed web files are required. For a server or NAS, prefer HTTPS at a reverse proxy and configure only that proxy in `server.trusted_proxies`. Plain HTTP is an explicit private-network -tradeoff because its session cookie cannot be marked `Secure`. See [Web UI](/web-ui/) +tradeoff because its session cookie cannot be marked `Secure`. See [Web UI](/docs/web-ui/) for URL discovery, search/index states, keyboard controls, and deployment examples. @@ -345,7 +345,7 @@ msgvault sync-teams user@example.com ``` Teams messages become searchable with `--message-type teams`. See -[Microsoft Teams](/usage/teams/) for required Graph permissions, scheduling, +[Microsoft Teams](/docs/usage/teams/) for required Graph permissions, scheduling, and inline media backfill. ## Optional: Sync Discord @@ -361,7 +361,7 @@ msgvault sync-discord 123456789012345678 Discord messages become searchable with `--message-type discord`. The bot API is guild-only and does not expose personal direct-message history. See -[Discord](/usage/discord/) for least-privilege setup, scheduling, filters, +[Discord](/docs/usage/discord/) for least-privilege setup, scheduling, filters, repair behavior, and attachment limits. ## Optional: Configure Backups @@ -382,5 +382,5 @@ Record the repository in `config.toml` so future commands can omit `--repo`: repo = "~/Backups/msgvault" ``` -See [Backup](/usage/backup/) for restore, verification, scheduling, and +See [Backup](/docs/usage/backup/) for restore, verification, scheduling, and secret-handling details. diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 35e8609a4..c63e1ddc7 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -21,7 +21,7 @@ client_secrets = "C:/Users/you/Downloads/client_secret.json" client_secrets = 'C:\Users\you\Downloads\client_secret.json' ``` -See [Configuration: Windows paths](/configuration/#windows-paths) for more detail. +See [Configuration: Windows paths](/docs/configuration/#windows-paths) for more detail. ## OAuth Errors @@ -35,7 +35,7 @@ The Gmail scope isn't configured. Verify you added `gmail.modify` on the **Data ### "Access blocked" or "app not approved" for Workspace accounts -The Workspace organization restricts OAuth to apps created within their org. Create a separate Google Cloud project inside the org and configure it as a named OAuth app. See [OAuth Setup: Google Workspace Accounts](/guides/oauth-setup/#google-workspace-accounts). +The Workspace organization restricts OAuth to apps created within their org. Create a separate Google Cloud project inside the org and configure it as a named OAuth app. See [OAuth Setup: Google Workspace Accounts](/docs/guides/oauth-setup/#google-workspace-accounts). ### "redirect_uri_mismatch" @@ -165,7 +165,7 @@ docker logs msgvault Common causes: - Missing `config.toml` with `[oauth] client_secrets` and `[server] bind_addr = "0.0.0.0"` + `api_key` - Port 8080 already in use — change `api_port` in config or the port mapping in docker-compose.yml -- Volume mount permissions — on Synology, add `user: root` to docker-compose.yml (see [Platform Notes](/guides/remote-deployment/#platform-notes)) +- Volume mount permissions — on Synology, add `user: root` to docker-compose.yml (see [Platform Notes](/docs/guides/remote-deployment/#platform-notes)) ### `permission denied` on Synology @@ -281,21 +281,21 @@ If the TUI launches but shows no data: ### "api_key is required for non-loopback bind address" -You set `bind_addr` to a non-loopback address (e.g., `0.0.0.0`) without configuring an API key. Either add an `api_key` to your `[server]` config, or set `allow_insecure = true` if you understand the security implications. See [Web UI & API Server: Security Model](/api-server/#security-model). +You set `bind_addr` to a non-loopback address (e.g., `0.0.0.0`) without configuring an API key. Either add an `api_key` to your `[server]` config, or set `allow_insecure = true` if you understand the security implications. See [Web UI & API Server: Security Model](/docs/api-server/#security-model). ### The UI reports a missing, stale, or unavailable analytical cache Run `msgvault build-cache`, then restart a daemon that was already running. The Web UI preserves the reported cache state instead of silently switching some -modalities to a different query engine. See [Web UI: Cache states](/web-ui/#cache-states) -and [Configuration: analytics](/configuration/#analytics). +modalities to a different query engine. See [Web UI: Cache states](/docs/web-ui/#cache-states) +and [Configuration: analytics](/docs/configuration/#analytics). ### The UI warns that the session cookie is not secure The browser reached a remote daemon over plain HTTP. Prefer HTTPS at a reverse proxy and configure that proxy in `server.trusted_proxies`. Do not trust a whole client network merely to suppress the warning. See [Web UI: Remote access and -HTTPS](/web-ui/#remote-access-and-https). +HTTPS](/docs/web-ui/#remote-access-and-https). ### Port already in use @@ -353,7 +353,7 @@ msgvault tui --log-sql-slow-ms 50 msgvault tui --log-sql ``` -Log files are stored in `/logs/` by default. Run `msgvault logs --path` to print the selected daemon's directory. See [Configuration: Log](/configuration/#log) for all options. +Log files are stored in `/logs/` by default. Run `msgvault logs --path` to print the selected daemon's directory. See [Configuration: Log](/docs/configuration/#log) for all options. ## Still Stuck? diff --git a/docs/usage/analytics.md b/docs/usage/analytics.md index 11db8c392..c4bde079e 100644 --- a/docs/usage/analytics.md +++ b/docs/usage/analytics.md @@ -14,7 +14,7 @@ msgvault stats Displays total message count, account breakdown, date range, storage size, and attachment count.
- msgvault stats command output + msgvault stats command output
## List Senders @@ -37,9 +37,9 @@ msgvault list-domains --limit 20 msgvault list-labels ```
- msgvault list-senders command output + msgvault list-senders command output
These commands query the configured daemon or remote server. For interactive -exploration, use the [Web UI](/web-ui/) to combine search, filters, grouping, +exploration, use the [Web UI](/docs/web-ui/) to combine search, filters, grouping, and modality-aware drill-down in a shareable URL-backed context. The -[TUI](/usage/tui/) provides a terminal-native analytical workflow. +[TUI](/docs/usage/tui/) provides a terminal-native analytical workflow. diff --git a/docs/usage/backup.md b/docs/usage/backup.md index 30a063ebf..742e61aa7 100644 --- a/docs/usage/backup.md +++ b/docs/usage/backup.md @@ -221,4 +221,4 @@ The flip side is a privacy consideration: purging a message from the archive doe ## Compatibility -The repository records a format version and a **minimum reader version**. A newer msgvault that changes the format in a way old readers cannot safely handle will raise the minimum reader version; an older msgvault opening such a repository refuses cleanly with an upgrade message instead of misreading it. Every binary object in the repository additionally carries its own magic number, version, and SHA-256 integrity trailer, and each snapshot manifest records the msgvault version that wrote it. See [Backup Repository Format](/architecture/backup-format/) for how msgvault integrates with the backup engine; the full on-disk format specification lives with the engine, in `backup/FORMAT.md` of [`go.kenn.io/kit`](https://go.kenn.io/kit). +The repository records a format version and a **minimum reader version**. A newer msgvault that changes the format in a way old readers cannot safely handle will raise the minimum reader version; an older msgvault opening such a repository refuses cleanly with an upgrade message instead of misreading it. Every binary object in the repository additionally carries its own magic number, version, and SHA-256 integrity trailer, and each snapshot manifest records the msgvault version that wrote it. See [Backup Repository Format](/docs/architecture/backup-format/) for how msgvault integrates with the backup engine; the full on-disk format specification lives with the engine, in `backup/FORMAT.md` of [`go.kenn.io/kit`](https://go.kenn.io/kit). diff --git a/docs/usage/calendar.md b/docs/usage/calendar.md index fe07281a7..da0965435 100644 --- a/docs/usage/calendar.md +++ b/docs/usage/calendar.md @@ -14,7 +14,7 @@ anything on your Google Calendar. ## Prerequisites -- An OAuth client already configured for Gmail (see [OAuth Setup](/guides/oauth-setup/)). +- An OAuth client already configured for Gmail (see [OAuth Setup](/docs/guides/oauth-setup/)). Calendar reuses the same `client_secret.json`. - The **Google Calendar API** enabled on that OAuth project. In the [Google Cloud Console](https://console.cloud.google.com/), go to @@ -110,7 +110,7 @@ msgvault search "standup" --message-type calendar_event msgvault search "after:2024-01-01 before:2024-04-01" --message-type calendar_event ``` -When [vector search](/usage/vector-search/) is enabled, events become eligible +When [vector search](/docs/usage/vector-search/) is enabled, events become eligible for embedding after sync and can be found semantically with `--mode vector` or `--mode hybrid` once the embedding worker has processed them. For manual `sync-calendar` runs, follow up with `msgvault embeddings build`. In the @@ -132,7 +132,7 @@ enabled = true ``` The first scheduled run full-syncs and registers calendars; later runs are -incremental. See [Configuration](/configuration/#google-calendar-sources) for +incremental. See [Configuration](/docs/configuration/#google-calendar-sources) for every field. !!! note @@ -183,7 +183,7 @@ Workspace admins using domain-wide delegation do not need per-user browser tokens for Calendar. Enable the Google Calendar API, authorize the service account client ID for `https://www.googleapis.com/auth/calendar.readonly`, and configure `[oauth].service_account_key` or `[oauth.apps.].service_account_key` -as described in [OAuth Setup](/guides/oauth-setup/#google-workspace-service-accounts). +as described in [OAuth Setup](/docs/guides/oauth-setup/#google-workspace-service-accounts). Then sync the account directly or add a scheduled `[[gcal]]` entry: diff --git a/docs/usage/chat.md b/docs/usage/chat.md index 6e972fb1e..1a9779b58 100644 --- a/docs/usage/chat.md +++ b/docs/usage/chat.md @@ -3,7 +3,7 @@ title: MCP Server description: Expose your email, chat, calendar, and meeting archive to AI assistants via MCP. --- -The MCP server operates on your msgvault archive through the selected daemon, not your live Gmail account. Without `[remote].url`, `msgvault mcp` starts or reuses the local background daemon; with `[remote].url`, it uses that remote server. The AI cannot send emails, modify labels, or access your Google credentials. Standard read and search operations go through the daemon. If [vector search](/usage/vector-search/) is enabled, semantic and hybrid searches also call the embedding endpoint configured in `[vector.embeddings]`; use a local or self-hosted endpoint if message text must stay on your machine or network. The `stage_deletion` tool asks the selected daemon to save a deletion manifest, and `export_attachment` saves an attachment to a requested path on the MCP server's filesystem. Neither modifies the database, and actual deletion still requires you to run `msgvault delete-staged` from the CLI. You control when data enters the archive (via sync and import commands) and when anything is deleted (via the explicit [deletion workflow](/usage/deletion/)). Compared to giving an AI assistant direct OAuth access to your mailbox, this is a fundamentally smaller attack surface. +The MCP server operates on your msgvault archive through the selected daemon, not your live Gmail account. Without `[remote].url`, `msgvault mcp` starts or reuses the local background daemon; with `[remote].url`, it uses that remote server. The AI cannot send emails, modify labels, or access your Google credentials. Standard read and search operations go through the daemon. If [vector search](/docs/usage/vector-search/) is enabled, semantic and hybrid searches also call the embedding endpoint configured in `[vector.embeddings]`; use a local or self-hosted endpoint if message text must stay on your machine or network. The `stage_deletion` tool asks the selected daemon to save a deletion manifest, and `export_attachment` saves an attachment to a requested path on the MCP server's filesystem. Neither modifies the database, and actual deletion still requires you to run `msgvault delete-staged` from the CLI. You control when data enters the archive (via sync and import commands) and when anything is deleted (via the explicit [deletion workflow](/docs/usage/deletion/)). Compared to giving an AI assistant direct OAuth access to your mailbox, this is a fundamentally smaller attack surface. ## Setup @@ -94,7 +94,7 @@ The MCP server exposes the following tools to connected AI clients: | `search_messages` | Deprecated compatibility wrapper. Omitted mode dispatches to `search_metadata`; `vector`/`hybrid` dispatch to `semantic_search_messages`. | `query` (string, required), `mode` (string: `vector`/`hybrid`), `explain` (bool), `min_score` (number), `limit` (int), `offset` (int), `account` (string) | | `search_metadata` | Search message metadata with a subset of Gmail query syntax (not full Gmail compatibility). Matches subject, snippet, and sender/recipient metadata, not message bodies. | `query` (string, required), `limit` (int), `offset` (int), `account` (string) | | `search_message_bodies` | Keyword full-text search inside message bodies. Returns `matches` excerpts (up to 5 per message), ordered newest-first. Backend excerpts may omit `char_offset` and `line`; use `search_in_message` when exact locations are needed. | `query` (string, required), `limit` (int), `offset` (int), `account` (string) | -| `semantic_search_messages` | Semantic search over preprocessed message subjects and bodies when [vector search](/usage/vector-search/) is configured. Returns scored chunk excerpts; `min_score` filters excerpts, not ranked messages. | `query` (string, required), `mode` (string: `vector`/`hybrid`, default `hybrid`), `explain` (bool), `min_score` (number), `limit` (int), `offset` (int), `account` (string) | +| `semantic_search_messages` | Semantic search over preprocessed message subjects and bodies when [vector search](/docs/usage/vector-search/) is configured. Returns scored chunk excerpts; `min_score` filters excerpts, not ranked messages. | `query` (string, required), `mode` (string: `vector`/`hybrid`, default `hybrid`), `explain` (bool), `min_score` (number), `limit` (int), `offset` (int), `account` (string) | | `search_in_message` | Find case-insensitive literal matches within one message body, with raw-body offsets and line numbers. | `id` (int, required), `query` (string, required), `limit` (int), `offset` (int) | | `find_similar_messages` | Nearest-neighbor search from a seed message's embedding. Requires vector search to be configured and an active index generation. | `message_id` (int, required), `limit` (int), `account` (string), `message_type` (string), `after` (string), `before` (string), `has_attachment` (bool) | | `search_by_domains` | Find messages where any participant (`from`, `to`, or `cc`) belongs to one of several domains, regardless of direction. | `domains` (comma-separated string, required), `limit` (int), `offset` (int), `after` (string), `before` (string) | @@ -195,7 +195,7 @@ Claude will automatically call the appropriate msgvault tools to retrieve and an The `stage_deletion` tool lets an AI assistant help you clean up your inbox. It accepts either a Gmail-style query string or structured filters (sender, domain, label, date range), but not both at once. Results are capped at 100,000 messages per call. -When called, `stage_deletion` creates a pending deletion manifest through the selected daemon. With a remote server configured, the manifest is saved on that remote host; otherwise it is saved by the local daemon. It does **not** delete anything. To execute the deletion, you must run `msgvault delete-staged` from the CLI. See [Deleting Email](/usage/deletion/) for the full workflow. +When called, `stage_deletion` creates a pending deletion manifest through the selected daemon. With a remote server configured, the manifest is saved on that remote host; otherwise it is saved by the local daemon. It does **not** delete anything. To execute the deletion, you must run `msgvault delete-staged` from the CLI. See [Deleting Email](/docs/usage/deletion/) for the full workflow. The tool returns the batch ID, message count, and next steps: @@ -220,7 +220,7 @@ msgvault mcp --http 8080 | Flag | Default | Description | |---|---|---| -| `--force-sql` | `false` | Deprecated in 0.17.0; use `[analytics].engine = "sql"` in `config.toml` instead. See [Configuration: analytics](/configuration/#analytics). | +| `--force-sql` | `false` | Deprecated in 0.17.0; use `[analytics].engine = "sql"` in `config.toml` instead. See [Configuration: analytics](/docs/configuration/#analytics). | | `--no-sqlite-scanner` | `false` | Deprecated in 0.17.0; cache engine selection is daemon-managed. Use `[analytics].engine = "sql"` for live SQL. | | `--http` | — | Serve over MCP StreamableHTTP instead of stdio. Bare ports bind to `127.0.0.1`; non-loopback addresses require `[server].api_key` or `--http-allow-insecure`. | | `--http-allow-insecure` | `false` | Allow non-loopback HTTP binding without `[server].api_key`. A configured key is still enforced. Without a key, use only behind your own network or authentication layer. | @@ -238,5 +238,5 @@ msgvault skills install ``` The skills teach agents the CLI; the MCP server exposes structured tool calls. -They can be used independently or together. See [Agent Skills](/guides/agent-skills/) +They can be used independently or together. See [Agent Skills](/docs/guides/agent-skills/) for installation targets, update behavior, and uninstall instructions. diff --git a/docs/usage/deduplication.md b/docs/usage/deduplication.md index cd9c10156..fd6bbaab3 100644 --- a/docs/usage/deduplication.md +++ b/docs/usage/deduplication.md @@ -9,7 +9,7 @@ A long-running archive accumulates overlapping sources: a current Gmail sync, an The defining principle: **deduplication hides redundant copies, it does not delete them.** One survivor stays visible. The other copies drop out of normal reads but remain on disk, and `--undo` restores them. Removing data is always a separate, explicit step that you opt into.
- Deduplication keeps one survivor visible per duplicate group and hides the other copies, which remain on disk. Deleting those copies is a separate step. + Deduplication keeps one survivor visible per duplicate group and hides the other copies, which remain on disk. Deleting those copies is a separate step.
## How Duplicates Are Detected @@ -25,7 +25,7 @@ The two passes are sequential, not merged into one transitive set. A content-has Survivor selection is deterministic and explainable, and the reasoning is printed in dry-run output. It runs in two stages. -**Stage 1, sent-copy eligibility.** If any message in a group looks like a copy you sent, only sent copies are eligible to survive, and received copies drop out before tie-breaking. A message looks sent when any of these is true: it carries a Gmail `SENT` label, ingest metadata flagged it as from you, or its `From` address matches a confirmed [identity](/usage/multi-account/#identities) for that account. The reasoning is that "I sent this" is harder to recover from data than "I received this," so a richer received copy is never allowed to silently win. +**Stage 1, sent-copy eligibility.** If any message in a group looks like a copy you sent, only sent copies are eligible to survive, and received copies drop out before tie-breaking. A message looks sent when any of these is true: it carries a Gmail `SENT` label, ingest metadata flagged it as from you, or its `From` address matches a confirmed [identity](/docs/usage/multi-account/#identities) for that account. The reasoning is that "I sent this" is harder to recover from data than "I received this," so a richer received copy is never allowed to silently win. **Stage 2, priority list.** Among the eligible copies, msgvault prefers, in order: @@ -41,7 +41,7 @@ Survivor selection is deterministic and explainable, and the reasoning is printe Earlier rules win outright; later rules apply only when all earlier ones tie. The attachment-count, attachment-presence, and payload-size rules apply only when both copies have raw MIME and their normalized MIME hashes match. A shared `Message-ID` alone cannot make those payload-completeness signals authoritative. The survivor inherits the union of labels from the copies it replaces, and backfills raw MIME from a non-survivor if it was missing the original payload.
- Survivor selection runs the sent-copy eligibility filter first, then a priority list: source preference, raw MIME, more attachments, an attachment-presence signal, a larger payload, richer labels, earlier archive time, and finally a stable row ID. + Survivor selection runs the sent-copy eligibility filter first, then a priority list: source preference, raw MIME, more attachments, an attachment-presence signal, a larger payload, richer labels, earlier archive time, and finally a stable row ID.
## Choosing a Scope @@ -63,7 +63,7 @@ This protects sent-message provenance. If Alice's Sent copy and Bob's Inbox copy Every dedup-related command sits on one of five rungs (00 through 04). Rung 00 is an automatic backup; the others you climb deliberately, one explicit action at a time. msgvault never escalates from one rung to the next on its own: applying dedup never implies a local hard delete, and a local hard delete never implies a remote delete.
- The safety ladder: five rungs, 00 through 04. Rung 00 is an automatic SQLite-only backup (PostgreSQL uses pg_dump); rungs 01 scan, 02 hide, 03 local hard delete, and 04 remote delete are deliberate, opt-in actions. Remote deletes go to Gmail trash by default but are permanent on IMAP. Deletion is never required. + The safety ladder: five rungs, 00 through 04. Rung 00 is an automatic SQLite-only backup (PostgreSQL uses pg_dump); rungs 01 scan, 02 hide, 03 local hard delete, and 04 remote delete are deliberate, opt-in actions. Remote deletes go to Gmail trash by default but are permanent on IMAP. Deletion is never required.
| Rung | Action | Command | Reversibility | @@ -81,7 +81,7 @@ Every dedup-related command sits on one of five rungs (00 through 04). Rung 00 i - **Rung 01, scan.** `deduplicate --dry-run` reports the duplicate groups it found, the proposed survivor for each, and why. Nothing is modified. - **Rung 02, hide.** `deduplicate` applies the scan. Pruned copies are hidden from normal reads but kept on disk, and the run prints a batch ID. `--undo ` restores them. - **Rung 03, local hard delete.** `delete-deduped` permanently removes hidden rows from the local archive to reclaim disk. It acts on named batches via `--batch` and refuses to touch rows it did not hide; all selected batches commit as one transaction, so cancellation rolls the whole selection back. `--all-hidden` purges every hidden row and always prompts for confirmation. Undo cannot recover purged rows. -- **Rung 04, remote delete.** This rung is two parts, stage then execute, and only the staging part is dedup-specific. To stage, run `deduplicate --delete-dups-from-source-server`; it writes pending deletion manifests only when the loser and survivor share a source and have matching normalized raw MIME. A group spanning two sources or lacking content equivalence stages nothing. To execute, run `delete-staged`, the generic executor for any staged deletion manifest (not just dedup), which acts on the source server and leaves your local archive untouched. Inspect first with `delete-staged --list`, target one batch with `delete-staged `, and note that execution is gated behind `MSGVAULT_ENABLE_REMOTE_DELETE=1`. The same-source and content-equivalence restrictions live in the staging step, not in `delete-staged`. See [Deleting Email](/usage/deletion/) for how remote deletion works. +- **Rung 04, remote delete.** This rung is two parts, stage then execute, and only the staging part is dedup-specific. To stage, run `deduplicate --delete-dups-from-source-server`; it writes pending deletion manifests only when the loser and survivor share a source and have matching normalized raw MIME. A group spanning two sources or lacking content equivalence stages nothing. To execute, run `delete-staged`, the generic executor for any staged deletion manifest (not just dedup), which acts on the source server and leaves your local archive untouched. Inspect first with `delete-staged --list`, target one batch with `delete-staged `, and note that execution is gated behind `MSGVAULT_ENABLE_REMOTE_DELETE=1`. The same-source and content-equivalence restrictions live in the staging step, not in `delete-staged`. See [Deleting Email](/docs/usage/deletion/) for how remote deletion works. !!! note "What \"hidden\" means" A hidden copy is excluded from search, the Web UI, the TUI, vector and hybrid retrieval, the API, MCP responses, exports, and stats, while still living on disk. Every read path applies the same visibility rule, so a hidden duplicate cannot leak back into results through one backend. @@ -152,7 +152,7 @@ msgvault delete-staged --list MSGVAULT_ENABLE_REMOTE_DELETE=1 msgvault delete-staged ``` -See [Deleting Email](/usage/deletion/) for the full workflow. +See [Deleting Email](/docs/usage/deletion/) for the full workflow. ## What Undo Restores @@ -178,4 +178,4 @@ msgvault embeddings build --full-rebuild ## Command Reference -See the [CLI Reference](/cli-reference/#deduplicate) for the complete flag list on `deduplicate`, `delete-deduped`, `identity`, and `collection`. +See the [CLI Reference](/docs/cli-reference/#deduplicate) for the complete flag list on `deduplicate`, `delete-deduped`, `identity`, and `collection`. diff --git a/docs/usage/deletion.md b/docs/usage/deletion.md index c98bb80cd..affa87b4d 100644 --- a/docs/usage/deletion.md +++ b/docs/usage/deletion.md @@ -34,7 +34,7 @@ staging an all-matching selection that could span more than one account. The fastest way to clean up your inbox is through the TUI's aggregate views. Navigate to the Senders, Domains, or Labels view, find the group you want to remove (e.g., a prolific spam sender or an unwanted mailing list), and press `D` to stage every message in that group for deletion at once.
- msgvault TUI deletion confirmation dialog showing bulk staging of all messages from a sender + msgvault TUI deletion confirmation dialog showing bulk staging of all messages from a sender
A confirmation dialog shows exactly how many messages will be staged. Nothing is deleted until you explicitly run `msgvault delete-staged`. @@ -42,21 +42,21 @@ A confirmation dialog shows exactly how many messages will be staged. Nothing is For finer control, drill into any group and use `Space` to select individual rows, then press `d` to stage only the selected messages.
- msgvault TUI with rows selected for deletion staging + msgvault TUI with rows selected for deletion staging
## Staging via MCP (AI-Assisted) -You can also stage deletions through the [MCP server](/usage/chat/) by asking an AI assistant like Claude to find and stage messages for you. For example: +You can also stage deletions through the [MCP server](/docs/usage/chat/) by asking an AI assistant like Claude to find and stage messages for you. For example: - *"Stage all messages from noreply@linkedin.com for deletion"* - *"Stage all promotional emails older than 2024-01-01"* -The MCP `stage_deletion` tool creates a manifest through the selected daemon, the same format as TUI-staged deletions. Nothing is deleted until you run `msgvault delete-staged` from the CLI. See [MCP Server](/usage/chat/#staged-deletion-via-mcp) for details. +The MCP `stage_deletion` tool creates a manifest through the selected daemon, the same format as TUI-staged deletions. Nothing is deleted until you run `msgvault delete-staged` from the CLI. See [MCP Server](/docs/usage/chat/#staged-deletion-via-mcp) for details. ## Staging via HTTP API Web dashboards and automation scripts can stage deletion manifests through the -[web API](/api-server/#post-apiv1deletions) without constructing a manifest +[web API](/docs/api-server/#post-apiv1deletions) without constructing a manifest themselves. `POST /api/v1/deletions` accepts structured filters and/or internal message IDs, resolves the Gmail IDs on the server, and supports `"dry_run": true` to preview the count and a sample before writing anything. @@ -64,7 +64,7 @@ internal message IDs, resolves the Gmail IDs on the server, and supports Actually staging depends on the request shape. An explicit `message_ids` list stages directly. Filter-based staging requires a preflighted selection: run the reviewed predicate through -[`POST /api/v1/explore/preflight`](/api-server/#post-apiv1explorepreflight) +[`POST /api/v1/explore/preflight`](/docs/api-server/#post-apiv1explorepreflight) to get a single-use `operation_token` (valid for five minutes), then stage with the same selection and token. A non-dry-run filter request without a preflighted selection is rejected with `428 preflight_required`. diff --git a/docs/usage/discord.md b/docs/usage/discord.md index 8986cdf36..201df7865 100644 --- a/docs/usage/discord.md +++ b/docs/usage/discord.md @@ -278,10 +278,10 @@ msgvault query --format table " " ``` -In the [TUI](/usage/tui/), press `m` to switch to Texts mode. Discord channels +In the [TUI](/docs/usage/tui/), press `m` to switch to Texts mode. Discord channels and threads appear alongside other chat conversations. The HTTP search and message endpoints use the same `message_type = discord` filter; see the -[Web Server](/api-server/). If vector search is enabled, run +[Web Server](/docs/api-server/). If vector search is enabled, run `msgvault embeddings build` after a manual sync or enable `[vector.embed.schedule].run_after_sync` for scheduled syncs. diff --git a/docs/usage/document-indexing.md b/docs/usage/document-indexing.md index 8604ae09c..586dacae7 100644 --- a/docs/usage/document-indexing.md +++ b/docs/usage/document-indexing.md @@ -83,7 +83,7 @@ authenticated operation: export MISTRAL_API_KEY="..." ``` -See the [configuration reference](/configuration/#attachmentsdocuments) for +See the [configuration reference](/docs/configuration/#attachmentsdocuments) for all policy and run limits. ## Build and validate the synthetic fixtures diff --git a/docs/usage/importing.md b/docs/usage/importing.md index 07fee8fff..b92b5e0b7 100644 --- a/docs/usage/importing.md +++ b/docs/usage/importing.md @@ -3,7 +3,7 @@ title: Importing Local Email description: Import PST archives, MBOX archives, and Apple Mail exports into msgvault. --- -msgvault can import email from local files, not just Gmail. This lets you archive Microsoft Outlook PST files, messages from any provider that supports MBOX export, or Apple Mail's on-disk storage. For live syncing from non-Gmail providers, see [IMAP account setup](/setup/#add-an-imap-account). +msgvault can import email from local files, not just Gmail. This lets you archive Microsoft Outlook PST files, messages from any provider that supports MBOX export, or Apple Mail's on-disk storage. For live syncing from non-Gmail providers, see [IMAP account setup](/docs/setup/#add-an-imap-account). Imported messages are stored in the same database as Gmail messages. You can search, browse, export, and analyze them with all the same tools (Web UI, TUI, CLI, MCP server, and REST API). Labels, threading, attachments, and full-text search all work the same way. diff --git a/docs/usage/meetings.md b/docs/usage/meetings.md index c55be3fb4..4bfbbf696 100644 --- a/docs/usage/meetings.md +++ b/docs/usage/meetings.md @@ -147,7 +147,7 @@ without a limit to continue normal incremental operation. ### Browse in the Web UI or TUI -Start `msgvault serve` and open the [Web UI](/web-ui/) to include meetings in +Start `msgvault serve` and open the [Web UI](/docs/web-ui/) to include meetings in Everything, search their titles and transcripts, group them with other archive modalities, or filter to meeting notes only. Open a result to read the note in its containing context. diff --git a/docs/usage/multi-account.md b/docs/usage/multi-account.md index 6af3ed4ea..10d393384 100644 --- a/docs/usage/multi-account.md +++ b/docs/usage/multi-account.md @@ -18,21 +18,21 @@ msgvault introduces three concepts, always in the same order: account, then iden **A collection is a named group of accounts.** The `All` collection exists by default and contains every account. You create others (`work`, `personal`, or any grouping you like) to search, report, and deduplicate a logical group without changing the underlying sources. A collection is the boundary for every cross-account operation. A collection's identity is the union of its member accounts' identities, computed at read time, so you never manage it directly. Collections contain accounts only, never other collections.
- Accounts on the left are individual ingest sources, each carrying the identifiers that mean you inside that source. Collections on the right are named groups of accounts: All contains every account, with Personal and Work as deliberate subsets. + Accounts on the left are individual ingest sources, each carrying the identifiers that mean you inside that source. Collections on the right are named groups of accounts: All contains every account, with Personal and Work as deliberate subsets.
-Deduplication operates over all three concepts and has its own [Deduplication](/usage/deduplication/) page. +Deduplication operates over all three concepts and has its own [Deduplication](/docs/usage/deduplication/) page. ## OAuth Apps and Tokens For personal Gmail accounts, a single `client_secret.json` supports all of them. Each `add-account` call authorizes one account and stores a separate token file. -Google Workspace organizations often restrict OAuth to apps created within their own org. If a Workspace account fails to authorize with your default app, create a separate OAuth app inside that org and add it as a named app in `config.toml`. See the [OAuth Setup Guide](/guides/oauth-setup/#google-workspace-accounts) for the full walkthrough. +Google Workspace organizations often restrict OAuth to apps created within their own org. If a Workspace account fails to authorize with your default app, create a separate OAuth app inside that org and add it as a named app in `config.toml`. See the [OAuth Setup Guide](/docs/guides/oauth-setup/#google-workspace-accounts) for the full walkthrough. Workspace admins can also use a Google service account with domain-wide delegation. Configure `service_account_key` under `[oauth]` or `[oauth.apps.]`, authorize the service account client in the Google Admin Console, then run `msgvault add-account user@domain.com`. Service-account accounts do not store per-user refresh tokens; msgvault mints delegated tokens on demand.
- Two OAuth apps and the token files they create. A default app (config block [oauth]) authorizes personal Gmail accounts personal@gmail.com and other@gmail.com; a named app ([oauth.apps.acme]) authorizes the Workspace account you@acme.com. Each add-account run writes its own token file under ~/.msgvault/tokens/, color-matched to its account. + Two OAuth apps and the token files they create. A default app (config block [oauth]) authorizes personal Gmail accounts personal@gmail.com and other@gmail.com; a named app ([oauth.apps.acme]) authorizes the Workspace account you@acme.com. Each add-account run writes its own token file under ~/.msgvault/tokens/, color-matched to its account.
## Adding Accounts @@ -87,7 +87,7 @@ Each account has a confirmed "me" identity: the email addresses, phone numbers, Source identities are different from the observed people and durable profiles used by relationship exploration. See [People, Profiles, and Source -Identities](/usage/people/) for evidence discovery, bulk import, optional +Identities](/docs/usage/people/) for evidence discovery, bulk import, optional Fastmail alias inventory, person promotion, and typed attributes. New Gmail, IMAP, Microsoft 365, MBOX, EMLX, WhatsApp, and Google Voice sources auto-confirm the source identifier by default. Use `--no-default-identity` on supported add/import commands when that is not correct. (iMessage imports are exempt, because iMessage contacts are not self-identifying.) @@ -156,7 +156,7 @@ msgvault stats --collection Work ## Deduplication -Once several accounts hold overlapping copies of the same message, [deduplication](/usage/deduplication/) collapses each set to one visible survivor while keeping every source's provenance intact. It hides redundant copies rather than deleting them, and every step beyond hiding is a separate, opt-in action. See the [Deduplication](/usage/deduplication/) page for the detection rules, survivor selection, and the reversible safety ladder. +Once several accounts hold overlapping copies of the same message, [deduplication](/docs/usage/deduplication/) collapses each set to one visible survivor while keeping every source's provenance intact. It hides redundant copies rather than deleting them, and every step beyond hiding is a separate, opt-in action. See the [Deduplication](/docs/usage/deduplication/) page for the detection rules, survivor selection, and the reversible safety ladder. ## TUI Filtering @@ -168,4 +168,4 @@ Email account filter. ## Command Reference -See the [CLI Reference](/cli-reference/#add-account) for the complete flag list on `add-account`, `add-imap`, `add-o365`, `identity`, and `collection`. +See the [CLI Reference](/docs/cli-reference/#add-account) for the complete flag list on `add-account`, `add-imap`, `add-o365`, `identity`, and `collection`. diff --git a/docs/usage/querying.md b/docs/usage/querying.md index 2d81cfbfe..a930ee9d7 100644 --- a/docs/usage/querying.md +++ b/docs/usage/querying.md @@ -218,4 +218,4 @@ msgvault query " ## See Also -For pre-built analytics commands (top senders, domains, labels, overall stats), see [Analytics & Stats](/usage/analytics/). The `query` command is for when you need more flexibility than those commands provide. +For pre-built analytics commands (top senders, domains, labels, overall stats), see [Analytics & Stats](/docs/usage/analytics/). The `query` command is for when you need more flexibility than those commands provide. diff --git a/docs/usage/searching.md b/docs/usage/searching.md index 1145ca6f4..3a23abb10 100644 --- a/docs/usage/searching.md +++ b/docs/usage/searching.md @@ -92,7 +92,7 @@ msgvault search --collection Work The two flags are mutually exclusive. Collection filters work in full-text, vector, and hybrid local search modes. -SQLite FTS ranking is weighted to better match PostgreSQL-backed search behavior, so subject/body weighting should feel more consistent across local tools. The rankers are still different; see [Search Ranking Across Backends](/architecture/search-ranking/). +SQLite FTS ranking is weighted to better match PostgreSQL-backed search behavior, so subject/body weighting should feel more consistent across local tools. The rankers are still different; see [Search Ranking Across Backends](/docs/architecture/search-ranking/). ## Source-Deleted Messages @@ -165,5 +165,5 @@ The same `msgvault search` command supports semantic search when the selected local daemon or remote server has `[vector]` configured with an embedding endpoint. Pass `--mode vector` for pure semantic search, or `--mode hybrid` to fuse -BM25 and vector ranking. See [Vector Search](/usage/vector-search/) +BM25 and vector ranking. See [Vector Search](/docs/usage/vector-search/) for setup, initial embedding, and incremental update workflows. diff --git a/docs/usage/slack.md b/docs/usage/slack.md index 15b5c0ced..42f02cac0 100644 --- a/docs/usage/slack.md +++ b/docs/usage/slack.md @@ -138,7 +138,7 @@ schedule = "*/30 * * * *" ``` The daemon then syncs every registered workspace on the schedule. See -[Configuration](/configuration/#slack) for the full option list +[Configuration](/docs/configuration/#slack) for the full option list (channel include/exclude filters, media caps). ## Identity unification diff --git a/docs/usage/teams.md b/docs/usage/teams.md index 1c5ff984c..e74878a02 100644 --- a/docs/usage/teams.md +++ b/docs/usage/teams.md @@ -19,7 +19,7 @@ separate Graph token under `tokens/teams_.json`. An Outlook IMAP token created by `add-o365` does not authorize Teams sync. Register a Microsoft Entra app as described in -[OAuth Setup](/guides/oauth-setup/#microsoft-365-outlook-hotmail), with: +[OAuth Setup](/docs/guides/oauth-setup/#microsoft-365-outlook-hotmail), with: - Redirect URI: `http://localhost:8089/callback/microsoft` - Public client flows enabled @@ -148,7 +148,7 @@ search is enabled and you want newly synced Teams messages in semantic/hybrid results, run `msgvault embeddings build` after the sync, or configure `[vector.embed.schedule].run_after_sync = true` for scheduled daemon syncs. -In the [Web UI](/web-ui/), Teams direct chats, group chats, and channel +In the [Web UI](/docs/web-ui/), Teams direct chats, group chats, and channel conversations appear as conversation rows in Everything and can be combined with the same search, filters, and grouping as other archive modalities. In -the [TUI](/usage/tui/), press `m` to switch from Email mode to Texts mode. +the [TUI](/docs/usage/tui/), press `m` to switch from Email mode to Texts mode. diff --git a/docs/usage/text-messages.md b/docs/usage/text-messages.md index 517dbf390..5aa452712 100644 --- a/docs/usage/text-messages.md +++ b/docs/usage/text-messages.md @@ -5,11 +5,11 @@ description: Import chats and texts from common exports, and browse synchronized msgvault can import chats and text messages from WhatsApp, iMessage, Google Voice, Facebook Messenger, and SMS Backup & Restore. It can also sync Microsoft -Teams chats and channels through [Microsoft Teams](/usage/teams/) and Discord -guild channels and threads through [Discord](/usage/discord/). These records -are stored in the same database as email. The [Web UI](/web-ui/) presents chats +Teams chats and channels through [Microsoft Teams](/docs/usage/teams/) and Discord +guild channels and threads through [Discord](/docs/usage/discord/). These records +are stored in the same database as email. The [Web UI](/docs/web-ui/) presents chats as conversation rows in Everything, with individual messages available on -drill-down; in the [TUI](/usage/tui/), press `m` to switch to text mode. +drill-down; in the [TUI](/docs/usage/tui/), press `m` to switch to text mode. ## import-whatsapp @@ -233,7 +233,7 @@ msgvault sync-synctech-sms phone-backups ## Browsing Texts -Start `msgvault serve` and open the [Web UI](/web-ui/) to search email, chats, +Start `msgvault serve` and open the [Web UI](/docs/web-ui/) to search email, chats, calendar events, and meeting notes together. Chat results stay grouped as conversations so short message fragments do not overwhelm Everything. Open a conversation to inspect its matching messages in context. @@ -247,7 +247,7 @@ to Email. msgvault tui ``` -Text mode is only available when text data has been imported. See the [TUI documentation](/usage/tui/) for keyboard shortcuts and navigation. +Text mode is only available when text data has been imported. See the [TUI documentation](/docs/usage/tui/) for keyboard shortcuts and navigation. ## Deduplication diff --git a/docs/usage/tui.md b/docs/usage/tui.md index a30c1761f..bcabc9ab6 100644 --- a/docs/usage/tui.md +++ b/docs/usage/tui.md @@ -37,7 +37,7 @@ repairs it on demand. Set `auto_build_cache = false` to skip automatic startup maintenance; use `msgvault build-cache` for an explicit build. Deprecated in 0.17.0: the old `msgvault tui --force-sql`, `--no-cache-build`, and `--no-sqlite-scanner` flags are hidden because these choices are now daemon -configuration. See [Configuration: analytics](/configuration/#analytics). +configuration. See [Configuration: analytics](/docs/configuration/#analytics). ### Local And Remote @@ -53,17 +53,17 @@ msgvault tui --local Deletion staging and attachment export use the selected daemon. When connected to a configured remote server, staged deletion manifests are saved on that remote host; attachment export streams bytes from the daemon and writes the zip file on the CLI machine.
- msgvault TUI showing the Senders view with message counts and sizes + msgvault TUI showing the Senders view with message counts and sizes
Press `a` from any aggregate view to show all individual messages in that view. Press `Enter` on a message to view its full detail, including headers and body.
- msgvault TUI showing all messages list + msgvault TUI showing all messages list
All messages
- msgvault TUI showing a single message detail view + msgvault TUI showing a single message detail view
Message detail
@@ -82,7 +82,7 @@ The TUI provides seven aggregate view modes. Press `g` to cycle through them: | Labels | Aggregate by Gmail label | | Time | Aggregate by time period (year/month/day) |
- msgvault TUI Labels view showing Gmail label breakdown + msgvault TUI Labels view showing Gmail label breakdown
### Time View @@ -90,15 +90,15 @@ Press `t` from any view to jump directly to the Time view. The Time view aggrega
- Time view: monthly granularity + Time view: monthly granularity
Monthly
- Time view: daily granularity + Time view: daily granularity
Daily
- Time view: yearly granularity + Time view: yearly granularity
Yearly
@@ -107,9 +107,9 @@ Press `t` from any view to jump directly to the Time view. The Time view aggrega Press `m` to cycle through Email, Texts, and Meetings. Texts is skipped when no text/chat engine is available, but Meetings remains in the cycle even before a -meeting source has been configured. See [Text Messages](/usage/text-messages/) -for local chat imports, [Microsoft Teams](/usage/teams/) for Teams sync, and -[Discord](/usage/discord/) for guild channel and thread sync. +meeting source has been configured. See [Text Messages](/docs/usage/text-messages/) +for local chat imports, [Microsoft Teams](/docs/usage/teams/) for Teams sync, and +[Discord](/docs/usage/discord/) for guild channel and thread sync. Text mode provides the following view types. Press `g` to cycle through them: @@ -127,7 +127,7 @@ Navigation and interaction in Text mode work the same as Email mode. Press `Ente ## Meetings Meetings mode is a read-only browser for transcripts and notes imported from -[Granola or Circleback](/usage/meetings/). It shows a flat, newest-first list +[Granola or Circleback](/docs/usage/meetings/). It shows a flat, newest-first list with each meeting's date, title, organizer, and source. Press `Enter` to open the transcript and notes, `Esc` or `Backspace` to return to the list, and the left/right arrow keys to move between meeting details. @@ -150,17 +150,17 @@ read-only, and browsing a transcript never changes the source service. Press `Enter` to drill into any row. For example, selecting a sender shows their individual messages. Press `Esc` or `Backspace` to go back.
- msgvault TUI drill-down showing messages from a specific sender + msgvault TUI drill-down showing messages from a specific sender
From a drill-down view, press `g` to re-aggregate the filtered messages by a different dimension. You can think of this like an interactive pivot table. The cycle skips the dimension you drilled into — and when drilling from an email address view (Senders or Recipients), it also skips the corresponding name view since it would be redundant. For example, drilling into a sender and pressing `g` cycles through Recipients, Recipient Names, Domains, Labels, and Time.
- Sub-grouped by Recipients after drilling into a sender + Sub-grouped by Recipients after drilling into a sender
A sender's email grouped by recipient
- Sub-grouped by Time after drilling into a sender + Sub-grouped by Time after drilling into a sender
A sender's mail grouped by month
@@ -169,16 +169,16 @@ From a drill-down view, press `g` to re-aggregate the filtered messages by a dif Press `/` to open a search bar that filters the current view in real time. Matching text is highlighted in the results. At the aggregate level (Senders, Domains, etc.), search uses the daemon's configured analytics engine, so the default local-daemon setup uses DuckDB over Parquet when the cache is usable.
- msgvault TUI search filtering senders by name with highlighted matches + msgvault TUI search filtering senders by name with highlighted matches
Search also works after drill-down. Drill into a result, then press `/` again to search within that context. This second-level search uses deep FTS5 full-text search over message subjects and bodies. You can progressively narrow results: find a sender, drill in, then search for a specific subject or keyword.
- Drilled into search result showing messages from a specific sender + Drilled into search result showing messages from a specific sender
- Searching within a sender's messages by subject keyword with highlighted matches + Searching within a sender's messages by subject keyword with highlighted matches
@@ -191,7 +191,7 @@ Press `f` to open the filter modal. The modal presents two independent toggles t | Only with attachments | Show only messages that have attachments | | Hide deleted from source | Exclude messages that have been deleted from Gmail |
- msgvault TUI filter modal with checkbox toggles for attachments and hide deleted + msgvault TUI filter modal with checkbox toggles for attachments and hide deleted
Use `↑`/`↓` to navigate, `Space` or `x` to toggle a filter, and `Enter` or `Esc` to apply and close. Active filters are shown in the title bar (e.g. `[Attachments]`, `[Hide Deleted]`). Filters apply to all views: aggregates, drill-downs, sub-aggregates, search results, and stats. @@ -199,7 +199,7 @@ Use `↑`/`↓` to navigate, `Space` or `x` to toggle a filter, and `Enter` or ` From any message list (after drilling into a sender, label, domain, etc.), press `T` to open the full email thread for the highlighted message. This renders the complete conversation inline in the terminal, including sender, date, and body text for each message in the thread.
- msgvault TUI showing a full email thread conversation + msgvault TUI showing a full email thread conversation
Press `Esc` to return to the message list. @@ -230,16 +230,16 @@ Press `Esc` to return to the message list. You can stage individual messages or bulk-delete entire aggregate groups (e.g. all emails from a sender, all messages with a given label) at once. Use `Space` to select one or more rows, then press `d` to stage them. From any aggregate view, press `D` to stage every message in the current group without selecting individual rows.
- msgvault TUI with rows selected for deletion staging + msgvault TUI with rows selected for deletion staging
A confirmation dialog shows exactly how many messages will be staged before anything happens. Messages are not deleted immediately; they are placed in a deletion batch that you review and execute separately with `msgvault delete-staged`.
- msgvault TUI deletion confirmation dialog showing bulk staging + msgvault TUI deletion confirmation dialog showing bulk staging
-See [Deleting Email](/usage/deletion/) for the full deletion workflow. +See [Deleting Email](/docs/usage/deletion/) for the full deletion workflow. ## Performance The default SQLite archive path uses DuckDB querying Parquet metadata exports for aggregate views. This architecture delivers aggregate queries (top senders, domains, labels, time series) **hundreds of times faster** than equivalent SQLite JOINs. The Parquet analytics layer has a small footprint, so drill-down and re-aggregation feel instant even on very large archives. Configure `[analytics].engine` if you need to force live SQL or require DuckDB. -See [Data Storage](/architecture/storage/) for details on how this works. +See [Data Storage](/docs/architecture/storage/) for details on how this works. diff --git a/docs/usage/vector-search.md b/docs/usage/vector-search.md index 0ba6fca6a..b343d6271 100644 --- a/docs/usage/vector-search.md +++ b/docs/usage/vector-search.md @@ -174,7 +174,7 @@ query_prefix = "search_query: " pgvector embeddings live in the PostgreSQL database. `db_path` and `vectors.db` apply only to the SQLite sqlite-vec backend. See -[PostgreSQL Backend](/architecture/postgresql/) for database setup. +[PostgreSQL Backend](/docs/architecture/postgresql/) for database setup. ### Model task prefixes @@ -444,7 +444,7 @@ curl "http://localhost:8080/api/v1/search?q=planning+offsite&mode=vector&explain ``` Response shape differs from the FTS path; see the -[Web UI & API Server](/api-server/#get-apiv1search) reference for details. +[Web UI & API Server](/docs/api-server/#get-apiv1search) reference for details. HTTP vector/hybrid responses support only the first page; bump `page_size` (capped at `max_page_size_hybrid`) to retrieve a larger candidate page. @@ -579,6 +579,6 @@ are marked complete and not sent to the embedding endpoint. ## See Also -- [Web UI & API Server](/api-server/): browser interface and HTTP API reference. -- [Searching](/usage/searching/): Full-text search syntax. -- [Search Ranking Across Backends](/architecture/search-ranking/): Ranking differences between SQLite, PostgreSQL, sqlite-vec, and pgvector. +- [Web UI & API Server](/docs/api-server/): browser interface and HTTP API reference. +- [Searching](/docs/usage/searching/): Full-text search syntax. +- [Search Ranking Across Backends](/docs/architecture/search-ranking/): Ranking differences between SQLite, PostgreSQL, sqlite-vec, and pgvector. diff --git a/docs/vercel-build.sh b/docs/vercel-build.sh index 21a0b736d..58cdf2de1 100755 --- a/docs/vercel-build.sh +++ b/docs/vercel-build.sh @@ -1,5 +1,77 @@ #!/usr/bin/env bash set -euo pipefail script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +repo_root="$(cd "$script_dir/.." && pwd)" +site_dir="$script_dir/site" + "$script_dir/assets/hydrate-assets.sh" -"$script_dir/zensical-docs.sh" build + +rm -rf "$site_dir" +MSGVAULT_DOCS_SITE_DIR="site/docs" "$script_dir/zensical-docs.sh" build + +# The sitemap indexes every tier, so serve it from the standard root path +# crawlers probe, as the pre-tiered site did. +mv "$site_dir/docs/sitemap.xml" "$site_dir/sitemap.xml" + +# The marketing and guide tiers ship as static files from website/; the +# zensical docs tier lives under /docs/. +website_dir="$repo_root/website" +for entry in index.html index.md guide guide.md llms.txt favicon.svg assets fonts scripts styles; do + source_path="$website_dir/$entry" + if [[ ! -e "$source_path" ]]; then + printf 'missing website source: %s\n' "$entry" >&2 + exit 1 + fi + cp -R "$source_path" "$site_dir/$entry" +done + +# Local credential/secret artifacts that must never enter the published site. +# Keep in sync with credential_globs in docs/zensical-docs.sh and +# FORBIDDEN_SITE_FILENAMES in docs/scripts/check_built_site.py. +credential_globs=( + 'client_secret*.json' + 'oauth_client*.json' + 'credentials*.json' + 'service_account*.json' + 'service-account*.json' + 'token.json' + 'tokens.json' + 'token-*.json' + '*.pem' + '*.key' + '*.crt' + '*.cer' + '*.der' + '*.p12' + '*.pfx' + '*.p8' + '*.jks' + '*.keystore' + '*.ppk' + 'id_rsa*' + 'id_dsa*' + 'id_ecdsa*' + 'id_ed25519*' + '*.tfstate' + '*.tfstate.backup' + '*.tfvars' +) + +# The website copy is recursive over the working tree, so prune dotfiles and +# credential-pattern files at any depth before publishing. +prune_expr=('(' -name '.*') +for glob in "${credential_globs[@]}"; do + prune_expr+=(-o -iname "$glob") +done +prune_expr+=(')') +find "$site_dir" -depth "${prune_expr[@]}" -exec rm -rf {} + + +# Fail the build if anything slipped past the prune. This is the same +# inventory gate check-docs.sh runs; here it guards every deployment. +if command -v python3 >/dev/null 2>&1; then + python_bin="python3" +else + python_bin="python" +fi +"$python_bin" -c "import sys; sys.path.insert(0, '$script_dir/scripts'); \ +import check_built_site; check_built_site.check_public_site_file_inventory()" diff --git a/docs/vercel.json b/docs/vercel.json index 812149982..40436d5a7 100644 --- a/docs/vercel.json +++ b/docs/vercel.json @@ -15,6 +15,86 @@ "source": "/install.ps1", "destination": "https://raw.githubusercontent.com/kenn-io/msgvault/main/scripts/install.ps1", "permanent": false + }, + { + "source": "/introduction/:path*", + "destination": "/docs/introduction/:path*", + "permanent": true + }, + { + "source": "/setup/:path*", + "destination": "/docs/setup/:path*", + "permanent": true + }, + { + "source": "/web-ui/:path*", + "destination": "/docs/web-ui/:path*", + "permanent": true + }, + { + "source": "/configuration/:path*", + "destination": "/docs/configuration/:path*", + "permanent": true + }, + { + "source": "/cli-reference/:path*", + "destination": "/docs/cli-reference/:path*", + "permanent": true + }, + { + "source": "/api-server/:path*", + "destination": "/docs/api-server/:path*", + "permanent": true + }, + { + "source": "/changelog/:path*", + "destination": "/docs/changelog/:path*", + "permanent": true + }, + { + "source": "/troubleshooting/:path*", + "destination": "/docs/troubleshooting/:path*", + "permanent": true + }, + { + "source": "/development/:path*", + "destination": "/docs/development/:path*", + "permanent": true + }, + { + "source": "/faq/:path*", + "destination": "/docs/faq/:path*", + "permanent": true + }, + { + "source": "/usage/:path*", + "destination": "/docs/usage/:path*", + "permanent": true + }, + { + "source": "/guides/:path*", + "destination": "/docs/guides/:path*", + "permanent": true + }, + { + "source": "/architecture/:path*", + "destination": "/docs/architecture/:path*", + "permanent": true + }, + { + "source": "/assets/static/:path*", + "destination": "/docs/assets/static/:path*", + "permanent": true + }, + { + "source": "/assets/generated/:path*", + "destination": "/docs/assets/generated/:path*", + "permanent": true + }, + { + "source": "/search/:path*", + "destination": "/docs/search/:path*", + "permanent": true } ] } diff --git a/docs/web-ui.md b/docs/web-ui.md index c22f1f38b..2723f2d57 100644 --- a/docs/web-ui.md +++ b/docs/web-ui.md @@ -16,22 +16,22 @@ provenance-documented Enron-derived fixture imported through the real daemon; the ordinary browser checks continue to use small synthetic API fixtures.
- Experimental Relationships workspace in dark theme with ranked people and activity timeline + Experimental Relationships workspace in dark theme with ranked people and activity timeline
Relationships ranked view and selected activity timeline.
- Experimental Relationships workspace in light theme with compact density + Experimental Relationships workspace in light theme with compact density
Relationships workspace in light theme with compact density.
- Experimental analytical web UI in dark theme with comfortable density + Experimental analytical web UI in dark theme with comfortable density
Dark theme with comfortable density.
- Experimental analytical web UI in light theme with compact density + Experimental analytical web UI in light theme with compact density
Light theme with compact density.
@@ -148,7 +148,7 @@ and filters continue to scope both the timeline and file table. People in this workspace are observed identity clusters. Source identities that mean “me,” explicit durable profile promotion, display-name overrides, and typed profile attributes are separate curated operations; see [People, -Profiles, and Source Identities](/usage/people/). +Profiles, and Source Identities](/docs/usage/people/). Domains provides the same activity-and-files analysis for an exact domain fact. A domain is not treated as an inferred organization identity. Selecting diff --git a/docs/zensical.toml b/docs/zensical.toml index b3cb347a9..252df493c 100644 --- a/docs/zensical.toml +++ b/docs/zensical.toml @@ -1,6 +1,6 @@ [project] site_name = "msgvault" -site_url = "https://msgvault.io" +site_url = "https://msgvault.io/docs/" site_description = "Offline email, chat, calendar, and meeting archive with a daemon-served analytical Web UI, full-text and semantic search, and sync for Gmail, IMAP, Teams, Discord, Slack, Beeper, Granola, and Circleback." site_author = "Kenn Software LLC" copyright = 'Copyright © 2026 Kenn Software LLC. MIT License.' diff --git a/scripts/check-docs.sh b/scripts/check-docs.sh index c41e67962..9fb62de75 100755 --- a/scripts/check-docs.sh +++ b/scripts/check-docs.sh @@ -63,6 +63,7 @@ public_doc_globs=( --glob '!docs/assets/**' --glob '!docs/site/**' --glob '!docs/zensical-public-docs.*/**' + --glob '!docs/vercel-build.sh' ) root_media_refs="$( @@ -94,7 +95,7 @@ fi ( cd docs - uv run --frozen bash ./zensical-docs.sh build + uv run --frozen bash ./vercel-build.sh uv run --frozen python scripts/check_built_site.py uv run --frozen python scripts/selftest_check_built_site.py uv run --frozen python scripts/check_vercel_redirects.py diff --git a/website/assets/intelligence-pipeline.svg b/website/assets/intelligence-pipeline.svg new file mode 100644 index 000000000..7d32c87d3 --- /dev/null +++ b/website/assets/intelligence-pipeline.svg @@ -0,0 +1,63 @@ + + msgvault intelligence pipeline + The archive feeds a keyword index, vector embeddings, and Docbank document text; hybrid retrieval fuses them for the CLI, Web UI, and MCP agents. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ARCHIVE + messages + attachments + every source, one schema + + FTS5 keyword + vector embeddings + Docbank documents + + + Gmail-style operators + local embedding server + attachment text + OCR + + RETRIEVAL + hybrid fusion + BM25 + vectors · RRF + + CLI · Web · TUI + MCP agents + + + diff --git a/website/assets/interface-map.svg b/website/assets/interface-map.svg new file mode 100644 index 000000000..2114690fe --- /dev/null +++ b/website/assets/interface-map.svg @@ -0,0 +1,61 @@ + + msgvault interface map + CLI, Web UI, TUI, HTTP API, MCP server, and agent skills all read the same msgvault archive. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + MSGVAULT ARCHIVE + one archive + local first · daemon owned + + CLI + WEB + TUI + HTTP + MCP + SKILLS + + + scriptable workflows + analytical workspaces + keyboard drill-down + authenticated API + AI assistant access + Claude Code · Codex + + + diff --git a/website/assets/people-graph.svg b/website/assets/people-graph.svg new file mode 100644 index 000000000..37215f30d --- /dev/null +++ b/website/assets/people-graph.svg @@ -0,0 +1,54 @@ + + msgvault people layer + Addresses, handles, and phone numbers from different sources resolve to one durable person profile. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + alice@example.com + alice.w@example.org + @alice · slack + +1 555 0134 · sms + alice#4021 · discord + + + gmail + imap + workspace + phone backup + guild + + PERSON + Alice Warren + 5 identities · 4 sources + first contact 2009 + 12,408 messages + + diff --git a/website/assets/sources-map.svg b/website/assets/sources-map.svg new file mode 100644 index 000000000..94f7c9150 --- /dev/null +++ b/website/assets/sources-map.svg @@ -0,0 +1,63 @@ + + msgvault sources map + Mail, chat, meetings, calendar, contacts, and offline imports all land in one msgvault archive. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + MSGVAULT ARCHIVE + one record + SQLite · attachments · Parquet + + MAIL + CHAT + MEETINGS + CALENDAR + CONTACTS + IMPORTS + + + Gmail · IMAP + Slack · Teams · Discord + Beeper Desktop + Granola · Circleback + Google Calendar + CardDAV + MBOX · EMLX · PST · SMS + WhatsApp · Messenger + + + diff --git a/website/favicon.svg b/website/favicon.svg new file mode 100644 index 000000000..543f04cd1 --- /dev/null +++ b/website/favicon.svg @@ -0,0 +1,8 @@ + + + + + + + + diff --git a/website/fonts/Inter-Medium.woff2 b/website/fonts/Inter-Medium.woff2 new file mode 100644 index 000000000..fdfdcc699 Binary files /dev/null and b/website/fonts/Inter-Medium.woff2 differ diff --git a/website/fonts/Inter-Regular.woff2 b/website/fonts/Inter-Regular.woff2 new file mode 100644 index 000000000..2bcd222ec Binary files /dev/null and b/website/fonts/Inter-Regular.woff2 differ diff --git a/website/fonts/Inter-SemiBold.woff2 b/website/fonts/Inter-SemiBold.woff2 new file mode 100644 index 000000000..fbae113d2 Binary files /dev/null and b/website/fonts/Inter-SemiBold.woff2 differ diff --git a/website/fonts/JetBrainsMono-Bold.woff2 b/website/fonts/JetBrainsMono-Bold.woff2 new file mode 100644 index 000000000..4917f4341 Binary files /dev/null and b/website/fonts/JetBrainsMono-Bold.woff2 differ diff --git a/website/fonts/JetBrainsMono-Regular.woff2 b/website/fonts/JetBrainsMono-Regular.woff2 new file mode 100644 index 000000000..40da42765 Binary files /dev/null and b/website/fonts/JetBrainsMono-Regular.woff2 differ diff --git a/website/fonts/JetBrainsMono-SemiBold.woff2 b/website/fonts/JetBrainsMono-SemiBold.woff2 new file mode 100644 index 000000000..5ead7b0d6 Binary files /dev/null and b/website/fonts/JetBrainsMono-SemiBold.woff2 differ diff --git a/website/guide.md b/website/guide.md new file mode 100644 index 000000000..c8f9f29e0 --- /dev/null +++ b/website/guide.md @@ -0,0 +1,98 @@ +# The archive lifecycle + +One archive moves through nine stages. Your data stays local and complete at +every stop. + +1. [Capture](#capture) +2. [Preserve](#preserve) +3. [Resolve](#resolve) +4. [Curate](#curate) +5. [Understand](#understand) +6. [Search](#search) +7. [Analyze](#analyze) +8. [Act](#act) +9. [Own](#own) + +## Capture + +Live sources sync on a schedule — Gmail, IMAP, Slack, Teams, Discord, Beeper, +Google Calendar, CardDAV, meeting notes. Dead exports import once — MBOX, +Apple Mail, PST, WhatsApp, iMessage, Messenger, SMS backups. Interrupted syncs +resume from checkpoints. + +[Importing local email](/docs/usage/importing/) + +## Preserve + +Raw provider payloads are retained compressed beside the parsed record. +Attachments are content-addressed by SHA-256, deduplicated, and sealed into +immutable packs. Cross-account duplicates hide behind a reversible safety +ladder — the surviving copy is always the complete one. + +[Data storage](/docs/architecture/storage/) + +## Resolve + +Every source knows you and your contacts by different addresses and handles. +Identity discovery classifies the evidence; observed people cluster from +explicit archive links, never from matching display names. Nothing merges +without proof. + +[People, profiles, and identities](/docs/usage/people/) + +## Curate + +Promote the people who matter into durable profiles with stable IDs and vCard +UIDs. Attach typed attributes, organizations, employment history, and +relationships over a fact ledger with evidence and reversible merges. Watch +each relationship's activity calendar and temperature across every channel. + +[Curating people](/docs/usage/people/) + +## Understand + +Opt in to semantic search by pointing msgvault at an embedding server you +choose — local ones included. The embedded Docbank document engine extracts +and indexes attachment text and images behind explicit, fail-closed consent. +Every intelligence lane is disposable and rebuildable; the record is not. + +[Vector search](/docs/usage/vector-search/) + +## Search + +Full-text search with Gmail-style operators answers instantly and offline. +Semantic and hybrid modes fuse BM25 with vectors through reciprocal rank +fusion, with explainable ranking and honest coverage states; msgvault never quietly +substitutes one mode for another. + +[Searching](/docs/usage/searching/) + +## Analyze + +A DuckDB-over-Parquet analytics cache answers aggregate questions across +hundreds of thousands of messages in milliseconds: senders, domains, labels, +time. Drill down from a decade to a single message in the TUI or the browser. + +[Analytics and stats](/docs/usage/analytics/) + +## Act + +Staging and execution never share a surface. Any interface can stage a +deletion manifest for review; only the CLI executes it, behind an explicit +environment gate, defaulting to recoverable trash. The local archive is never +modified, and deleted mail remains searchable. + +[Deleting email](/docs/usage/deletion/) + +## Own + +Run it on a laptop or serve it from your own NAS: the daemon carries the Web +UI, HTTP API, scheduler, and MCP server in one binary. Verifiable backup +snapshots restore the archive with no provider in the loop. + +[Backup and restore](/docs/usage/backup/) + +## Next + +Move from the lifecycle model to [installation and setup](/docs/setup/) or +[all documentation](/docs/). diff --git a/website/guide/index.html b/website/guide/index.html new file mode 100644 index 000000000..8631e5cc7 --- /dev/null +++ b/website/guide/index.html @@ -0,0 +1,286 @@ + + + + + + The archive lifecycle — msgvault + + + + + + + + + + + + + + + + + +
+ + + msgvault + + +
+ +
+
+

Lifecycle guide

+

The archive lifecycle

+

One archive moves through nine stages. Your data stays local and complete at every stop.

+
    +
  1. 01 Capture
    2. +
  2. 02 Preserve
    3. +
  3. 03 Resolve
    4. +
  4. 04 Curate
    5. +
  5. 05 Understand
    6. +
  6. 06 Search
    7. +
  7. 07 Analyze
    8. +
  8. 08 Act
    9. +
  9. 09 Own
    10. +
+
+ +
    +
  1. +
    +

    Capture every channel

    +

    Live sources sync on a schedule — Gmail, IMAP, Slack, Teams, Discord, Beeper, Google Calendar, CardDAV, meeting notes. Dead exports import once — MBOX, Apple Mail, PST, WhatsApp, iMessage, Messenger, SMS backups. Interrupted syncs resume from checkpoints.

    + Importing local email +
    +
    +
    + Mail, chat, meetings, calendar, contacts, and offline imports converging into one msgvault archive +
    +
    +
    2. + +
  2. +
    +

    Preserve the original

    +

    Raw provider payloads are retained compressed beside the parsed record. Attachments are content-addressed by SHA-256, deduplicated, and sealed into immutable packs. Cross-account duplicates hide behind a reversible safety ladder — the surviving copy is always the complete one.

    + Data storage +
    +
    +
    + + msgvault deduplication concept showing duplicate copies collapsing to one surviving complete record + +
    Duplicates collapse to the most complete survivor, reversibly.
    +
    +
    +
    3. + +
  3. +
    +

    Resolve identities into people

    +

    Every source knows you and your contacts by different addresses and handles. Identity discovery classifies the evidence; observed people cluster from explicit archive links, never from matching display names. Nothing merges without proof.

    + People, profiles, and identities +
    +
    +
    + Email addresses, chat handles, and phone numbers from different sources resolving to one person profile +
    +
    +
    4. + +
  4. +
    +

    Curate durable profiles

    +

    Promote the people who matter into durable profiles with stable IDs and vCard UIDs. Attach typed attributes, organizations, employment history, and relationships over a fact ledger with evidence and reversible merges. Watch each relationship's activity calendar and temperature across every channel.

    + Curating people +
    +
    +
    + + msgvault Relationships workspace with ranked people, an activity heatmap, and a message timeline + +
    The Relationships workspace over the public Enron research corpus.
    +
    +
    +
    5. + +
  5. +
    +

    Understand with your models

    +

    Opt in to semantic search by pointing msgvault at an embedding server you choose — local ones included. The embedded Docbank document engine extracts and indexes attachment text and images behind explicit, fail-closed consent. Every intelligence lane is disposable and rebuildable; the record is not.

    + Vector search +
    +
    +
    + The archive feeding keyword, vector, and Docbank document indexes fused by hybrid retrieval +
    +
    +
    6. + + + +
  6. +
    +

    Analyze decades in milliseconds

    +

    A DuckDB-over-Parquet analytics cache answers aggregate questions across hundreds of thousands of messages in milliseconds: senders, domains, labels, time. Drill down from a decade to a single message in the TUI or the browser.

    + Analytics and stats +
    +
    +
    + + msgvault TUI showing sender aggregates with counts and sizes over the whole archive + +
    The TUI's sender aggregates, backed by DuckDB over Parquet.
    +
    +
    +
    7. + +
  7. +
    +

    Act with a safety ladder

    +

    Staging and execution never share a surface. Any interface can stage a deletion manifest for review; only the CLI executes it, behind an explicit environment gate, defaulting to recoverable trash. The local archive is never modified, and deleted mail remains searchable.

    + Deleting email +
    +
    +
    + + msgvault deletion safety ladder from staging and review to gated execution + +
    Staged deletions are inspectable and cancellable before execution.
    +
    +
    +
    8. + +
  8. +
    +

    Own the whole system

    +

    Run it on a laptop or serve it from your own NAS: the daemon carries the Web UI, HTTP API, scheduler, and MCP server in one binary. Verifiable backup snapshots restore the archive with no provider in the loop.

    + Backup and restore +
    +
    +
    + CLI, Web UI, TUI, HTTP API, MCP server, and agent skills sharing one msgvault archive +
    +
    +
    9. +
+ +
+
+
+

Next

+

Build your archive.

+
+
+

Move from the lifecycle model to installation, OAuth setup, exact command behavior, configuration, and architecture.

+
+ Run the setup guide + Open all documentation +
+
+
+
+
+ + + + +
+ + +
+ Expanded documentation capture + + + + diff --git a/website/index.html b/website/index.html new file mode 100644 index 000000000..e58d0aeda --- /dev/null +++ b/website/index.html @@ -0,0 +1,358 @@ + + + + + + msgvault — the system of record for your communications + + + + + + + + + + + + + + + + + +
+ + + msgvault + + +
+ +
+
+

+ + msgvault +

+

The system of record for your communications and relationships.

+

msgvault is a local-first, open-source archive for a lifetime of email, chat, meetings, calendars, and contacts. It keeps everything in one database on your own hardware, resolves the people behind decades of messages, and searches by keyword or by meaning.

+
+
+ macOS / Linux + curl -fsSL https://msgvault.io/install.sh | bash + +
+
+ Homebrew + brew install msgvault + +
+
+ Windows + irm https://msgvault.io/install.ps1 | iex + +
+
+

Installers fetch the latest release and verify its SHA-256 checksum. Also on conda-forge, or build from source.

+
+ See how it works + Read the docs +
+
+ +
+
+
+

01 / Record

+

Every channel. One archive.

+
+

Twenty years of correspondence should not be scattered across a dozen walled gardens. msgvault syncs live sources and imports dead exports into one schema, keeping raw payloads and content-addressed attachments intact.

+
+
+ Mail, chat, meetings, calendar, contacts, and offline imports converging into one msgvault archive of SQLite, attachments, and Parquet +
+
    +
  1. MailGmail, IMAP, and Microsoft 365 sync; MBOX, Apple Mail, PST, and EML imports.
    2. +
  2. ChatSlack, Teams, Discord, and every network behind Beeper; WhatsApp, iMessage, Messenger, and SMS imports.
    3. +
  3. MeetingsGranola and Circleback notes and transcripts in the same searchable record.
    4. +
  4. CalendarGoogle Calendar events, organizers, and attendees, read-only.
    5. +
  5. ContactsBidirectional CardDAV: pull address books, publish curated people back.
    6. +
+
+ +
+
+
+

02 / People

+

Messages come from addresses. Relationships come from people.

+
+

The people layer resolves decades of addresses, handles, and phone numbers into the people behind them — with archive evidence and user curation kept strictly apart.

+
+
+
+

Observed, not guessed

+

Cluster identities on evidence.

+

Observed people are assembled from explicit archive links across sources. Equal display names alone never merge two people.

+
+
+

Durable profiles

+

Promote the people who matter.

+

A promoted profile gets a stable ID and vCard UID, so names, notes, and typed attributes survive later identity changes. Merges are atomic and reversible.

+
+
+

Fact ledger

+

Curate facts with provenance.

+

Organizations, employment history, typed relationships, and custom attributes rest on immutable evidence, deterministic decisions, and per-person pins.

+
+
+

Activity

+

Watch each relationship over time.

+

An activity calendar tracks interaction with each person across email, chat, calendar, and meetings, year by year, including current and peak relationship temperature.

+
+
+
+ +
+
+
+

03 / Operation

+

Work the archive in the browser.

+
+

The daemon serves a dense, keyboard-driven browser application: relationships, a unified Everything table, files, saved views, source status, deletion staging, and settings. Every analytical slice is URL-addressable, so Back and Forward restore exact views.

+
+
+ + msgvault Relationships workspace showing ranked people, a relationship activity calendar, and a message timeline over public research data + +
The Relationships workspace over the public Enron research corpus. Select the image to inspect it at full size.
+
+
+

In development

+

The Directory workspace

+

An open pull request adds Directory and Reviews workspaces: durable-person search, profile maintenance and history, identity and merge review queues, the privacy-gated fact ledger, CardDAV publication, and a self-describing Settings surface with write-only credential management. Captures land here when it merges.

+
+
+ +
+
+
+

04 / Intelligence

+

Semantic search and document understanding.

+
+

Keyword search works offline, always. Semantic search, document extraction, and visual search are opt-in, with explicit consent recording exactly what leaves your machine and where it goes.

+
+
+ The msgvault archive feeding an FTS5 keyword index, vector embeddings, and Docbank document extraction, fused by hybrid retrieval for the CLI, Web UI, TUI, and MCP agents +
+
+
+

Hybrid search

+

Fuse keywords and meaning.

+

FTS5 with Gmail-style operators, pure semantic search, or hybrid BM25-plus-vector fusion via reciprocal rank fusion, with an explain mode that shows why each result ranked.

+
+
+

Local models

+

Point at any embedding server.

+

Any OpenAI-compatible endpoint works: Ollama, llama.cpp, LM Studio, or Apple's on-device model. Embedding scope is a privacy boundary; out-of-scope accounts are never sent anywhere.

+
+
+

Attachments

+

Read the attachments too.

+

The embedded Docbank document engine handles OCR extraction, normalized chunks, lexical and semantic document search, and visual search over images. Consent-gated and fail-closed.

+
+
+

Agents

+

Give your AI the whole archive.

+

An MCP server exposes search, people, files, and analytics tools to Claude Desktop and other agents; bundled agent skills install into Claude Code and Codex. Profile writes stay behind explicit flags.

+
+
+
+ +
+
+
+

05 / Interfaces

+

One archive across every surface.

+

The daemon owns all writes and serializes every mutation. People, scripts, and agents work through the interface suited to the task, against the same record.

+
    +
  • CLIScriptable sync, search, and repair.
    • +
  • WebAnalytical workspaces in the browser.
    • +
  • TUIKeyboard drill-down analytics.
    • +
  • HTTPAn authenticated, versioned API.
    • +
  • MCPArchive tools for AI assistants.
    • +
  • SkillsWorkflows for Claude Code and Codex.
    • +
+
+ Connect an agent + Inspect the API +
+
+
+ CLI, Web UI, TUI, HTTP API, MCP server, and agent skills sharing one msgvault archive +
+
+
+ +
+
+
+

06 / Ownership

+

Archive everything. Then delete upstream.

+
+

Once the archive is complete and verified, you can start deleting from the provider. Every step is explicit and reviewed, and nothing is irreversible until the last one.

+
+
+
+

Verify

+

Prove the copy is complete.

+

Integrity verification checks the archive against the mailbox before you trust it with anything irreversible.

+
+
+

Stage

+

Review before anything moves.

+

Deletions are staged into manifests from the Web UI, TUI, or MCP — inspected, counted, and cancellable. No surface executes them.

+
+
+

Execute

+

Delete upstream, keep the record.

+

Execution is a separate CLI step behind an explicit environment gate, defaulting to recoverable trash. The local archive is never modified.

+
+
+

Restore

+

Back up the vault itself.

+

Append-only, verifiable backup snapshots cover the database and attachments, with restore paths that need no provider at all.

+
+
+
+ +
+
+
+

07 / Boundary

+

Not a mail client. Not a takeout file.

+
+

msgvault is a data warehouse for your communications: a system of record you operate, query, and extend. Not a viewport, and not cold storage.

+
+
+
+

Mail client

+

The provider is the record.

+

A client renders whatever the server still holds. Identity, search, and history live and die with the account.

+
+
+

Export archive

+

The zip is a snapshot.

+

A takeout captures one moment in one format. It does not sync, resolve people, answer questions, or talk to agents.

+
+
+

msgvault

+

The archive is the record.

+

Providers become replaceable feeds around a database you own — continuously synced, people-resolved, searchable by meaning, and open to your tools.

+
+
+
+ +
+
+
+

08 / Start

+

Follow one archive through the system.

+
+
+

The guide walks the archive lifecycle from capture to ownership. The documentation carries setup, exact command behavior, configuration, and architecture.

+
+ Read the lifecycle guide + Open the documentation +
+
+
+
+
+ +
+

Copyright 2026 Kenn Software LLC. Open source under MIT. Alpha software — back up your data.

+ +
+ + +
+ + +
+ Expanded documentation capture + + + + diff --git a/website/index.md b/website/index.md new file mode 100644 index 000000000..330033896 --- /dev/null +++ b/website/index.md @@ -0,0 +1,164 @@ +# msgvault + +**The system of record for your communications and relationships.** + +msgvault is a local-first, open-source archive for a lifetime of email, chat, +meetings, calendars, and contacts. It keeps everything in one database on your +own hardware, resolves the people behind decades of messages, and searches by +keyword or by meaning. + +msgvault is usable through the CLI, browser application, terminal interface, +HTTP API, MCP server, and bundled agent skills. It is alpha software — back up +your data. + +## Install + +On macOS or Linux: + +```sh +curl -fsSL https://msgvault.io/install.sh | bash +``` + +Or with Homebrew: + +```sh +brew install msgvault +``` + +On Windows (PowerShell): + +```powershell +irm https://msgvault.io/install.ps1 | iex +``` + +The installers fetch the latest GitHub release and verify its SHA-256 +checksum. msgvault is also on +[conda-forge](https://prefix.dev/channels/conda-forge/packages/msgvault), and +the [setup documentation](/docs/setup/) covers building from source. + +Then [follow the archive lifecycle](/guide/). + +## Every channel. One archive. + +Twenty years of correspondence should not be scattered across a dozen walled +gardens. msgvault syncs live sources and imports dead exports into one schema, +keeping raw payloads and content-addressed attachments intact. + +- **Mail** — Gmail, IMAP, and Microsoft 365 sync; MBOX, Apple Mail, PST, and + EML imports. +- **Chat** — Slack, Teams, Discord, and every network behind Beeper; WhatsApp, + iMessage, Messenger, and SMS imports. +- **Meetings** — Granola and Circleback notes and transcripts in the same + searchable record. +- **Calendar** — Google Calendar events, organizers, and attendees, read-only. +- **Contacts** — bidirectional CardDAV: pull address books, publish curated + people back. + +## Messages come from addresses. Relationships come from people. + +The people layer resolves decades of addresses, handles, and phone numbers into +the people behind them — with archive evidence and user curation kept strictly +apart. + +### Observed, not guessed + +Observed people are assembled from explicit archive links across sources. Equal +display names alone never merge two people. + +### Durable profiles + +A promoted profile gets a stable ID and vCard UID, so names, notes, and typed +attributes survive later identity changes. Merges are atomic and reversible. + +### Fact ledger + +Organizations, employment history, typed relationships, and custom attributes +rest on immutable evidence, deterministic decisions, and per-person pins. + +### Activity + +An activity calendar tracks interaction with each person across email, chat, +calendar, and meetings, year by year, including current and peak relationship +temperature. + +## Work the archive in the browser + +The daemon serves a dense, keyboard-driven browser application: relationships, +a unified Everything table, files, saved views, source status, deletion +staging, and settings. Every analytical slice is URL-addressable, so Back and +Forward restore exact views. + +**In development:** an open pull request adds Directory and Reviews workspaces — +durable-person search, profile maintenance and history, identity and merge +review queues, the privacy-gated fact ledger, CardDAV publication, and a +self-describing Settings surface with write-only credential management. + +## Semantic search and document understanding + +Keyword search works offline, always. Semantic search, document extraction, +and visual search are opt-in, with explicit consent recording exactly what +leaves your machine and where it goes. + +- **Hybrid search:** FTS5 with Gmail-style operators, pure semantic search, + or hybrid BM25-plus-vector fusion via reciprocal rank fusion, with an + explain mode that shows why each result ranked. +- **Local models:** any OpenAI-compatible endpoint works: Ollama, llama.cpp, + LM Studio, or Apple's on-device model. Embedding scope is a privacy + boundary; out-of-scope accounts are never sent anywhere. +- **Attachments:** the embedded + [Docbank](https://github.com/kenn-io/docbank) document engine handles OCR + extraction, normalized chunks, lexical and semantic document search, and + visual search over images. Consent-gated and fail-closed. +- **Agents:** an MCP server exposes search, people, files, and analytics + tools to Claude Desktop and other agents; bundled agent skills install into + Claude Code and Codex. Profile writes stay behind explicit flags. + +## One archive across every surface + +The daemon owns all writes and serializes every mutation. People, scripts, and +agents work through the interface suited to the task, against the same record. + +- **CLI:** scriptable sync, search, and repair. +- **Web:** analytical workspaces in the browser. +- **TUI:** keyboard drill-down analytics. +- **HTTP:** an authenticated, versioned API. +- **MCP:** archive tools for AI assistants. +- **Skills:** workflows for Claude Code and Codex. + +[Connect an agent](/docs/usage/chat/) or [inspect the API](/docs/api-server/). + +## Archive everything. Then delete upstream. + +Once the archive is complete and verified, you can start deleting from the +provider. Every step is explicit and reviewed, and nothing is irreversible +until the last one. + +- **Verify:** integrity verification checks the archive against the mailbox + before you trust it with anything irreversible. +- **Stage:** deletions are staged into manifests from the Web UI, TUI, or + MCP — inspected, counted, and cancellable. No surface executes them. +- **Execute:** execution is a separate CLI step behind an explicit environment + gate, defaulting to recoverable trash. The local archive is never modified. +- **Restore:** append-only, verifiable backup snapshots cover the database and + attachments, with restore paths that need no provider at all. + +## Not a mail client. Not a takeout file. + +msgvault is a data warehouse for your communications: a system of record you +operate, query, and extend. Not a viewport, and not cold storage. + +- **Mail client:** the provider is the record. A client renders whatever the + server still holds; identity, search, and history live and die with the + account. +- **Export archive:** the zip is a snapshot. A takeout captures one moment in + one format; it does not sync, resolve people, answer questions, or talk to + agents. +- **msgvault:** the archive is the record. Providers become replaceable feeds + around a database you own — continuously synced, people-resolved, searchable + by meaning, and open to your tools. + +## Follow one archive through the system + +The [lifecycle guide](/guide/) walks the archive from capture to ownership. +The [documentation](/docs/) carries setup, exact command behavior, +configuration, and architecture. diff --git a/website/llms.txt b/website/llms.txt new file mode 100644 index 000000000..8ecd87952 --- /dev/null +++ b/website/llms.txt @@ -0,0 +1,38 @@ +# msgvault + +> msgvault is a local-first, open-source archive for a lifetime of email, +> chat, meetings, calendars, and contacts — one database with a durable people +> layer, keyword and semantic search, an MCP server, and agent skills, owned +> and operated by you. + +The product and guide pages have exact Markdown twins; documentation pages are +served as HTML. Markdown versions are canonical for machine readers. + +## Product + +- [Product overview](https://msgvault.io/index.md): what msgvault is, every + supported source, the people layer, search modes, interfaces, and the + deletion safety model +- [Archive lifecycle guide](https://msgvault.io/guide.md): capture, preserve, + resolve, curate, understand, search, analyze, act, own + +## Docs + +- [Documentation index](https://msgvault.io/docs/): all operating documentation +- [Setup guide](https://msgvault.io/docs/setup/): installation and first sync +- [CLI reference](https://msgvault.io/docs/cli-reference/): every command and flag +- [Configuration](https://msgvault.io/docs/configuration/): config.toml reference +- [Web UI](https://msgvault.io/docs/web-ui/): the browser application +- [Searching](https://msgvault.io/docs/usage/searching/): query syntax and modes +- [Vector search](https://msgvault.io/docs/usage/vector-search/): semantic and hybrid setup +- [Document attachment indexing](https://msgvault.io/docs/usage/document-indexing/): Docbank-backed extraction +- [People, profiles, and identities](https://msgvault.io/docs/usage/people/): the people layer +- [MCP server](https://msgvault.io/docs/usage/chat/): AI assistant integration +- [Deleting email](https://msgvault.io/docs/usage/deletion/): staging and gated execution +- [Backup](https://msgvault.io/docs/usage/backup/): snapshot repositories +- [Architecture overview](https://msgvault.io/docs/architecture/overview/): design decisions + +## Other + +- [GitHub repository](https://github.com/kenn-io/msgvault): source, issues, releases +- [Changelog](https://msgvault.io/docs/changelog/): release history diff --git a/website/scripts/site.js b/website/scripts/site.js new file mode 100644 index 000000000..95425ba19 --- /dev/null +++ b/website/scripts/site.js @@ -0,0 +1,130 @@ +const repoApi = "https://api.github.com/repos/kenn-io/msgvault"; +const cacheMaxAgeMs = 60 * 60 * 1000; + +function installLightboxes() { + const dialog = document.querySelector("[data-lightbox-dialog]"); + if (!(dialog instanceof HTMLDialogElement)) return; + + const image = dialog.querySelector("img"); + const title = dialog.querySelector("[data-lightbox-title]"); + const close = dialog.querySelector("[data-lightbox-close]"); + let trigger = null; + + for (const link of document.querySelectorAll("a[data-lightbox]")) { + link.addEventListener("click", (event) => { + if (!(image instanceof HTMLImageElement)) return; + event.preventDefault(); + trigger = link; + const source = link.getAttribute("href"); + const preview = link.querySelector("img"); + if (!source || !(preview instanceof HTMLImageElement)) return; + image.src = source; + image.alt = preview.alt; + if (title) title.textContent = preview.alt; + dialog.showModal(); + if (close instanceof HTMLElement) close.focus(); + }); + } + + close?.addEventListener("click", () => dialog.close()); + dialog.addEventListener("click", (event) => { + if (event.target === dialog) dialog.close(); + }); + dialog.addEventListener("close", () => { + if (image instanceof HTMLImageElement) image.removeAttribute("src"); + if (trigger instanceof HTMLElement) trigger.focus(); + }); +} + +function installCopyButtons() { + const status = document.querySelector("[data-install-status]"); + if (!(status instanceof HTMLElement)) return; + + let resetTimer; + for (const root of document.querySelectorAll("[data-install-command]")) { + const button = root.querySelector("[data-install-copy]"); + const command = root instanceof HTMLElement ? root.dataset.command : undefined; + if (!(button instanceof HTMLButtonElement) || !command) continue; + + button.addEventListener("click", async () => { + clearTimeout(resetTimer); + try { + await navigator.clipboard.writeText(command); + status.textContent = "Copied"; + resetTimer = setTimeout(() => { + status.textContent = ""; + }, 2000); + } catch { + status.textContent = "Copy failed — select the command text instead"; + } + }); + } +} + +function readCache(key, now) { + try { + const raw = localStorage.getItem(key); + if (!raw) return null; + const entry = JSON.parse(raw); + if (now - entry.at > cacheMaxAgeMs) return null; + return entry.value; + } catch { + return null; + } +} + +function writeCache(key, value, now) { + try { + localStorage.setItem(key, JSON.stringify({ at: now, value })); + } catch { + // Storage can be unavailable (private browsing); facts refetch next visit. + } +} + +async function cachedJson(key, url) { + const now = Date.now(); + const cached = readCache(key, now); + if (cached !== null) return cached; + const response = await fetch(url, { headers: { Accept: "application/vnd.github+json" } }); + if (!response.ok) return null; + const value = await response.json(); + writeCache(key, value, now); + return value; +} + +function setFact(name, text) { + const fact = document.querySelector(`[data-fact="${name}"]`); + if (!(fact instanceof HTMLElement)) return; + const label = fact.querySelector("[data-fact-text]"); + if (!label) return; + label.textContent = text; + fact.hidden = false; + const row = document.querySelector("[data-facts]"); + if (row instanceof HTMLElement) row.hidden = false; +} + +function formatCount(count) { + if (count < 1000) return String(count); + const thousands = count / 1000; + const rounded = thousands >= 10 ? Math.round(thousands) : Math.round(thousands * 10) / 10; + return `${rounded}k`; +} + +async function installRepoFacts() { + if (!document.querySelector("[data-facts]")) return; + const [repo, release] = await Promise.all([ + cachedJson("msgvault:repo", repoApi), + cachedJson("msgvault:release", `${repoApi}/releases/latest`), + ]); + if (repo) { + setFact("stars", formatCount(repo.stargazers_count)); + setFact("forks", formatCount(repo.forks_count)); + } + if (release) setFact("version", release.tag_name); +} + +installLightboxes(); +installCopyButtons(); +installRepoFacts().catch((error) => { + console.warn("github api unavailable, keeping the static header", error); +}); diff --git a/website/styles/site.css b/website/styles/site.css new file mode 100644 index 000000000..00ccf1a76 --- /dev/null +++ b/website/styles/site.css @@ -0,0 +1,841 @@ +@font-face { + font-family: "Inter"; + src: url("/fonts/Inter-Regular.woff2") format("woff2"); + font-style: normal; + font-weight: 400; + font-display: swap; +} + +@font-face { + font-family: "Inter"; + src: url("/fonts/Inter-Medium.woff2") format("woff2"); + font-style: normal; + font-weight: 500; + font-display: swap; +} + +@font-face { + font-family: "Inter"; + src: url("/fonts/Inter-SemiBold.woff2") format("woff2"); + font-style: normal; + font-weight: 600; + font-display: swap; +} + +@font-face { + font-family: "JetBrains Mono"; + src: url("/fonts/JetBrainsMono-Regular.woff2") format("woff2"); + font-style: normal; + font-weight: 400; + font-display: swap; +} + +@font-face { + font-family: "JetBrains Mono"; + src: url("/fonts/JetBrainsMono-SemiBold.woff2") format("woff2"); + font-style: normal; + font-weight: 600; + font-display: swap; +} + +@font-face { + font-family: "JetBrains Mono"; + src: url("/fonts/JetBrainsMono-Bold.woff2") format("woff2"); + font-style: normal; + font-weight: 700; + font-display: swap; +} + +:root { + color-scheme: dark; + --ink: #0a0a0a; + --panel: #131313; + --panel-raised: #1b1b1b; + --rule: #282828; + --carbon: #e8e8e8; + --carbon-strong: #ffffff; + --text: #f2f2f2; + --muted: #a3a3a3; + --body: "Inter", system-ui, sans-serif; + --mono: "JetBrains Mono", ui-monospace, monospace; + background: var(--ink); + color: var(--text); + font-family: var(--body); + font-synthesis: none; + line-height: 1.6; +} + +* { + box-sizing: border-box; +} + +html { + scroll-behavior: smooth; +} + +body { + margin: 0; + background: var(--ink); +} + +body > header, +body > main, +body > footer { + width: min(100% - 2.5rem, 76rem); + margin-inline: auto; +} + +a { + color: var(--carbon); + text-decoration-thickness: 1px; + text-underline-offset: 0.2em; +} + +a:hover { + color: var(--carbon-strong); +} + +:focus-visible { + outline: 2px solid var(--carbon); + outline-offset: 4px; +} + +.skip-link { + position: fixed; + z-index: 20; + top: 0.75rem; + left: 0.75rem; + padding: 0.55rem 0.75rem; + color: var(--ink); + background: var(--carbon); + transform: translateY(-200%); +} + +.skip-link:focus { + transform: none; +} + +.site-header { + display: flex; + align-items: center; + justify-content: space-between; + min-height: 4.75rem; + gap: 2rem; + border-bottom: 1px solid var(--rule); +} + +.site-header nav, +.footer-links, +.actions { + display: flex; + flex-wrap: wrap; + align-items: center; + gap: 0.75rem 1.25rem; +} + +.site-header nav a { + color: var(--muted); + text-decoration: none; +} + +.site-header nav a:hover, +.site-header nav a[aria-current="page"] { + color: var(--text); +} + +.sr-only { + position: absolute; + overflow: hidden; + width: 1px; + height: 1px; + margin: -1px; + padding: 0; + white-space: nowrap; + clip-path: inset(50%); + border: 0; +} + +.repo { + display: inline-flex; + align-items: center; + gap: 0.5rem; + padding: 0.2rem 0.3rem; + border-radius: 3px; + font-family: var(--mono); +} + +.site-header nav a.repo:hover { + background: var(--panel); +} + +.repo > svg { + flex: 0 0 auto; +} + +.repo-meta { + display: flex; + flex-direction: column; + gap: 0.1rem; + line-height: 1.2; +} + +.repo-name { + font-size: 0.72rem; +} + +.facts { + display: flex; + gap: 0.5rem; + font-size: 0.64rem; +} + +.facts[hidden], +.fact[hidden] { + display: none; +} + +.fact { + display: inline-flex; + align-items: center; + gap: 0.22rem; +} + +.icon-link { + display: inline-flex; +} + +.wordmark, +.eyebrow, +.section-index, +.ledger strong, +.proof-tag, +.choice-label, +code, +kbd { + font-family: var(--mono); +} + +.wordmark { + display: inline-flex; + align-items: center; + gap: 0.55rem; + color: var(--text); + font-family: var(--mono); + font-size: 1.05rem; + font-weight: 700; + letter-spacing: -0.03em; + text-decoration: none; +} + +.wordmark img { + flex: 0 0 auto; +} + +main { + display: block; +} + +.hero { + position: relative; + padding: clamp(5rem, 12vw, 9rem) 0 clamp(4rem, 9vw, 7rem); + border-bottom: 1px solid var(--rule); +} + +.hero::before { + content: ""; + position: absolute; + inset: 0; + background: radial-gradient(42rem 26rem at 18% 8%, rgb(232 232 232 / 6%), transparent 70%); + pointer-events: none; +} + +.hero > * { + position: relative; +} + +.hero.compact { + padding-bottom: clamp(3rem, 7vw, 5rem); +} + +.hero-brand { + display: flex; + align-items: center; + gap: clamp(0.8rem, 1.6vw, 1.2rem); + margin: 0 0 2.25rem; + font-family: var(--mono); + font-size: clamp(2.4rem, 6vw, 4.25rem); + font-weight: 700; + letter-spacing: -0.04em; + line-height: 1; +} + +.hero-brand img { + width: clamp(2.6rem, 6vw, 4.25rem); + height: auto; + flex: 0 0 auto; +} + +.hero.branded h1 { + max-width: 30ch; + margin-bottom: 1.5rem; + font-size: clamp(1.7rem, 3.6vw, 2.8rem); + font-weight: 500; + letter-spacing: -0.03em; + color: var(--muted); +} + +.eyebrow, +.section-index, +.proof-tag, +.choice-label { + margin: 0 0 1rem; + color: var(--muted); + font-size: 0.76rem; + font-weight: 600; + letter-spacing: 0.08em; + text-transform: uppercase; +} + +h1, +h2, +h3, +p { + margin-top: 0; +} + +h1, +h2, +h3 { + line-height: 1.08; + text-wrap: balance; +} + +h1 { + max-width: 19ch; + margin-bottom: 1.5rem; + font-size: clamp(2.75rem, 7.5vw, 5.75rem); + font-weight: 500; + letter-spacing: -0.055em; +} + +h2 { + max-width: 24ch; + margin-bottom: 1rem; + font-size: clamp(1.8rem, 4vw, 3rem); + font-weight: 500; + letter-spacing: -0.035em; +} + +h3 { + margin-bottom: 0.65rem; + font-size: 1.08rem; + font-weight: 600; +} + +.lede { + max-width: 56rem; + margin-bottom: 2rem; + color: var(--muted); + font-size: clamp(1.08rem, 2vw, 1.35rem); +} + +.caption, +.fine-print, +.install-hint { + color: var(--muted); + font-size: 0.88rem; +} + +.install-matrix { + width: fit-content; + max-width: 100%; + margin-top: 0.5rem; + border: 1px solid var(--rule); + background: var(--panel); +} + +.install-row { + display: grid; + grid-template-columns: 8.5rem minmax(0, 1fr) max-content; + align-items: center; + gap: 1.25rem; + padding: 0.45rem 0.45rem 0.45rem 1.1rem; +} + +.install-row + .install-row { + border-top: 1px solid var(--rule); +} + +.install-label { + color: var(--muted); + font-family: var(--mono); + font-size: 0.72rem; + font-weight: 600; + letter-spacing: 0.08em; + text-transform: uppercase; +} + +.install-row code { + color: var(--carbon); + font-size: 0.92rem; + overflow-wrap: anywhere; +} + +.install-row code::before { + color: var(--muted); + content: "$ "; +} + +.install-row code.ps::before { + content: "> "; +} + +.install-hint { + margin-top: 0.75rem; +} + +.install-hint [data-install-status]:not(:empty)::before { + content: " · "; +} + +.hero-meta { + margin: 3rem 0 0; + color: var(--muted); + font-family: var(--mono); + font-size: 0.76rem; + letter-spacing: 0.04em; +} + +.actions { + margin-top: 2rem; +} + +.button, +button.button { + min-height: 2.8rem; + border: 1px solid var(--rule); + border-radius: 3px; + font: inherit; +} + +.button, +button.button { + display: inline-flex; + align-items: center; + justify-content: center; + padding: 0.65rem 1rem; + color: var(--text); + background: var(--panel); + cursor: pointer; + text-decoration: none; +} + +.button:hover, +button.button:hover { + border-color: var(--carbon); + color: var(--text); + background: var(--panel-raised); +} + +.button.primary { + border-color: var(--carbon); + color: var(--ink); + background: var(--carbon); + font-weight: 600; +} + +.button.primary:hover { + color: var(--ink); + background: var(--carbon-strong); +} + +.site-section { + padding: clamp(4rem, 9vw, 7rem) 0; + border-bottom: 1px solid var(--rule); +} + +.section-heading { + display: grid; + grid-template-columns: minmax(0, 0.65fr) minmax(0, 1.35fr); + gap: 2rem; + margin-bottom: clamp(2rem, 5vw, 4rem); +} + +.section-heading p:last-child { + max-width: 44rem; + color: var(--muted); +} + +.ledger { + display: grid; + grid-template-columns: repeat(5, minmax(0, 1fr)); + gap: 1px; + margin: 0; + padding: 1px; + background: var(--rule); + list-style: none; +} + +.ledger li { + min-height: 10rem; + padding: 1.25rem; + background: var(--panel); +} + +.ledger strong, +.ledger span { + display: block; +} + +.ledger strong { + margin-bottom: 2rem; + color: var(--carbon); + font-size: 0.76rem; + text-transform: uppercase; +} + +.ledger span { + color: var(--muted); +} + +.proof-grid, +.choice-grid { + display: grid; + grid-template-columns: repeat(2, minmax(0, 1fr)); + gap: 1px; + padding: 1px; + background: var(--rule); +} + +.boundary-grid { + display: grid; + grid-template-columns: repeat(3, minmax(0, 1fr)); + gap: 1px; + padding: 1px; + background: var(--rule); +} + +.pipeline { + margin: 0 0 clamp(2rem, 5vw, 3rem); + border: 1px solid var(--rule); + background: var(--panel); +} + +.proof, +.choice, +.boundary { + padding: clamp(1.5rem, 4vw, 2.5rem); + background: var(--panel); +} + +.proof p:last-child, +.choice p:last-child, +.boundary p:last-child { + margin-bottom: 0; + color: var(--muted); +} + +.capture { + margin: 0; +} + +.capture > a { + display: block; + border: 1px solid var(--rule); + background: var(--panel); +} + +.capture > a:hover { + border-color: var(--carbon); +} + +.capture img, +.diagram img, +dialog img { + display: block; + width: 100%; + height: auto; +} + +.caption { + margin: 0.8rem 0 0; +} + +.placeholder-panel { + display: grid; + gap: 0.5rem; + padding: clamp(1.5rem, 4vw, 2.5rem); + border: 1px dashed var(--rule); + background: var(--panel); +} + +.placeholder-panel p { + max-width: 44rem; + margin: 0; + color: var(--muted); +} + +.placeholder-panel .proof-tag { + margin-bottom: 0.25rem; +} + +.interface-layout { + display: grid; + grid-template-columns: minmax(0, 1.2fr) minmax(18rem, 0.8fr); + gap: clamp(2rem, 6vw, 5rem); + align-items: center; +} + +.interface-list { + display: grid; + grid-template-columns: repeat(2, minmax(0, 1fr)); + gap: 1rem 2rem; + margin: 2rem 0 0; + padding: 0; + list-style: none; +} + +.interface-list strong, +.interface-list span { + display: block; +} + +.interface-list strong { + font-family: var(--mono); + font-size: 0.86rem; +} + +.interface-list span { + color: var(--muted); +} + +.guide-nav { + display: grid; + grid-template-columns: repeat(9, minmax(0, 1fr)); + gap: 1px; + margin: 2.5rem 0 0; + padding: 1px; + background: var(--rule); + list-style: none; +} + +.guide-nav a { + display: block; + height: 100%; + padding: 0.8rem; + color: var(--muted); + background: var(--panel); + font-family: var(--mono); + font-size: 0.75rem; + text-decoration: none; +} + +.guide-nav a:hover { + color: var(--text); + background: var(--panel-raised); +} + +.guide-stops { + margin: 0; + padding: 0; + list-style: none; + counter-reset: guide; +} + +.guide-stop { + display: grid; + grid-template-columns: minmax(0, 0.7fr) minmax(0, 1.3fr); + gap: clamp(2rem, 7vw, 6rem); + padding: clamp(4rem, 9vw, 7rem) 0; + border-bottom: 1px solid var(--rule); + scroll-margin-top: 2rem; + counter-increment: guide; +} + +.guide-copy h2::before { + display: block; + margin-bottom: 1rem; + color: var(--muted); + font-family: var(--mono); + font-size: 0.76rem; + letter-spacing: 0.08em; + content: "0" counter(guide) " / 09"; +} + +.guide-copy p { + color: var(--muted); +} + +.guide-copy a { + font-weight: 600; +} + +.guide-media { + align-self: center; +} + +.guide-media .capture, +.guide-media .diagram { + margin: 0; +} + +.guide-media .diagram { + border: 1px solid var(--rule); + background: var(--panel); +} + +.site-footer { + display: flex; + align-items: flex-start; + justify-content: space-between; + gap: 2rem; + padding: 2rem 0 3rem; + color: var(--muted); + font-size: 0.88rem; +} + +.site-footer p { + margin: 0; +} + +dialog { + width: min(96vw, 90rem); + max-height: 94vh; + padding: 0; + border: 1px solid var(--carbon); + border-radius: 4px; + color: var(--text); + background: var(--ink); +} + +dialog::backdrop { + background: rgb(5 5 5 / 88%); +} + +.dialog-header { + display: flex; + justify-content: space-between; + align-items: center; + gap: 1rem; + padding: 0.8rem 1rem; + border-bottom: 1px solid var(--rule); +} + +.dialog-header p { + margin: 0; + font-family: var(--mono); + font-size: 0.8rem; +} + +.dialog-close { + min-width: 2.5rem; + min-height: 2.5rem; + border: 1px solid var(--rule); + border-radius: 3px; + color: var(--text); + background: var(--panel); + font: inherit; + cursor: pointer; +} + +dialog img { + max-height: calc(94vh - 4.25rem); + object-fit: contain; +} + +@media (max-width: 900px) { + .section-heading, + .interface-layout, + .guide-stop { + grid-template-columns: 1fr; + } + + .ledger { + grid-template-columns: repeat(2, minmax(0, 1fr)); + } + + .ledger li:last-child { + grid-column: 1 / -1; + } + + .guide-nav { + grid-template-columns: repeat(3, minmax(0, 1fr)); + } + + .boundary-grid { + grid-template-columns: 1fr; + } + + .guide-media { + order: -1; + } +} + +@media (max-width: 620px) { + body > header, + body > main, + body > footer { + width: min(100% - 1.5rem, 76rem); + } + + .site-header { + align-items: flex-start; + flex-direction: column; + gap: 0.7rem; + padding: 1rem 0; + } + + .site-header nav { + width: 100%; + justify-content: space-between; + } + + .repo-meta { + display: none; + } + + .install-row { + grid-template-columns: minmax(0, 1fr) max-content; + gap: 0.4rem 1rem; + } + + .install-label { + grid-column: 1 / -1; + } + + h1 { + font-size: clamp(2.5rem, 13vw, 4rem); + } + + .ledger, + .proof-grid, + .choice-grid, + .boundary-grid, + .interface-list, + .guide-nav { + grid-template-columns: 1fr; + } + + .ledger li:last-child { + grid-column: auto; + } + + .site-footer { + flex-direction: column; + } +} + +@media (prefers-reduced-motion: reduce) { + html { + scroll-behavior: auto; + } + + *, + *::before, + *::after { + scroll-behavior: auto !important; + transition-duration: 0.01ms !important; + animation-duration: 0.01ms !important; + animation-iteration-count: 1 !important; + } +}