DATA_CONTRACT.md

DOCSight data contract

DOCSight is a self-hosted evidence tool. This contract defines which data belongs to the local installation, which files are part of the shipped system, which artifacts are generated by the user, and what must be redacted before sharing diagnostics or exports.

Core rule

DOCSight does not upload modem data, logs, credentials, tokens, reports, or installation identifiers automatically.

Local monitoring, storage, report generation, backups, and dismissal state remain on the user's own machine unless the user deliberately configures an integration or exports a file. Optional integrations may send the specific payloads they are configured to send to the destinations chosen by the user.

User-owned local data

User-owned local data is created by running DOCSight against a real setup or by entering information in the app. Updates must preserve it unless the user explicitly deletes, migrates, restores, or replaces it.

Typical locations and contents:

Data	Typical location	Notes
Configuration	${DATA_DIR:-/data}/config.json	Contains selected modem type, polling settings, enabled modules, integration settings, and local preferences.
Secrets	config.json plus local key files such as .config_key and .session_key	Secret values are encrypted or hashed where supported. Do not share config files unless secrets are removed.
DOCSIS and signal history	/data/docsis_history.db	SQLite database with snapshots, event history, incident journal data, Speedtest/BQM/Smokeping-derived records, imports, and report-related state.
Connection Monitor samples	/data/connection_monitor.db	SQLite database with latency probes, outage evidence, traceroute-related state, and raw or aggregated ping evidence.
Backups	User-configured backup path or generated backup archives	Backups can contain databases, configuration, encrypted secrets, session keys, and metadata. Treat them as private.
Incident journal	Stored in DOCSight data databases	User-entered descriptions, notes, timestamps, reviewed events, imported evidence, and incident groupings.
Attachments	User-selected files or imported evidence where enabled	May include screenshots, CSV/PDF imports, BQM images, or other support evidence. Redact before sharing.
API tokens	Token metadata and hashes in local storage	Plaintext tokens are shown once at creation and are not stored as reusable plaintext.

System-owned files

System-owned files are part of the shipped DOCSight application. They can be replaced by an update, image pull, package reinstall, or source checkout.

Examples:

• app/ application code, templates, static assets, collectors, modules, and translations.
• tests/ and .github/ validation and CI definitions.
• Root documentation such as README.md, SECURITY.md, ARCHITECTURE.md, and this DATA_CONTRACT.md.
• Built-in demo fixtures, built-in module manifests, bundled maintainer notices, and static product assets.
• Container image layers outside the mounted data directory.

Built-in system files differ from user-installed extension files. Community modules and themes installed into MODULES_DIR, usually /data/modules in the Docker image, are local installation files chosen by the user. Their code and assets are not overwritten by DOCSight core updates unless the user installs, updates, removes, or replaces them.

System updates must not rely on overwriting user-owned local data. Data migrations should preserve existing local records, create backup or rollback paths where risk is meaningful, and keep raw evidence intact unless the user chooses a destructive action.

Generated artifacts

Generated artifacts are files DOCSight creates from local data at the user's request. They are user-owned once generated and can contain private information.

Examples:

• PDF reports and complaint text.
• AI/LLM export markdown.
• CSV, JSON, Markdown, and raw ping log exports.
• Backup archives and restore validation output.
• Downloaded BNetzA, Speedtest, BQM, Smokeping, event, or journal exports.
• Screenshots or support snippets created by the user.

Generated artifacts should be reviewed before sharing. They may include provider names, timestamps, public or private IP addresses, MAC addresses, serial numbers, modem model and firmware details, incident notes, customer details, ticket numbers, webhook destination hints, or other account-specific information.

Configuration and secrets

DOCSight stores local configuration under the configured data directory. Secret-bearing settings include router/modem credentials, MQTT credentials, webhook and Apprise settings, PWA Web Push VAPID private key material, admin password material, and API token hashes.

Report customer defaults are private configuration metadata encrypted at rest. Unlike password-style secrets, they remain displayable and editable in normal local Settings and report forms, and are hidden in demo mode.

Rules:

• Secret values must not be returned unmasked in normal settings responses.
• Password placeholders must preserve saved secrets without replacing them with mask text.
• Hash-only values such as admin passwords and API tokens must not be converted back to plaintext.
• Logs and diagnostics must avoid printing raw credentials, raw tokens, cookie values, bearer tokens, webhook URLs with embedded credentials, private key material, or provider account identifiers.
• Support output should use stable labels, counts, booleans, and redacted prefixes instead of raw secret values.

SQLite and history databases

The main DOCSight history is local SQLite data. It is evidence, not cache.

• Raw modem counters, channel snapshots, incident records, and imported measurements should remain faithful to the source data.
• Saved DOCSIS snapshots may include a bounded, local copy of the raw driver payload used to produce the analyzed snapshot. This is evidence for replaying future rule/analyzer changes, not telemetry sent to DOCSight maintainers.
• Raw driver payload persistence applies simple privacy safeguards: obvious secret-bearing keys are redacted before storage and oversized/non-JSON payloads are skipped rather than forced into the history database.
• Signal events generated from a saved poll can carry local snapshot row references in their structured details so event evidence can be traced back to the source snapshot inside the same installation database.
• The main history database tracks DOCSight schema-version metadata in _docsight_meta so local upgrades can apply explicit, repeatable schema changes.
• Retention cleanup should not delete snapshots or events that fall inside explicit incident windows; incident-linked evidence is treated as user-retained support context.
• Derived views may compute health windows, trends, ranges, summaries, and report sections, but they should not rewrite raw modem evidence just to fit a display model.
• Unsupported or unknown fields should stay distinguishable from measured zero values.
• Database migrations should preserve user-owned local data and document any rare destructive migration path before it runs.

Backups and restore archives

Backups exist to protect a local DOCSight installation. A backup archive can include configuration, encrypted secrets, session keys, databases, imported evidence, and metadata. It should be handled like a private copy of the instance.

Rules:

• Backup downloads are private artifacts.
• Restore validation should not require uploading the archive to a remote service.
• Backup metadata may describe format version, app version, timestamp, table counts, or included files, but should avoid exposing secrets.
• Any support request involving a backup should ask for a redacted description or validation result first, not the raw archive.

Incident journal, attachments, reports, and complaint text

Incident journal entries, attachments, generated reports, and complaint-ready text are user-owned evidence.

They may contain:

• names, addresses, phone numbers, customer IDs, ticket IDs, or contract references
• precise outage times and home-usage patterns
• provider names and support interactions
• screenshots, PDFs, CSV files, BQM images, or imported measurements
• local device names, modem models, serial numbers, MAC addresses, IP addresses, and firmware versions

DOCSight may help organize and export this evidence, but the user chooses what to send to an ISP, regulator, community thread, or maintainer.

AI/LLM export and redaction controls

The AI/LLM export is a local generated artifact for user-reviewed sharing. Its redaction controls should be treated as a safety boundary, not as a reason to share raw private data blindly.

Before sharing an AI/LLM export, review and redact:

• public and private IP addresses
• MAC addresses, serial numbers, hostnames, Wi-Fi names, and modem identifiers
• names, addresses, phone numbers, customer numbers, ticket numbers, and contract references
• precise location hints and provider account details
• webhook URLs, API tokens, passwords, cookies, VAPID private keys, Apprise credentials, MQTT credentials, and session material
• raw logs that may include request headers, upstream response bodies, or local filesystem paths

Diagnostic and support exports

Diagnostic output should be minimal, local, and redacted by default. Prefer structured facts over raw dumps.

The local doctor command is the preferred first diagnostic entrypoint for self-hosted runtime checks:

python -m app.doctor
python -m app.doctor --json

In Docker, run it inside the DOCSight container, for example docker exec docsight python -m app.doctor --json. The default command is passive: it reads local files, environment, and databases but does not actively probe modems, DNS, MQTT brokers, webhooks, Apprise, or other configured services.

Safe support output can include:

• DOCSight version
• runtime mode such as Docker, native Python, or Demo mode
• enabled feature names and module IDs
• configured collector type without credentials
• last poll time, status class, and exception type where useful
• database presence, table counts, migration state, and write-permission checks
• integration enabled/disabled state and sanitized destination type

Diagnostic/support output must redact or omit:

• passwords, token values, cookie values, API keys, private keys, and session secrets
• full webhook or Apprise target URLs when credentials or tokens are embedded
• local/private IP addresses, public IP addresses, MAC addresses, serial numbers, hostnames, SSIDs, and exact filesystem paths when not needed
• provider account IDs, customer IDs, ticket IDs, names, addresses, and phone numbers
• raw HAR captures, full modem HTML/API responses, full logs, or backup archives unless a maintainer explicitly asks for a narrow, redacted sample

Optional integration boundaries

Optional integrations are user-configured boundaries. They do not change the core local ownership model, but they may send selected payloads to configured destinations.

• Speedtest: pulls or records speed, ping, jitter, and related test metadata from a configured Speedtest Tracker source. Treat endpoint URLs, credentials, and enriched response details as private.
• BQM: imports ThinkBroadband BQM CSV or image data from user-provided sources. Treat share URLs, image URLs, timing patterns, and imported files as private unless deliberately published.
• Smokeping: imports latency and packet-loss evidence from a configured source. Treat source URLs and local correlation output as private.
• Home Assistant and MQTT: publishes selected status and discovery payloads to the user's broker. MQTT credentials are secret; payloads may reveal current line health and device state to that broker.
• Direct webhooks and Discord: send alert payloads to configured endpoints. Webhook URLs and headers are secrets; payloads should stay concise and redacted.
• Apprise: sends alert payloads through a user-configured Apprise API sidecar. Apprise URL, key, token, tags, and provider-specific targets are private.
• PWA Web Push: sends minimal alert payloads to browser push endpoints for browsers or apps that subscribed to this DOCSight instance. Browser push providers may see delivery metadata according to the browser or vendor push service. Subscriptions and private keys are local secrets.
• BNetzA imports: imported measurement PDFs and CSV files can contain measurement details and report context. Treat source files and generated report text as private.
• Module and theme registries: Settings can fetch configured module and theme registry JSON and install selected packages from user-approved download URLs. Registry URLs, installed IDs, and downloaded community module or theme files are part of the local installation boundary.

A disabled or unconfigured optional integration must not be required for DOCSight's local monitoring and evidence workflow to keep working.

Module-owned config

Community and built-in modules may define module-owned config. Module-owned config belongs to the local installation and is treated as user-owned local data when saved.

Rules:

• Community modules must not claim reserved core secret keys such as modem or admin credentials.
• Community collectors receive a restricted config proxy that hides core secret and hash-backed settings.
• Module manifests, default settings, and system module code are system-owned files. Saved module configuration is user-owned local data.

Demo mode and real monitored data

Demo mode uses synthetic fixtures and generated demo history so users can evaluate DOCSight without connecting real hardware. Demo data and public proof-pack screenshots are safe only because they are synthetic.

Real monitored data comes from the user's modem, router, configured integrations, imported files, and notes. It can reveal line quality, provider context, outage windows, local network identifiers, account details, and personal support history. Treat real monitored data as private unless the user intentionally exports and reviews it.

Switching from Demo mode to live mode should preserve user-created journal entries and incidents where the product promises that behavior, while allowing demo-seeded monitoring rows to be removed cleanly.

Optional remote behavior

Optional remote behavior must be explicit opt-in, disabled by default, and must preserve local operation when disabled.

Current optional remote behavior:

• Release update checks are controlled by update_check_enabled / UPDATE_CHECK_ENABLED and are disabled by default. When enabled, DOCSight checks the public GitHub Releases API for the latest DOCSight release tag. It does not upload modem data, logs, credentials, tokens, reports, analytics, or an installation ID.
• Module and theme registries are only fetched from the Settings extension/theme management surfaces using the configured registry URLs. Installed registry URLs, selected IDs, and downloaded files stay within the local installation boundary.

All optional remote behavior must follow these constraints:

• It must clearly explain what is fetched or sent before enablement.
• It must not upload private payloads, identifiers, analytics, modem data, logs, credentials, tokens, reports, or installation IDs as a side effect of checking for updates, notices, registries, or other optional remote sources.
• It must not render remote HTML, untrusted Markdown, scripts, or tracking pixels inside the app.
• It must fail safely when offline or unavailable so local monitoring, local history, and local exports continue to work.
• It must have tests for opt-in behavior, disabled-by-default behavior, and redaction of any shareable payload.

Sharing checklist

Before sharing any DOCSight output publicly or with support, check for:

• names, addresses, phone numbers, customer numbers, contract numbers, and ticket IDs
• public IPs, private IPs, MAC addresses, serial numbers, hostnames, SSIDs, and precise location hints
• raw credentials, tokens, cookies, API keys, private keys, webhook URLs, Apprise URLs with credentials, MQTT credentials, and session files
• provider account pages, modem HTML with hidden fields, HAR captures, raw logs, stack traces, or backup archives
• real ISP names in public marketing screenshots unless the page is specifically about that provider and the use is intentional

When in doubt, share a redacted excerpt, a synthetic demo asset, or a short description first.