Centralize application registration

ARCHITECTURE.md

@@ -30,10 +30,24 @@ use its own authentication and generic reverse-proxy contract.
 `app.app_factory.create_app()` is the single production construction path for
 the Flask application. Importing `app.web` defines the core HTTP surface and
 its accessor facade, but does not construct an application. Factory creation
-has a fixed order: configure Flask, establish the durable signing key, attach
-runtime state, synchronize authentication state, register core routes,
-register core blueprints, load module contributions and templates, install
-base-path handling, then apply reverse-proxy handling as the outer WSGI layer.
+first resolves one immutable registration plan containing core routes, core
+blueprints, built-in modules, and enabled community modules. Blueprint
+callbacks are preflighted on an isolated application, and URL/config ownership
+collisions are validated before the target application or process-wide module
+catalogs are changed. The validated plan then applies its HTTP, catalog, and
+template contributions once, followed by base-path handling and reverse-proxy
+handling as the outer WSGI layer.
+
+`app.registration` is the only productive Flask registrar. It owns blueprint
+and direct-rule application, endpoint/blueprint/route-method collision checks,
+and the canonical registration manifest. The manifest contains stable public
+identifiers only; its SHA-256 fingerprint is logged at startup and stored in
+the application extensions. Filesystem locations, callables, and configuration
+or secret names and values are excluded. `app.module_registry` owns manifest
+discovery, `app.module_config_registry` owns configuration-schema preflight,
+`app.module_contributions` owns contribution-specific resolution and preflight,
+and `app.module_loader` orchestrates ownership and rejection policy behind its
+compatible public facade.
 
 Each application owns a typed `DocsightRuntime` at
 `app.extensions["docsight"]`. It contains the configuration manager, storage,
@@ -515,7 +529,20 @@ Installed non-theme modules are discovered by `ModuleLoader` and persisted throu
 
 Module manifests can declare module-owned config defaults through the top-level `config` object. Normal module config remains plain local configuration. A community manifest may opt specific declared string-default keys into write-only secret handling with `config_secrets`; the value must be a list of unique strings, every listed key must exist in that manifest's `config`, and each corresponding default must itself be a string so encrypted values never enter scalar coercion paths. The pure-stdlib validator in `app/manifest_contract.py` is the authoritative manifest capability contract used by the runtime loader and can also validate external catalogs without importing Flask.
 
-Before any enabled module loads, `ModuleLoader` reserves secret ownership from every discovered community manifest, including disabled modules. Core secret, hash-backed, private, and core configuration keys cannot be claimed. A secret key must also be exclusive to its declaring module's `config`; claims that overlap another module's plain or secret config fail every affected module and grant no owner. The installer performs the same ownership evaluation against installed modules and rejects a conflicting package before it can be persisted. Valid module secrets are encrypted by `ConfigManager`, masked in Settings responses, and exposed through the community config proxy only to their owning module; other module secrets and all core protected values are removed.
+Before any enabled module loads, `ModuleLoader` resolves secret ownership from every discovered community manifest, including disabled modules, as fail-closed safety metadata. Core secret, hash-backed, private, and core configuration keys cannot be claimed. A secret key must also be exclusive to its declaring module's `config`; claims that overlap another module's plain or secret config fail every affected module and grant no owner. Normal community configuration keys also require one unambiguous owner. No ownership or classification is applied until the complete registration plan validates. The installer performs the same ownership evaluation against installed modules and rejects a conflicting package before it can be persisted. Valid module secrets are encrypted by `ConfigManager`, masked in Settings responses, and exposed through the community config proxy only to their owning module; other module secrets and all core protected values are removed.
+
+Module source order is stable: the explicit built-in directory registry, the
+built-in threshold and theme registries, then configured community search paths
+and lexicographically sorted directories within each path. Duplicate community
+IDs retain the first source for manifest-v1 compatibility and skip later
+sources deterministically. Disabled and rejected modules remain visible as
+metadata but apply no routes, static mounts, templates, config defaults, i18n,
+active themes, thresholds, collectors, or publishers. As a metadata-only
+exception, a disabled community theme's declared `theme.json` is resolved and
+validated through the same safe contribution path so Appearance can render its
+preview. Invalid or unsafe preview data rejects that module, and preview data
+never creates a registration-plan contribution. A built-in planning failure
+aborts application construction; a community failure rejects only that module.
 
 Settings receives the active module-secret and saved-secret key sets from the server, so installed templates from before the explicit field-marker contract still preserve masked values safely during a coordinated catalog rollout. Current templates also represent saved secrets locally with empty password inputs plus explicit `data-config-secret` and `data-saved-secret` metadata. Untouched saved fields submit the standard mask, which `ConfigManager` treats as preserve-existing, while edited fields submit the replacement value. Saved plaintext is never rendered into HTML.
 

CONTRIBUTING.md

@@ -105,6 +105,10 @@ Open `http://localhost:8765` to access the setup wizard.
 ```
 app/
   app_factory.py     - Deterministic Flask application construction
+  registration.py    - Registration planning, collision checks, and application
+  module_registry.py - Stable module manifest discovery and metadata
+  module_config_registry.py - Module configuration ownership preflight
+  module_contributions.py - Module filesystem/import/JSON contribution preflight
   main.py            - Entrypoint, ThreadPoolExecutor polling loop
   runtime.py         - Typed per-application runtime state and locks
   web.py             - Core routes, filters, auth, and runtime accessors
@@ -163,6 +167,25 @@ We prefer new languages to be contributed by people who actually use the tool in
 
 DOCSight supports community modules that extend functionality without modifying core code. Modules can add API endpoints, data collectors, settings panels, dashboard tabs, and more.
 
+Route contributions export a Flask `Blueprint`; they must not call
+`app.register_blueprint()`, `app.add_url_rule()`, or `Blueprint.register()`
+themselves. Routes and decorators recorded on the exported blueprint are the
+module contribution consumed by the registrar.
+Deferred Blueprint record callbacks run once on an isolated application during
+preflight and again during real registration. Community callbacks must
+therefore be idempotent and free of filesystem, network, process, or other
+external side effects.
+The shared application registration contract preflights the complete core and
+module plan before applying it. Endpoint names, blueprint names, route/method
+pairs, module IDs, static mounts, and configuration ownership must therefore be
+globally unique. Add registration-contract tests when changing this surface,
+including a failure case that proves the target application and process-wide
+catalogs remain unchanged.
+Every explicitly declared contribution must resolve and validate during that
+preflight. An unresolved route, static directory, template, catalog, class, or
+JSON contribution rejects the module's complete plan; no partial contribution
+is registered.
+
 Server-side module code should import the established accessors it needs from
 `app.web`, such as `get_config_manager()`, `get_storage()`, `get_state()`, or
 `get_module_loader()`. These accessors resolve the active application's typed

app/app_factory.py

@@ -9,13 +9,20 @@
 from typing import Any
 
 from flask import Flask
-from jinja2 import ChoiceLoader, FileSystemLoader
 from werkzeug.middleware.proxy_fix import ProxyFix
 
 from . import web
 from .base_path import configure_base_path
-from .blueprints import register_blueprints
 from .module_loader import ModuleLoader
+from .registration import (
+    RegistrationPlan,
+    build_core_plan,
+    canonical_manifest,
+    existing_rules,
+    manifest_fingerprint,
+    register_plan,
+    validate_plan,
+)
 from .runtime import (
     AuthStateStore,
     DocsightRuntime,
@@ -54,19 +61,6 @@ def build(app: Flask):
     return build
 
 
-def install_module_template_loader(app: Flask, module_loader) -> None:
-    """Add enabled module template directories to this app's Jinja loader."""
-    if module_loader is None:
-        return
-    loaders = [app.jinja_loader]
-    for module in module_loader.get_enabled_modules():
-        template_dir = os.path.join(module.path, "templates")
-        if os.path.isdir(template_dir):
-            loaders.append(FileSystemLoader(template_dir))
-    if len(loaders) > 1:
-        app.jinja_loader = ChoiceLoader(loaders)
-
-
 def apply_reverse_proxy(app: Flask, environ: Mapping[str, str]) -> None:
     """Install trusted reverse-proxy handling as the outer WSGI layer."""
     reverse_proxy = environ.get("REVERSE_PROXY", "").strip()
@@ -98,15 +92,30 @@ def create_app(
 ) -> Flask:
     """Create one fully isolated DOCSight application in a fixed order.
 
-    Construction configures Flask, attaches runtime state, initializes auth,
-    registers core and blueprint routes, loads modules and templates, then
-    installs base-path and reverse-proxy middleware.
+    Construction resolves and validates the complete core/module plan before
+    configuring the target app or applying process catalogs. It then attaches
+    runtime state, applies the complete registration once, and wraps the
+    base-path and reverse-proxy middleware.
     """
     if config_manager is None:
         raise TypeError("config_manager is required")
     env = os.environ if environ is None else environ
 
+    core_plan = build_core_plan()
+    planning_app = Flask("app.registration_planning", static_folder=None)
+    planning_app.config["TESTING"] = testing
+    planning_app.extensions["docsight_registration_deferred"] = True
+    planning_app.extensions["docsight_registration_base_plan"] = core_plan
+    loader = module_loader_factory(planning_app) if module_loader_factory else None
+    module_plan = getattr(loader, "registration_plan", RegistrationPlan())
+    complete_plan = core_plan.combined(module_plan)
+
     app = Flask("app.web", template_folder="templates")
+    validate_plan(
+        complete_plan,
+        existing=existing_rules(app),
+        existing_blueprints=tuple(app.blueprints),
+    )
     app.config.update(
         SESSION_COOKIE_HTTPONLY=True,
         SESSION_COOKIE_SAMESITE="Lax",
@@ -137,12 +146,15 @@ def create_app(
     )
     attach_runtime(app, runtime)
     web.bootstrap_auth_state(app, runtime)
-    web.register_core_routes(app)
-    register_blueprints(app)
-
-    loader = module_loader_factory(app) if module_loader_factory else None
+    register_plan(app, complete_plan)
+    web.install_core_template_hooks(app)
     runtime.module_loader = loader
-    install_module_template_loader(app, loader)
     configure_base_path(app, env)
     apply_reverse_proxy(app, env)
+    manifest = canonical_manifest(app, loader)
+    fingerprint = manifest_fingerprint(manifest)
+    app.extensions["docsight_module_loader"] = loader
+    app.extensions["docsight_registration_manifest"] = manifest
+    app.extensions["docsight_registration_fingerprint"] = fingerprint
+    LOG.info("registration manifest sha256=%s", fingerprint)
     return app

app/blueprints/__init__.py

@@ -1,7 +1,7 @@
-"""Flask Blueprint registration."""
+"""Stable providers for DOCSight's core Flask blueprints."""
 
 
-def register_blueprints(app):
+def core_blueprints():
     from .config_bp import config_bp
     from .polling_bp import polling_bp
     from .data_bp import data_bp
@@ -13,13 +13,15 @@ def register_blueprints(app):
     from .segment_bp import segment_bp
     from .smart_capture_bp import smart_capture_bp
 
-    app.register_blueprint(config_bp)
-    app.register_blueprint(polling_bp)
-    app.register_blueprint(data_bp)
-    app.register_blueprint(analysis_bp)
-    app.register_blueprint(events_bp)
-    app.register_blueprint(modules_bp)
-    app.register_blueprint(metrics_bp)
-    app.register_blueprint(notices_bp)
-    app.register_blueprint(segment_bp)
-    app.register_blueprint(smart_capture_bp)
+    return (
+        config_bp,
+        polling_bp,
+        data_bp,
+        analysis_bp,
+        events_bp,
+        modules_bp,
+        metrics_bp,
+        notices_bp,
+        segment_bp,
+        smart_capture_bp,
+    )

app/module_config_registry.py

@@ -0,0 +1,107 @@
+"""Preflight and application of module-owned configuration schema."""
+
+from __future__ import annotations
+
+from collections import defaultdict
+from typing import Any
+
+from . import config as _cfg
+from .module_registry import ModuleInfo
+
+
+def register_module_config(
+    config_defaults: dict[str, Any],
+    module_id: str | None = None,
+    builtin: bool = False,
+    config_secrets: list[str] | None = None,
+    config_private: list[str] | None = None,
+) -> set[str]:
+    """Apply prevalidated defaults and classification to the process schema."""
+    secret_keys = set(config_secrets or [])
+    private_keys = set(config_private or []) if builtin else set()
+    registered: set[str] = set()
+    for key, value in config_defaults.items():
+        if key in secret_keys and not builtin and _cfg.MODULE_SECRET_OWNERS.get(key) != module_id:
+            continue
+        if key in _cfg.DEFAULTS:
+            if key in private_keys:
+                _cfg.PRIVATE_KEYS.add(key)
+            continue
+        if not builtin and key in (_cfg.SECRET_KEYS | _cfg.PRIVATE_KEYS | _cfg.HASH_KEYS):
+            continue
+        _cfg.DEFAULTS[key] = value
+        registered.add(key)
+        if key in secret_keys and builtin:
+            _cfg.SECRET_KEYS.add(key)
+        if key in private_keys:
+            _cfg.PRIVATE_KEYS.add(key)
+        if _cfg.is_secret_key(key):
+            continue
+        if isinstance(value, bool):
+            _cfg.BOOL_KEYS.add(key)
+        elif isinstance(value, int):
+            _cfg.INT_KEYS.add(key)
+    return registered
+
+
+def evaluate_module_secret_ownership(
+    modules: list[ModuleInfo],
+) -> tuple[set[str], dict[str, str], dict[str, str]]:
+    """Return reserved keys, valid owners, and redacted ownership errors."""
+    protected = set(_cfg.CORE_CONFIG_KEYS) | _cfg.SECRET_KEYS | _cfg.HASH_KEYS | _cfg.PRIVATE_KEYS
+    for module in modules:
+        if module.builtin:
+            protected.update(module.config)
+    secret_claims: dict[str, list[ModuleInfo]] = defaultdict(list)
+    config_claims: dict[str, list[ModuleInfo]] = defaultdict(list)
+    for module in modules:
+        if module.builtin:
+            continue
+        for key in module.config:
+            config_claims[key].append(module)
+        for key in module.config_secrets:
+            secret_claims[key].append(module)
+    reserved = {key for key in secret_claims if key not in protected}
+    owners: dict[str, str] = {}
+    errors: dict[str, str] = {}
+    for key, claimants in secret_claims.items():
+        if key in protected:
+            for module in claimants:
+                errors[module.id] = "Module secret declaration conflicts with protected configuration"
+            continue
+        users = config_claims.get(key, [])
+        if len(claimants) != 1 or len(users) != 1 or users[0].id != claimants[0].id:
+            for module in [*claimants, *users]:
+                errors[module.id] = "Module secret ownership conflict"
+            continue
+        owners[key] = claimants[0].id
+    return reserved, owners, errors
+
+
+def reserve_module_secrets(modules: list[ModuleInfo]) -> None:
+    """Compatibility adapter for applying preflighted secret ownership."""
+    reserved, owners, errors = evaluate_module_secret_ownership(modules)
+    for module in modules:
+        if module.id in errors:
+            module.error = errors[module.id]
+    _cfg.set_module_secret_registry(reserved, owners)
+
+
+def evaluate_module_config_ownership(modules: list[ModuleInfo]) -> dict[str, str]:
+    """Return redacted errors for ambiguous community config ownership."""
+    protected = set(_cfg.CORE_CONFIG_KEYS)
+    protected.update(key for module in modules if module.builtin for key in module.config)
+    claims: dict[str, list[ModuleInfo]] = defaultdict(list)
+    errors: dict[str, str] = {}
+    for module in modules:
+        if module.builtin or not module.enabled:
+            continue
+        for key in module.config:
+            claims[key].append(module)
+            if key in protected:
+                errors[module.id] = "Module config ownership conflicts with protected configuration"
+    for claimants in claims.values():
+        if len(claimants) > 1:
+            for module in claimants:
+                errors[module.id] = "Module config ownership conflict"
+    return errors