Skip to content

Decouple inference request state from the HTTP administration shell #46

Description

@tiammomo

Summary

The application state is owned by the HTTP routes module and exposes all 17 runtime and administration dependencies to inference handlers and provider adapters, creating a reverse dependency from provider code into transport composition.

Actual behavior

src/routes.rs defines AppState; src/server.rs imports it from routes; OpenAI-compatible, Anthropic, and token-counting provider modules accept routes::AppState. The inference path therefore compiles against administration-only dependencies and the application composition root is owned by its HTTP adapter.

Expected behavior

An application-owned top-level state composes the process dependencies, while Axum FromRef exposes a typed inference substate containing only dependencies required by client inference and provider execution. Provider adapters no longer import the routes module.

Steps to reproduce

Inspect src/routes.rs:207, src/server.rs:25, and imports of routes::AppState under src/providers on the verified #45 branch.

Environment

ModelPort main at 14ea1f5; implementation depends on the domain-router composition in #44/#45; Axum 0.8.9.

Acceptance criteria

  • Move application-state ownership out of src/routes.rs into an application-layer module; model an inference substate and derive or implement Axum FromRef from the top-level state; migrate client inference and provider adapter entry points to the inference substate; ensure provider modules do not import routes; keep route composition, public APIs, authentication, routing, quotas, rate limits, streaming, metrics, persistence, configuration, migrations, and dependencies behavior-compatible; add compile-time or architectural regression coverage for the dependency boundary; document the state ownership rule and the remaining administration-state follow-up; pass scripts/check-all.sh.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions