Repository navigation
feat(doctor): add --deep runtime diagnostics and middleware slot matrix - #1046
Conversation
…MCP auth) Root cause of the visibility gap: conditional middleware slots (KG, HITL, model_fallback) fail silently at assembly time — the factory returns None plus one log line and the slot is simply absent. The host doctor could not see any of it, and kg-health covered only the KG. Python side (packages/decepticon/decepticon/diagnostics.py): - deep_report()/slot_matrix(): for all 26 configured roles, report which of the 20 slots resolve; conditional slots are evaluated against the same guard conditions the factories use (HITL env toggle, fallback chains, KGStore config) — no agent construction, no fake engagement state. - KG reachability probed read-only via a diagnostics-scoped count. - CLI: python -m decepticon.diagnostics (JSON, exit code 0/1). Launcher side: - compose.ExecCapture: capture output of a service exec (context-aware). - decepticon doctor --deep: folds the runtime report into the doctor checks (runtime_kg, runtime_skillogy, runtime_hitl, runtime_model_fallback, runtime_mcp_auth, runtime_slot_matrix) with per-component recovery hints; stack-down degrades to an optional failed check, not an error. Addresses #1040 (host half; in-session /doctor to follow)
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bd787e1231
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // langgraph container (python -m decepticon.diagnostics) and folds the | ||
| // result into the doctor report. Missing stack is not an error here: the | ||
| // base checks already report service status. | ||
| func addDeepRuntimeChecks(ctx context.Context, report *doctorReport) { |
There was a problem hiding this comment.
The launcher command in doctor.go still registers only --json and calls diagnose unconditionally, and a repository-wide search finds no caller of addDeepRuntimeChecks. Consequently decepticon doctor --deep is rejected as an unknown flag and none of the newly added runtime diagnostics can be reached; register the flag and invoke this function when it is set.
Useful? React with 👍 / 👎.
| out, err := cmd.Output() | ||
| if err != nil { | ||
| return nil, fmt.Errorf("%s compose exec %s: %w", c.Runtime.Name, service, err) | ||
| } |
There was a problem hiding this comment.
When KG or enabled Skillogy is unhealthy, diagnostics.py deliberately prints the component report and exits 1, but cmd.Output() returns that stdout alongside an ExitError and this branch discards it. These are exactly the failure cases deep doctor must explain, yet users receive only a generic compose-exec error instead of the KG/Skillogy details and slot matrix; retain and parse stdout for expected diagnostic exit failures.
Useful? React with 👍 / 👎.
| for role in SLOTS_PER_ROLE: | ||
| chain = os.environ.get(f"DECEPTICON_MODEL_{role.upper()}_FALLBACK", "").strip() | ||
| roles[role] = chain | ||
| if chain: | ||
| any_chain = True | ||
| return { | ||
| "ok": any_chain, |
There was a problem hiding this comment.
This does not match the runtime guard, which resolves each role's fallback_models through LLMFactory.get_fallback_models() from the configured credential chain. A normal installation with multiple authentication methods can therefore have active fallback middleware while no per-role environment variable exists and be reported absent; conversely, one role-specific variable makes any_chain mark the slot resolved for every role. Derive the result per role from the same model assignments used during agent construction.
Useful? React with 👍 / 👎.
| // addDeepRuntimeChecks runs the read-only diagnostics module inside the | ||
| // langgraph container (python -m decepticon.diagnostics) and folds the | ||
| // result into the doctor report. Missing stack is not an error here: the | ||
| // base checks already report service status. |
There was a problem hiding this comment.
This commit adds a user-visible launcher option and diagnostic behavior but leaves the root CHAGELOG.md unchanged. Add the proposed change and PR link under Unreleased as required by the repository's versioned-change policy.
AGENTS.md reference: AGENTS.md:L3-L8
Useful? React with 👍 / 👎.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
# Conflicts: # CHAGELOG.md
- register the --deep flag: addDeepRuntimeChecks existed but nothing called it, so the whole in-container diagnostics surface was dead code - _truthy(None) now reads False: str(None) is "none", which silently passed the falsy set — unset HITL reported OK and unset Skillogy failed instead of reporting the filesystem fallback - model_fallback resolves through LLMFactory.get_fallback_models (the credential-chain mapping assembly actually feeds the middleware) instead of the nonexistent DECEPTICON_MODEL_<ROLE>_FALLBACK env var, so the report reflects the operator's real fallback chains
2 participants
Partially addresses #1040
Root cause
Conditional middleware slots (KG, HITL, model_fallback) fail silently at assembly time — their factories return None plus one log line and the slot is absent. The host-side doctor only saw Docker/config/auth/LangGraph reachability;
kg-healthcovered KG alone. Operators couldn't tell which of the 20 slots resolved for any role.What
Python runtime (
packages/decepticon/decepticon/diagnostics.py):deep_report()/slot_matrix(): read-only matrix for all 26 roles, evaluating the same conditional guards (HITL env toggle, fallback chains, KG config/reachability) without constructing agents or creating engagement state.python -m decepticon.diagnosticsemits JSON and exits 0/1.Launcher:
compose.ExecCapturefor read-only service exec.decepticon doctor --deepfolds the runtime report into checks: runtime_kg, runtime_skillogy, runtime_hitl, runtime_model_fallback, runtime_mcp_auth, runtime_slot_matrix, with recovery hints.Verification
Follow-up
In-session
/doctor(scope/ROE/OPPLAN/workflow pins/checkpoints/budget) remains a separate client UX follow-up, as proposed in #1040.