Skip to content

Make Kubernetes log tools gather the right evidence for agents#10

Open
rshdhere wants to merge 2 commits into
codag-megalith:mainfrom
rshdhere:feat/tail-kubernetes-incident
Open

Make Kubernetes log tools gather the right evidence for agents#10
rshdhere wants to merge 2 commits into
codag-megalith:mainfrom
rshdhere:feat/tail-kubernetes-incident

Conversation

@rshdhere

@rshdhere rshdhere commented Jul 25, 2026

Copy link
Copy Markdown

What problem does this solve?

AI agents debugging Kubernetes usually have to invent a workflow: find the deployment’s pods, pull logs, remember --previous for crash loops, check Warning events, then paste a huge raw dump into context.

Codag already compresses logs. This PR makes tail_kubernetes better at gathering the right evidence first, then compacting once.

What changed

Agents can call structured fields instead of only raw kubectl logs args:

{
  "target": "deployment/prod-api",
  "namespace": "prod",
  "since": "1h"
}

That path:

  1. Resolves the deployment (or label selector) to pods
  2. Fetches current logs (--all-containers)
  3. Fetches --previous when restartCount > 0 (or when previous=true)
  4. Optionally includes Warning events
  5. Tags lines ([pod=…], [event]) and sends one compact request

Legacy args=[...] passthrough is unchanged.

Demo honesty (please read)

Token figures use Codag’s existing estimator: characters ÷ 4 (internal/passthrough.EstTokens). They are not tiktoken / model-exact counts.

What is real

  • Context + raw logs: live EKS cluster, argocd/argocd-repo-server, kubectl logs --since=6h.
  • Compression: real codag wrap --quiet --deterministic against a slice of those logs (hosted free/deterministic compact path — not invented in this PR).
  • Structured gather logic: covered by unit tests with a mocked kubectl runner (resolve → multi-pod → previous → events → tags).

What was misleading before (corrected)

An earlier draft compared full 6h raw (~314k est. tokens) to compact output from only the first 800 lines. That is not the same window.

Same-window measurement (800-line slice of the 6h dump):

Stage Est. tokens (chars÷4)
Raw (800 lines) ~76,600
After codag wrap --deterministic ~22,200
Reduction ~71%

Full 6h raw is still large (~314k est. tokens); that number is only “how big the unfiltered stream is,” not the compact baseline.

What this PR did not prove in the screenshots

  • The live structured MCP call (target=deployment/...) hit connection reset by peer against the hosted compact API on the full gather, so the AFTER image is codag wrap, not a successful end-to-end tail_kubernetes structured tool result on that cluster.
  • The multi-pod / previous / events screenshot is from the mocked fetch path (same code as production, fake kubectl responses).

Images

Context

Raw 6h sample + size

codag wrap compact on 800-line window

Structured fetch (mocked kubectl)

The side-by-side image still mentions the older framing; treat the table above as the source of truth for numbers.

Why this belongs here

Fits codag-cli: MCP + provider log gathering for agents, reusing the existing compact API. No new hosted product surface.

Happy to delete docs/tail-kubernetes-demo/ before merge if you want a code-only diff.

Test plan

  • go test ./...
  • Unit tests for resolve, previous logs, events, MCP schema/validation
  • Live cluster: raw kubectl size + codag wrap --deterministic on an 800-line window
  • Reviewer: structured tail_kubernetes against your API/cluster
  • Reviewer: legacy args=[pod…] still works

rshdhere added 2 commits July 25, 2026 17:55
…ents

Agents debugging a k8s deployment had to guess pod names, remember
--previous for CrashLoopBackOff, and fetch Warning events separately.
Structured target/selector/namespace fields now resolve pods, fan out
logs (auto --previous on restarts), include events, and compact once.
Legacy args passthrough to kubectl logs remains for power users.
Visual before/after for reviewers: raw Kubernetes log token cost versus
Codag compact output, plus the structured multi-pod fetch path.
@rshdhere

Copy link
Copy Markdown
Author

focus on internal/k8s gather/resolve + MCP wiring, legacy args unchanged.
demo images are optional happy to drop them for a code-only diff.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant