Problem
The maker package generates and executes AWS CLI plans that create/modify/delete real cloud resources, with retry, idempotency ("already exists"=success), glue-rule escalation, auto-remediation, and LLM-driven agentic fixes (up to 3 attempts, exponential backoff). The execution loop (exec.go) and the agentic failure-remediation path (agentic_fix.go) have no dedicated tests — the only tests present are narrow per-provider filter/binding tests (exec_flyio_test.go, exec_tencent_filter_test.go, exec_verda_test.go, binding_test.go, plan_cost_test.go). This is the single most dangerous code surface in the ecosystem (it spends money and mutates infra), and its core decision logic — when to treat a failure as success, when to prune vs preserve critical commands like run-instances/create-load-balancer, retry/backoff bounds — is unverified.
Where
/Users/nashrafeeg/workspace/clanker/internal/maker/exec.go (no exec_test.go)/Users/nashrafeeg/workspace/clanker/internal/maker/agentic_fix.go (no test)/Users/nashrafeeg/workspace/clanker/internal/maker/ (46 source files, only 8 test files, mostly per-provider filters)
Root cause
Testing effort went to peripheral provider filters; the central exec loop and agentic-fix retry logic were never given a test harness with a faked CLI executor.
How to fix
Introduce an executor seam (an interface around the os/exec call) so exec.go can run against a fake that returns canned stdout/stderr/exit codes, following the pattern already used in clanker-cloud's awsheal mocks_test.go. Add table-driven tests for: idempotent "already exists" success detection, the never-prune critical-command guard, placeholder (<UPPERCASE_SNAKE>) JSONPath binding from CLI outputs (extend binding_test.go), and agentic_fix.go backoff/attempt-cap behavior with a stubbed LLM. Run these under -race in CI.
Acceptance criteria
go test ./internal/maker/... covers exec.go and agentic_fix.go; a regression that prunes a critical command or mis-detects an idempotent failure as fatal fails a test.
Severity: high · from holistic hardening review.
Problem
The maker package generates and executes AWS CLI plans that create/modify/delete real cloud resources, with retry, idempotency ("already exists"=success), glue-rule escalation, auto-remediation, and LLM-driven agentic fixes (up to 3 attempts, exponential backoff). The execution loop (exec.go) and the agentic failure-remediation path (agentic_fix.go) have no dedicated tests — the only tests present are narrow per-provider filter/binding tests (exec_flyio_test.go, exec_tencent_filter_test.go, exec_verda_test.go, binding_test.go, plan_cost_test.go). This is the single most dangerous code surface in the ecosystem (it spends money and mutates infra), and its core decision logic — when to treat a failure as success, when to prune vs preserve critical commands like run-instances/create-load-balancer, retry/backoff bounds — is unverified.
Where
/Users/nashrafeeg/workspace/clanker/internal/maker/exec.go (no exec_test.go)/Users/nashrafeeg/workspace/clanker/internal/maker/agentic_fix.go (no test)/Users/nashrafeeg/workspace/clanker/internal/maker/ (46 source files, only 8 test files, mostly per-provider filters)Root cause
Testing effort went to peripheral provider filters; the central exec loop and agentic-fix retry logic were never given a test harness with a faked CLI executor.
How to fix
Introduce an executor seam (an interface around the os/exec call) so exec.go can run against a fake that returns canned stdout/stderr/exit codes, following the pattern already used in clanker-cloud's awsheal mocks_test.go. Add table-driven tests for: idempotent "already exists" success detection, the never-prune critical-command guard, placeholder (<UPPERCASE_SNAKE>) JSONPath binding from CLI outputs (extend binding_test.go), and agentic_fix.go backoff/attempt-cap behavior with a stubbed LLM. Run these under -race in CI.
Acceptance criteria
go test ./internal/maker/... covers exec.go and agentic_fix.go; a regression that prunes a critical command or mis-detects an idempotent failure as fatal fails a test.
Severity: high · from holistic hardening review.