feat: job maintenance API (remove / drain / clean / obliterate)

[jotarios]

Summary

Adds the job maintenance API — remove, drain, clean, obliterate — across the engine and both shims. The Node Queue.remove / drain / clean / obliterate methods previously threw NotSupportedError; the Python Queue lacked them entirely. All four now route through new engine surface.

Streams-based semantics, adapted (not copied) from list-based queues:

• remove(jobId) — delete one job from every surface it could live on: the delayed ZSET (+ didx / dlid side-indexes), a waiting or active main-stream entry, the DLQ, and the result key. Idempotent — a missing id returns an all-false report, not an error.
• drain(delayed?) — clear waiting stream entries (not in any consumer-group PEL) plus, by default, the delayed ZSET. In-flight jobs are left running.
• clean(grace, limit, state) — age- + state-filtered bulk delete for completed / failed / delayed / waiting, capped at limit, returning the removed job ids.
• obliterate() — tear the whole {chasqui:<queue>} keyspace down via a batched SCAN + UNLINK.

What shipped

Engine (chasquimq/src/producer/maintenance.rs, new) — Producer::remove / drain / clean / obliterate, plus RemovalReport / DrainOptions. Three new atomic Lua scripts (REMOVE_STREAM_ENTRY, DRAIN_STREAM, CLEAN_STREAM) follow the existing EVALSHA → NOSCRIPT → EVAL self-heal pattern; the delayed branch reuses CANCEL_DELAYED_SCRIPT verbatim. All scans bounded so no call blocks Redis. Off the hot path — zero diff to consumer/, ack.rs, promoter.rs, scheduler.rs, and no add* method touched.

Node shim — four new napi producer bindings + un-stubbed Queue.remove / drain / clean / obliterate; Queue.removeReport exposes the per-surface RemovalReport.

Python shim — four PyO3 bindings + Queue.remove / remove_report / drain / clean / obliterate (previously absent); _native.pyi updated.

CLI — chasqui clean <queue> --state <s> --grace-ms <ms> --limit <n> and chasqui obliterate <queue>, both gated behind a y/N confirm unless --yes.

Docs — docs/engine.md (new section), docs/history.md (slice entry), both shim READMEs (symmetric sections), root README feature row, reference/{node-api,python-api,rust-api,cli}.md, a new guides/clean-and-obliterate.mdx registered in the sidebar. Stale claims fixed inline: the "drain/clean/obliterate are stubbed" note, the "engine doesn't implement most job states" note, and the DLQ guide's "no drop command" line.

Notable design call

clean(delayed) ages by the job's created_at_ms (when scheduled), not the ZSET run-at score — a future-scheduled job is otherwise impossible to clean deterministically. "Drop delayed jobs created more than grace ago" is a stable, testable semantic.

Bugs caught in review

A diff review surfaced two correctness bugs, both fixed with regression tests:

• drain multi-pass termination — the pass loop stopped on a partial scan page, not on zero. With active entries interleaved near the front of the stream, a pass legitimately deletes fewer than a full page while thousands of waiting jobs remain further back. Now stops only when a pass deletes zero.
• clean(waiting) PEL skip — the candidate scan walked the main stream, which mixes waiting and active entries, so an old-enough in-flight job could be swept into the delete set. clean(waiting) now subtracts the consumer-group PEL.

Testing

• Engine: 21 live-Redis integration tests (every state, idempotency, active-survives-drain/clean, obliterate-then-reuse, multi-pass-past-scan-page).
• Node: 13 vitest cases. Python: 14 pytest cases.
• Full suites green: 356 Rust (38 suites), 185 Node, 149 Python. cargo fmt / clippy -D warnings / tsc / prettier all clean.

Benchmark

Same-host run (load avg ~3.5, contended): queue-add-bulk 179,572 jobs/s, worker-concurrent 108,701 jobs/s — within the host-load-noise envelope vs the chasquimq-1.0.md baseline. The maintenance API adds zero code to the producer/consumer hot paths (verified: empty diff to consumer/ and add*), so the small deltas are host contention, not a regression — consistent with the host-load gate in benchmarks/README.md.