You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
v0.4.0 -- Resilience verification, human or agent-driven
Context
AssaultObserver is the extension's observability SPI -- the single integration point both optional modules use
(quarkus-goblin-metrics, quarkus-goblin-opentelemetry) -- and its Javadoc already names post-assault assertions as a
future consumer. Today it reports two things:
There is no notification for a configuration change. Yet the configuration is the most important signal in the
whole system: the assault toggles, the latency range, the exception class, the target level, the armed layers. A
recorded AssaultRecord carries a configSnapshot string, but that only covers the assaults that actually fired -- an
application cannot know that someone armed a 5-second delay, or switched to the TIMEOUT profile, unless it is
watching.
The live test on quarkus-goblin-demo made this concrete: the application could not record the exact attack it went
through, so it could not replay it after a fix. This is also what an agent needs: without a
"the human/agent changed the configuration" event, an experiment is not reproducible.
Goal
Add AssaultObserver.onConfigChange so observers see every configuration change, not only assaults and activation
changes -- enough for an application (or a test) to record the exact attack and replay it after a fix.
Design decisions to make
Signature. Options:
onConfigChange(MutableAssaultConfig config) -- minimal, and the config object already carries describeAssaults().
onConfigChange(AssaultConfigChange change) with previous / current descriptions and a timestamp -- the
observer does not have to hold state to diff, and "the exact attack" is readable straight off the event.
Recommend (2): the use case in the roadmap is recording what happened, and a before/after pair is what makes that
record self-contained.
Frozen snapshot. The parameter must be a read-only snapshot (MutableAssaultConfig.snapshot()), never the live
mutable config: an observer running on the mutating thread must not be able to mutate it, and every field it reads
must come from the same state.
Wiring.MutableAssaultConfig has a single Runnable onChange listener, currently installed by the engine as setOnChange(this::persistConfig) -- and only in dev mode (see AssaultEngine.initialize). Persistence must stay
dev-only, but the observer notification must also fire in test mode, which is exactly where a test wants to
record the attack it just injected. So the engine's listener has to compose both, with the persistence half still
dev-only. Decide whether MutableAssaultConfig grows a listener list or the engine composes the two inside its
single Runnable; composing inside the engine keeps the config class unchanged and is the smaller change.
Not every mutation notifies.validateAndFix() and restoreProfile() deliberately publish without notifying,
and putRawHeaderRuleForTests is a test hook. Startup load happens before observers matter, so this is fine -- but
it must be documented on the method, otherwise an observer author will assume they see everything.
Failure handling.notifyObservers already swallows and logs a failing observer, which is right on the request
path ("observability must never break an assault"). A config change is a rare, deliberate event, so a failure
there is a real problem worth a WARN -- and worth surfacing in the Dev UI, not a debug line.
Threading. The listener runs on the thread that mutated the config (Dev UI click, JSON-RPC call, agent tool call)
-- not a request thread. Observers must not block; this must be stated on the SPI.
Scope
Add the method to the AssaultObserver SPI with a default no-op, so both existing modules and any third-party
observer keep working untouched.
Wire it in AssaultEngine (dev and test), composed with the dev-only persistence.
Update the SPI Javadoc: the threading contract, the frozen parameter, what is not notified, and the new
"integration point for post-assault assertions and experiment recording" role.
Acceptance criteria
Every Dev UI / JSON-RPC / Dev MCP configuration change notifies the observers exactly once, with a before/after pair.
A multi-field change published atomically (applyConfig, a profile switch, a scenario load from feat: saved chaos scenarios #49) produces one
notification, not one per field -- workingCopy() + replaceWith() is the path that guarantees this.
The notification fires in test mode, so an integration test can assert on the exact configuration an assault ran with.
The existing quarkus-goblin-metrics and quarkus-goblin-opentelemetry observers compile and behave unchanged (the
new method has a default), and their tests still pass.
A failing observer is logged at WARN and never propagates to the caller of the Dev UI / JSON-RPC.
An observer cannot mutate the configuration it is handed.
A unit test for the notification, and a test proving persistence is still dev-only (no .goblin-state.json written in
test mode).
how-it-works.adoc (which documents the observer SPI) and the roadmap item updated.
v0.4.0 -- Resilience verification, human or agent-driven
Context
AssaultObserveris the extension's observability SPI -- the single integration point both optional modules use(
quarkus-goblin-metrics,quarkus-goblin-opentelemetry) -- and its Javadoc already names post-assault assertions as afuture consumer. Today it reports two things:
There is no notification for a configuration change. Yet the configuration is the most important signal in the
whole system: the assault toggles, the latency range, the exception class, the target level, the armed layers. A
recorded
AssaultRecordcarries aconfigSnapshotstring, but that only covers the assaults that actually fired -- anapplication cannot know that someone armed a 5-second delay, or switched to the
TIMEOUTprofile, unless it iswatching.
The live test on
quarkus-goblin-demomade this concrete: the application could not record the exact attack it wentthrough, so it could not replay it after a fix. This is also what an agent needs: without a
"the human/agent changed the configuration" event, an experiment is not reproducible.
Goal
Add
AssaultObserver.onConfigChangeso observers see every configuration change, not only assaults and activationchanges -- enough for an application (or a test) to record the exact attack and replay it after a fix.
Design decisions to make
onConfigChange(MutableAssaultConfig config)-- minimal, and the config object already carriesdescribeAssaults().onConfigChange(AssaultConfigChange change)withprevious/currentdescriptions and a timestamp -- theobserver does not have to hold state to diff, and "the exact attack" is readable straight off the event.
Recommend (2): the use case in the roadmap is recording what happened, and a before/after pair is what makes that
record self-contained.
MutableAssaultConfig.snapshot()), never the livemutable config: an observer running on the mutating thread must not be able to mutate it, and every field it reads
must come from the same state.
MutableAssaultConfighas a singleRunnable onChangelistener, currently installed by the engine assetOnChange(this::persistConfig)-- and only in dev mode (seeAssaultEngine.initialize). Persistence must staydev-only, but the observer notification must also fire in test mode, which is exactly where a test wants to
record the attack it just injected. So the engine's listener has to compose both, with the persistence half still
dev-only. Decide whether
MutableAssaultConfiggrows a listener list or the engine composes the two inside itssingle
Runnable; composing inside the engine keeps the config class unchanged and is the smaller change.validateAndFix()andrestoreProfile()deliberately publish without notifying,and
putRawHeaderRuleForTestsis a test hook. Startup load happens before observers matter, so this is fine -- butit must be documented on the method, otherwise an observer author will assume they see everything.
notifyObserversalready swallows and logs a failing observer, which is right on the requestpath ("observability must never break an assault"). A config change is a rare, deliberate event, so a failure
there is a real problem worth a
WARN-- and worth surfacing in the Dev UI, not adebugline.-- not a request thread. Observers must not block; this must be stated on the SPI.
Scope
AssaultObserverSPI with a default no-op, so both existing modules and any third-partyobserver keep working untouched.
AssaultEngine(dev and test), composed with the dev-only persistence."integration point for post-assault assertions and experiment recording" role.
Acceptance criteria
applyConfig, a profile switch, a scenario load from feat: saved chaos scenarios #49) produces onenotification, not one per field --
workingCopy()+replaceWith()is the path that guarantees this.quarkus-goblin-metricsandquarkus-goblin-opentelemetryobservers compile and behave unchanged (thenew method has a default), and their tests still pass.
WARNand never propagates to the caller of the Dev UI / JSON-RPC..goblin-state.jsonwritten intest mode).
how-it-works.adoc(which documents the observer SPI) and the roadmap item updated.