Skip to content

feat: configuration change notifications (AssaultObserver.onConfigChange) #71

Description

@ErwanLT

v0.4.0 -- Resilience verification, human or agent-driven

Context

AssaultObserver is the extension's observability SPI -- the single integration point both optional modules use
(quarkus-goblin-metrics, quarkus-goblin-opentelemetry) -- and its Javadoc already names post-assault assertions as a
future consumer. Today it reports two things:

default void onAssault(AssaultEngine.AssaultRecord record) {}
default void onActiveChange(boolean active) {}

There is no notification for a configuration change. Yet the configuration is the most important signal in the
whole system: the assault toggles, the latency range, the exception class, the target level, the armed layers. A
recorded AssaultRecord carries a configSnapshot string, but that only covers the assaults that actually fired -- an
application cannot know that someone armed a 5-second delay, or switched to the TIMEOUT profile, unless it is
watching.

The live test on quarkus-goblin-demo made this concrete: the application could not record the exact attack it went
through, so it could not replay it after a fix. This is also what an agent needs: without a
"the human/agent changed the configuration" event, an experiment is not reproducible.

Goal

Add AssaultObserver.onConfigChange so observers see every configuration change, not only assaults and activation
changes -- enough for an application (or a test) to record the exact attack and replay it after a fix.

Design decisions to make

  • Signature. Options:
    1. onConfigChange(MutableAssaultConfig config) -- minimal, and the config object already carries
      describeAssaults().
    2. onConfigChange(AssaultConfigChange change) with previous / current descriptions and a timestamp -- the
      observer does not have to hold state to diff, and "the exact attack" is readable straight off the event.
      Recommend (2): the use case in the roadmap is recording what happened, and a before/after pair is what makes that
      record self-contained.
  • Frozen snapshot. The parameter must be a read-only snapshot (MutableAssaultConfig.snapshot()), never the live
    mutable config: an observer running on the mutating thread must not be able to mutate it, and every field it reads
    must come from the same state.
  • Wiring. MutableAssaultConfig has a single Runnable onChange listener, currently installed by the engine as
    setOnChange(this::persistConfig) -- and only in dev mode (see AssaultEngine.initialize). Persistence must stay
    dev-only, but the observer notification must also fire in test mode, which is exactly where a test wants to
    record the attack it just injected. So the engine's listener has to compose both, with the persistence half still
    dev-only. Decide whether MutableAssaultConfig grows a listener list or the engine composes the two inside its
    single Runnable; composing inside the engine keeps the config class unchanged and is the smaller change.
  • Not every mutation notifies. validateAndFix() and restoreProfile() deliberately publish without notifying,
    and putRawHeaderRuleForTests is a test hook. Startup load happens before observers matter, so this is fine -- but
    it must be documented on the method, otherwise an observer author will assume they see everything.
  • Failure handling. notifyObservers already swallows and logs a failing observer, which is right on the request
    path ("observability must never break an assault"). A config change is a rare, deliberate event, so a failure
    there is a real problem worth a WARN -- and worth surfacing in the Dev UI, not a debug line.
  • Threading. The listener runs on the thread that mutated the config (Dev UI click, JSON-RPC call, agent tool call)
    -- not a request thread. Observers must not block; this must be stated on the SPI.

Scope

  • Add the method to the AssaultObserver SPI with a default no-op, so both existing modules and any third-party
    observer keep working untouched.
  • Wire it in AssaultEngine (dev and test), composed with the dev-only persistence.
  • Update the SPI Javadoc: the threading contract, the frozen parameter, what is not notified, and the new
    "integration point for post-assault assertions and experiment recording" role.

Acceptance criteria

  • Every Dev UI / JSON-RPC / Dev MCP configuration change notifies the observers exactly once, with a before/after pair.
  • A multi-field change published atomically (applyConfig, a profile switch, a scenario load from feat: saved chaos scenarios #49) produces one
    notification, not one per field -- workingCopy() + replaceWith() is the path that guarantees this.
  • The notification fires in test mode, so an integration test can assert on the exact configuration an assault ran with.
  • The existing quarkus-goblin-metrics and quarkus-goblin-opentelemetry observers compile and behave unchanged (the
    new method has a default), and their tests still pass.
  • A failing observer is logged at WARN and never propagates to the caller of the Dev UI / JSON-RPC.
  • An observer cannot mutate the configuration it is handed.
  • A unit test for the notification, and a test proving persistence is still dev-only (no .goblin-state.json written in
    test mode).
  • how-it-works.adoc (which documents the observer SPI) and the roadmap item updated.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions