Skip to content

Feature Request: Progressive ramp workflow with validation checkpoints #106346

Description

@jakesciotto

Is your feature request related to a problem?

Teams want to start an experiment at a small rollout (for example 5%), confirm that exposure works and that no guardrail metric moves, and then increase the rollout in steps. Today this is a manual loop: change the rollout percentage on the flag, wait, open the exposures tab, check the split and the $multiple count, check the guardrail metrics, and repeat. Nothing in the product guides or records this loop. Existing work covers parts of it: SRM explanation (PR #106289), $multiple warning (#59710), guardrail alerts (#67202), and alerts that can stop an experiment (#101995). No issue covers the ramp itself.

Describe the solution you'd like

  1. A Ramp plan on the experiment: an ordered list of rollout steps, for example 5% → 25% → 50% → 100%, each with a minimum hold time.
  2. A checkpoint between steps. PostHog shows a checklist for the current step: exposure split matches the rollout (SRM check), $multiple rate is under the threshold, exposures are counted in the expected event, no guardrail metric has a significant negative movement.
  3. The user confirms each step by hand. Optionally, the step can advance automatically when all checks pass (opt-in).
  4. Each step change is written to the activity log with the checkpoint results.
  5. If a checkpoint fails, show the diagnosis and the next action (link to Experiment diagnostics: advise what to do #22101).

Describe alternatives you've considered

Additional context

Related: #67202, #101995, #59710, #22101, #35750, PR #106289, PR #57767 (significance alerts).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions