Skip to content

Add guided faucet activation experiment#677

Draft
juandolealt wants to merge 2 commits into
agent/docs-growth-funnel-analyticsfrom
agent/guided-faucet-activation
Draft

Add guided faucet activation experiment#677
juandolealt wants to merge 2 commits into
agent/docs-growth-funnel-analyticsfrom
agent/guided-faucet-activation

Conversation

@juandolealt

@juandolealt juandolealt commented Jul 11, 2026

Copy link
Copy Markdown
Member

Summary

  • Adds a docs-only guided activation experiment from a confirmed faucet claim to a confirmed testnet payment.
  • Treats a faucet claim as successful only when at least one receipt returns and every receipt succeeds.
  • Treats a payment as successful only when the receipt succeeds and the decoded transfer recipient matches the requested recipient.
  • Uses only explicit control or guided_handoff multivariate assignments and locks the first valid assignment.
  • Shows the normal control experience to unresolved users but labels them unassigned so they can be excluded from A/B analysis.
  • Retains a scope-limited seven-day journey marker to support the planned repeat-payment metric while keeping the primary conversion window at 24 hours.
  • Records whether journey persistence succeeded so storage failures can be excluded and reported.
  • Uses typed, privacy-safe PostHog events without identify calls, wallet addresses, transaction hashes, memos, query strings, or raw errors.

Why the hardening was needed

The initial draft could classify fast users as control before PostHog returned an assignment, and a later flag refresh could change the claim variant after the exposure event. PostHog can also invoke an already-loaded feature-flag callback without the optional context argument. Finally, the 24-hour journey expiry could not support the planned seven-day repeat-payment metric. The latest commit closes those measurement gaps without changing the normal faucet experience for unresolved users.

Stack and release status

This PR is stacked on #676 and targets agent/docs-growth-funnel-analytics.

This is not production-ready. Do not merge until #676 lands, the experiment setup is reviewed, and explicit production release permission is given. This PR does not create or enable the PostHog feature flag.

The launch configuration must use the exact multivariate keys control and guided_handoff. Boolean or missing values remain unassigned.

Measurement boundary

The experiment covers the interactive address-form and connected-wallet faucet flows plus the interactive send-payment guide. Direct cURL and RPC claims remain excluded because their current API response confirms submission, not receipt success.

Emitted events:

  • faucet_viewed
  • faucet_claim_started
  • faucet_claim_succeeded
  • faucet_claim_failed
  • quickstart_started
  • testnet_payment_succeeded
  • testnet_payment_failed

For A/B analysis, include only the first successful claim per anonymous browser where experiment_variant is control or guided_handoff and journey_persisted is true. Exclude unassigned journeys. First payment remains the first successful payment event within 24 hours. The seven-day retention supports the separate repeat-payment metric. Testnet developer activation remains separate from mainnet business activation.

Validation

  • pnpm check
  • pnpm check:types
  • CI=true pnpm test (174 tests)
  • CI=true pnpm build
  • CI=true VITE_E2E=true VITE_USE_HTTP=true pnpm build
  • CI=true VITE_E2E=true VITE_USE_HTTP=true pnpm exec playwright test e2e/faucet.test.ts --project=chromium (2 tests)
  • git diff --check

@vercel

vercel Bot commented Jul 11, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
tempo-docs Ready Ready Preview, Comment Jul 11, 2026 11:38am

Request Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant