fix(selfhost): stop the redaction scrubber nulling every AI token count (#10212)

SECRET_KEY matches /token/i, so scrubRecord -- wired as posthog-node's
before_send -- rewrote PostHog's own $ai_input_tokens and $ai_output_tokens to
the "[redacted]" STRING, which PostHog then coerced to null on its
numerically-typed properties.

The result: not one AI call in the project has ever carried a token count.
posthog.ai_events.input_tokens/output_tokens/total_tokens are NULL for every
model over the retention window, including claude-sonnet-5 at 2,321 calls and
$498.43 of real spend. $ai_total_cost_usd came through untouched because it has
no secret-shaped word in it. PostHog derives $ai_input_cost_usd/$ai_output_cost_usd
from tokens, so those could not be computed either -- and the miner-side split
landed in #10199 would have been scrubbed the same way.

A secret-shaped key holding a NUMBER is a counter, not a credential: every secret
this module exists to catch is a string, and there is no numeric form of one to
leak. Skip redaction for numbers only; a string, object, array or boolean under
the same key is still redacted exactly as before.

Deliberately general rather than an allowlist of the two $ai_* keys -- an
allowlist goes stale the moment PostHog adds $ai_cache_read_input_tokens, and it
would fail the same silent way.

Closes #10211

Author: JSONbored

src/selfhost/redaction-scrub.ts

@@ -232,7 +232,14 @@ export function scrubRecord(obj: unknown, depth: number): void {
       delete rec[key];
       continue;
     }
-    if (shouldRedactKey(key)) {
+    // #10211: a secret-shaped KEY holding a NUMBER is a counter, not a credential. Every secret this module
+    // exists to catch -- a token, an API key, a password, a cookie, a DSN, a bearer header -- is a string;
+    // there is no numeric form of one to leak. Redacting numbers anyway silently destroyed real telemetry:
+    // SECRET_KEY matches /token/i, so PostHog's own `$ai_input_tokens`/`$ai_output_tokens` were rewritten to
+    // the "[redacted]" STRING, which PostHog then coerced to null on its numerically-typed properties. The
+    // result was that not one AI call in the project ever carried a token count, while cost came through
+    // untouched (`$ai_total_cost_usd` has no secret-shaped word in it).
+    if (shouldRedactKey(key) && typeof rec[key] !== "number") {
       rec[key] = REDACTED;
       continue;
     }

test/unit/selfhost-redaction-scrub.test.ts

@@ -87,3 +87,56 @@ describe("scrubRecord — end-to-end via the structured-identifier keys (#9142)"
     expect(properties.detail).toContain("private context");
   });
 });
+
+describe("scrubRecord — a secret-shaped key holding a NUMBER is a counter, not a credential (#10211)", () => {
+  it("REGRESSION: PostHog's own token-count properties survive the scrub", () => {
+    // SECRET_KEY matches /token/i, so these were rewritten to the "[redacted]" STRING and PostHog then
+    // coerced that to null on its numerically-typed properties -- not one AI call in the live project
+    // ever carried a token count, while $ai_total_cost_usd came through untouched.
+    const properties: Record<string, unknown> = {
+      $ai_input_tokens: 1200,
+      $ai_output_tokens: 300,
+      $ai_total_cost_usd: 0.44,
+      tokens_used: 1500,
+    };
+    scrubRecord(properties, 0);
+    expect(properties.$ai_input_tokens).toBe(1200);
+    expect(properties.$ai_output_tokens).toBe(300);
+    expect(properties.$ai_total_cost_usd).toBe(0.44);
+    expect(properties.tokens_used).toBe(1500);
+  });
+
+  it("a zero count is preserved rather than read as absent", () => {
+    const properties: Record<string, unknown> = { $ai_input_tokens: 0 };
+    scrubRecord(properties, 0);
+    expect(properties.$ai_input_tokens).toBe(0);
+  });
+
+  it("still redacts a secret-shaped key holding a STRING, which is the only shape a real credential takes", () => {
+    const properties: Record<string, unknown> = {
+      api_token: ["ghp", "abcdefghijklmnopqrst123456"].join("_"),
+      password: "hunter2",
+      authorization: "Bearer abc.def.ghi",
+      session_cookie: "sid=abc123",
+    };
+    scrubRecord(properties, 0);
+    expect(properties.api_token).toBe(REDACTED);
+    expect(properties.password).toBe(REDACTED);
+    expect(properties.authorization).toBe(REDACTED);
+    expect(properties.session_cookie).toBe(REDACTED);
+  });
+
+  it("still redacts a secret-shaped key holding a non-numeric, non-string value", () => {
+    // An object or array under a secret-shaped key is not a counter, so the number carve-out must not
+    // widen into "anything that is not a string".
+    const properties: Record<string, unknown> = {
+      credentials: { token: "abc" },
+      api_keys: ["one", "two"],
+      secret_flag: true,
+    };
+    scrubRecord(properties, 0);
+    expect(properties.credentials).toBe(REDACTED);
+    expect(properties.api_keys).toBe(REDACTED);
+    expect(properties.secret_flag).toBe(REDACTED);
+  });
+});