Repository navigation
DO NOT MERGE: combined CI for batch 10 (#921) - #1000
Closed
jayden1711 wants to merge 15 commits into
Closed
jayden1711 wants to merge 15 commits into
jayden1711 wants to merge 15 commits into
Conversation
…luded (#889) A Discord bot token passed through redactString unredacted: its pattern lived only in src/watchdog/incidents.ts, so the event log, comments, chat notices and the dashboard's service log kept it. Tailscale keys had the same split in src/tailscale-setup.ts. Both patterns move into PATTERNS in src/events/redact.ts, the one list of credential shapes; incidents.ts, the watchdog's posts and the Tailscale step lines call redactString. The dashboard's service-log file is redacted as one text (with a margin above the kept lines) before it is split, so a private key over several lines is found whole. A bare 32-character key with no prefix stays uncovered by decision: a commit hash, a digest or an engine id has the same shape. test/redact-shapes.test.ts drives a stand-in for every shape (GitHub, Anthropic, Discord, Slack, Linear, Tailscale and the rest) through redactString and each output path: events, comments (file and GitHub backlogs), the superfan digest, notifications (incident post, merge post, chat notice) and the dashboard's service log (journal and file). The backstop, test/redaction-one-module.test.ts, fails on a replace/split in src/, web/src/ or scripts/ whose regex takes in a token stand-in, outside src/events/redact.ts; three own redactors broader than the shared shapes are listed in KNOWN_OWN_REDACTORS (never-grow). PATTERNS, the stand-ins and the lint's fixtures are registered in the weakening hold. Test (fails without): test/redact-shapes.test.ts and test/redaction-one-module.test.ts, 13 of 15 failing on origin/main (Discord and Tailscale in every path, the PEM in the log file, and the lint naming incidents.ts and tailscale-setup.ts). Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
The release note for #889's redaction change, as a fragment under the guardrail heading. Also rewraps the one-module lint's header comment (no code change). The first commit's body counts 13 of 15 tests failing on origin/main; with the two tests added in the adversarial pass (a Discord token after an underscore, a private key above the kept log lines) it is 15 of 17, as the PR body says. Test (fails without): none new; changelog.test.ts and changelog-fragments*.test.ts pass with it. Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com> # Conflicts: # src/merge-policy-weakening.ts
…acted as one text (#889) Round 2, on the owner's two decisions and the final check (#921): - Dashboard keys are now wld_ and 32 random characters, in PATTERNS, so a key is redacted wherever it stands; a 32-character hex hash, a sha or a UUID is not. An older key still opens the dashboard and is not rewritten; doctor warns on each one it can read and says how to rotate it. - KNOWN_OWN_REDACTORS is gone. gh-login, the bench and the root updater redact through the shared redactor; what they masked on their own (a token cut short, a clone URL's user info, a value after a secret's name, a short bearer token) is BROAD in redact.ts, applied with broad: true and each module's own mask word. The updater stays one standalone file with a generated copy of redact.ts (scripts/gen-redact-vendor.mjs), held to the module byte for byte by test/redact-vendored.test.ts. - The dashboard's journal view redacts its entries as one text (redactRecords, with a margin of 100 entries), and clip() redacts before it cuts at 2,000 characters. - Shapes glued to a name after `_` and a lowercase bearer are found. Test (fails without): test/redact-records-keys.test.ts and test/redact-vendored.test.ts, 12 of 14 failing at 2c0142e; with redact-shapes and redaction-one-module, 27 of 30 failing on origin/main 41f6dd4. Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
…keys fail closed, glue class (#889) Round 3, on the second adversarial review and the round-2 final check (#921): - The broad mode (the root updater, gh-login, the bench) no longer applies rules in turn: every rule runs on the text as it was, again with boundaries at the glue points and where a span ends between word characters, and the union is masked once, read again on the result until it settles. The updater's own token rule on main is one of the rules; the URL user-info rule takes a scheme of up to 32 characters from any start; a JWT takes its further dotted parts. On main's three module rules (fixtures in the test), generated input shows no secret more often than before (the regression the final check measured). - Glue class: a token right after an ANSI escape or a JSON backslash escape is found, by a second read with a boundary marked after each (default mode too). - Private keys fail closed: no END redacts to the end of the text, no BEGIN from the start; PGP's PRIVATE KEY BLOCK too. - HTTP Basic credentials are redacted; a dashboard key after `-` or a letter and a token followed by `_` are found. - decision-detail and chat-attachments redact before they cut; doctor checks the hub's key. - The bench drops format characters before redacting and keeps its control characters for after (the redactor finds ANSI glue); its email rule reads a mask inside an address. Test (fails without): test/redact-fail-closed.test.ts and test/redact-broad-union.test.ts, 11 of 14 failing at 8633356. Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
…ed, linear URL rule (#889) Round 4, on the third adversarial review (#921): - Every shape with a distinctive prefix (gh*_, github_pat_, sk-ant-, xox?-, xapp-, lin_*, AKIA/ASIA, AIza, eyJ with a bounded header, wld_, tskey-) is matched with no left boundary, in default and broad mode, so nothing glued before it hides it. Shapes with no distinctive prefix keep a boundary, and the glue points now take ESC in every encoding (the byte, \u001b, \x1b, \033, \e, ^[), each sequence form (CSI, nF, single character) and the 8-bit CSI. - Bearer and Basic match in any case. - The default URL-password rule takes a scheme of up to 32 characters from any start: linear on long dotted runs (it was quadratic). redact.test's linear-time inputs gain dotted runs, and its KNOWN_SLOW exemption for that rule is emptied. - redactCut: decisionDiff, a PR section, a chat attachment and the dashboard's log views redact only what they keep plus a margin (16 KB; 4 KB per log record), not the whole input. - Broad mode's mask subtraction walks the sorted spans and masks once. Test (fails without): test/redact-escapes-windows.test.ts at d0f4ab9: 3 fail and 2 did not finish in 280 s (the linear-time and window tests); 1 passes (a key cut at a window edge). Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
…d, unscannable text fails closed (#889) Round 5, on the fourth adversarial review (#921): - readWindow: a cut site's read window (redactCut's max + 16 KB, a log record's 6,096) ends at its last whitespace, or is dropped when it has none. Redaction earlier in the window (an END line with no BEGIN, a closed key over 16 KB) could shrink the text until the window's raw end, a token cut too short for its rule, landed inside what is shown. - A JWT is read from the start of its header's run with no length bound, and a function finds `eyJ` in the run, so a header over 512 characters glued after a letter, digit, `-` or `%3D` is found. Each start is a run start: linear. - redactString catches V8's RangeError on a run of millions of characters and returns a fixed placeholder (the line count kept for redactRecords) instead of throwing. Test (fails without): test/redact-window-edge.test.ts, 5 of 5 failing at 88ec30b. Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
… any dotted part, no stack overflow (#889) Round 6, on the round-3+4 and round-5 delta checks (#921): - Default mode is order-free: every shape runs on the text as it was (and from each glue point), and each run of overlapping spans is replaced once; a span that covers its run keeps its rule's marker, otherwise `[REDACTED]`. Then again on the result, up to 3 passes, as broad mode does. One rule's match can no longer take in the text another rule starts from (`wld_<key>-redis://u:<pw>@`, `<jwt>.Bearer <tok>`, `xoxb-...-lin_api_<key>`). - The orphan-END rule decides in a function: an END line counts only when no BEGIN opened it, since the rules now read the text with each closed key still in it. - The JWT rule reads every dotted part of a run as a possible header (`sess_0123456789.<jwt>`), and steps over its first two parts with a lookahead and backreference, so a plain run of millions of characters no longer overflows V8's backtracking stack (it is left as it was, as on main). - secretKinds fails closed on a RangeError: it names the text `text too large to scan`, so an attachment is refused (409) until attached anyway. - The dashboard's duplicate doc comment is gone. test/redact-default-union.test.ts's fixtures are registered never-shrink. Tests (fail without): test/redact-default-union.test.ts and test/redact-unscannable.test.ts, 6 of 6 failing at a5c86ec. Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
…eclines nothing, passes fail closed (#889) Round 7, on the round-6 second review and the round-6 delta (#921): - The JWT rule is a plain replacement again: it starts only at a part holding `eyJ` (a lookahead looks in the part, so each part is read once), steps over the header and payload whole (no stack overflow on a huge run), and takes further dotted parts, empty ones too. An unsigned JWT (`alg: none`) or one with a `+`/`/`-led signature after a dotted name (`a.<jwt>.`) was shown and secretKinds missed it: the match stopped before the empty part, its function declined it, and the 64-character restart stepped past the JWT's own start. - By design for every rule: a match a rule's function declines restarts the search at its next character (spansOf), so no start inside it is skipped. test/redact-declined.test.ts holds every function-backed rule to a reference that tries every start, fails when a new one isn't listed, and times 1 MB of declined text. - The passes fail closed (fixedPoint): each mode passes over its own output until nothing changes, at most MAX_PASSES (8) times; a text still changing becomes the placeholder. Before, each stopped after 3 passes. - The Basic value ends where its run and padding end, whatever follows: broad mode showed `Basic <v>==<more>` that default found only in a later pass. - test/redact-default-union.test.ts adds unsigned and `+`-led JWTs and the `x.` and `a.b.` separators; its KEY_QUALIFIERS and CREDENTIAL_ENDINGS are registered never-shrink. Tests (fail without): test/redact-declined.test.ts and test/redact-fixed-point.test.ts, 4 of 6 failing at 1eef362 (the other two are a timing guard and an idempotence guard). Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
src/merge-policy-weakening.ts is main's (40ef312) byte for byte: #886 replaced CHECK_TABLES and LINT_TESTS with in-file marks. Each table #921 registered there is now marked where it is declared, none dropped: - src/events/redact.ts: PATTERNS, BROAD (and CREDENTIAL_WORDS, BROAD_RULES), never-shrink. - scripts/machine/<cli>-update.cjs: the generator puts each of the module's table marks back before the same table in the copy (the transpile drops comments), so PATTERNS, CREDENTIAL_WORDS, BROAD and BROAD_RULES are marked. - test/redact-records-keys.test.ts CUT_SHORT, test/token-standins.ts SHAPES, test/redaction-one-module.test.ts BAD_SOURCES: never-shrink; its GOOD_SOURCES: never-grow; that file carries the lint-test mark. - test/redact-default-union.test.ts: MAIN_PATTERNS, CREDENTIAL_WORDS, KEY_QUALIFIERS, CREDENTIAL_ENDINGS, never-shrink. Measured with the registry probe (central lists plus marks): 0 registrations lost against origin/main, 1128389e or d2a549f. Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
…s, helper tables marked (#889) Round 8, on the round-7 second review (#921): - The URL-password rule and the broad user-info rule read the last 1 to 32 characters of the scheme, with no letter required: a scheme longer than 32 characters whose last 32 are digits, `-` or `.` hid the password from both, where main's rule (a scheme of any length) found it. Still linear (each `://` is reached from at most 32 starts). - src/tailscale-setup.ts replaces the login link before redacting, as main did: a token glued before the link (`tskey-...https://login...`) took in its `https` and the link was shown. - basicShaped also takes a value that decodes to printable text with a `:` (a short user:password with no digit, `+`, `/` or `=`); prose ("Basic Authentication") doesn't decode to that. - GLUED_PREFIXES, KEY_QUALIFIERS, CREDENTIAL_ENDINGS and the glue sequences (now GLUE_PARTS, an array) carry never-shrink marks, in the module and, through the generator, in the updater's copy. SHAPES has a plain `sk-` key. - test/redact-default-union.test.ts adds a long-scheme URL password. Test (fails without): test/redact-round8.test.ts, 4 of 4 failing at fa1318f. Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
No conflicts. #976 adds redactKeys to src/events/redact.ts; the updater's copy is regenerated (node scripts/gen-redact-vendor.mjs --write) so test/redact-vendored.test.ts holds it to the module. Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
Signed-off-by: Jayden Vasquez <jaydenvasquez1711@gmail.com>
This was referenced Oct 21, 2026
Owner
Author
|
Batch 10 done: #921 merged; main's tree equals the tested 1403a632. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Never merged. This draft runs the combined CI for the next batch merge. The orchestrator closes it and deletes the branch when CI finishes.
It starts from main at 1808043 and merges this 1 PR at the exact head shown (
git merge --no-ff, no conflicts). The combined head's tree, 1403a632, is the tree the batch-merge script checks main against afterwards.#921 is treated as critical (shared redactor). On that head it passed: