Skip to content

fix(adapters): make base64url.decode reject invalid input everywhere#56

Open
Tleaoo wants to merge 5 commits into
tempoxyz:mainfrom
Tleaoo:fix/base64url-decode-strictness
Open

fix(adapters): make base64url.decode reject invalid input everywhere#56
Tleaoo wants to merge 5 commits into
tempoxyz:mainfrom
Tleaoo:fix/base64url-decode-strictness

Conversation

@Tleaoo

@Tleaoo Tleaoo commented Jul 14, 2026

Copy link
Copy Markdown

Align base64url.decode failure semantics across all six adapters

Where we are today

operations.json declares exactly one legal error type for base64url.decode
encoding_error — yet the six adapters disagree three ways about what
invalid input even is:

Axis 1: characters outside the URL-safe alphabet ("not-valid!!!")

Adapter Behavior on main
golden acceptsBuffer.from(value, 'base64url') silently ignores foreign characters
python repairsre.sub strips foreign characters, then decodes what's left
ruby errors, but as parse_error (undeclared for this op)
go / rust / java encoding_error

Axis 2: valid base64url, bytes not valid UTF-8 ("_w"0xFF)

Adapter Behavior on main
golden succeeds with U+FFFD replacement output
go succeeds with replacement bytes in the JSON result
python errors, but as parse_error
ruby force_encoding("UTF-8") produces an invalid string that blows up later in JSON.generate as unknown_error
rust / java encoding_error

So the same wire input can produce a success, a parse_error, an
unknown_error, or an encoding_error depending on which SDK you ask. The
reference adapter is among the most wrong (it manufactures data from garbage
input). Nothing catches this because the vector file only round-trips valid
data.

What this PR changes

Target semantics — the strict behavior rust and java already implement, and
the only one compatible with the declared errorTypes:
reject foreign characters, reject undecodable bytes, always as encoding_error.

  • golden — validate the alphabet before decoding; decode UTF-8 with
    TextDecoder(..., { fatal: true }).
  • python — replace the character-stripping "repair" with validation;
    route base64url-* failures to encoding_error instead of lumping decode
    in with parse_error. (The helper's only caller is the decode command, so
    nothing else shifts.)
  • ruby — move base64url-decode out of the parse_error bucket in
    error_type_for_command; add a valid_encoding? guard so invalid UTF-8
    fails as encoding_error at the operation instead of corrupting the
    response serialization.
  • go — add a utf8.Valid check after decoding (alphabet rejection was
    already correct).

Two new error-case vectors, invalid_characters_rejected and
invalid_utf8_rejected, pin both axes for every adapter from now on.

Verification

  • typescript + python: full vector suites pass, including the two new
    scenarios (base64url vector: 64/64 across both adapters).
  • ruby: local bundler cannot install the SDK gem in this environment, so the
    changed logic was exercised standalone (Base64.urlsafe_decode64 +
    valid_encoding? + the new mapping): invalid chars → encoding_error,
    0xFFencoding_error, round-trip unchanged. The gem is not involved in
    either changed path.
  • go: not compilable here (go.mod wants a newer toolchain than this machine
    has); change is 6 lines against stdlib (unicode/utf8), gofmt -e clean.
    CI compiles it.
  • rust / java: untouched; verified by reading handle_base64url_decode and
    base64urlDecodeToString that they already implement the target semantics,
    so the new vectors pass them as-is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant