fix(build): emit ASCII-only bundles so doctrenderer's no-ICU V8 can parse them - #81
Conversation
…arse them
Terser was configured with neither format.ascii_only nor format.quote_keys, so
the webpack pipeline emitted the LaTeX symbol table's supplementary-plane
("astral") keys as bare identifiers -- 131 of them per bundle in word/ and
slide/ sdk-all.js, alongside 43348 raw non-ASCII bytes.
doctrenderer/x2t execute these bundles on a V8 built with
v8_enable_i18n_support=false (core: Common/3dParty/v8/tools/8.9/*/nc-build.sh).
Without ICU its reduced Unicode tables do not classify astral characters as
ID_Start, so it rejects the entire bundle with "SyntaxError: Invalid or
unexpected token". ScriptCompiler::Compile then returns an empty MaybeLocal and
an unguarded ToLocalChecked() aborts the process -- the "Fatal error in
v8::ToLocalChecked / Empty MaybeLocal" and "Illegal instruction (core dumped)"
that make Euro-Office 9.3.4 fail on first start and on every document save.
The Closure Compiler build this replaced escaped all non-ASCII to \uXXXX, so
the ASCII-only invariant held implicitly. The webpack migration dropped it
silently, and no gate noticed: every JS check in check-build.yml runs on Node
and headless Chromium, both full-ICU, which parse these bundles happily.
- format.ascii_only restores the invariant.
- format.quote_keys is redundant with ascii_only today but pins the behaviour
against a future Terser bump. It must sit inside `format`; Terser rejects a
top-level quote_keys outright with "`quote_keys` is not a supported option".
- Two unit tests assert the options and that Terser escapes astral keys rather
than emitting them bare.
- A CI step asserts the emitted bundles contain no byte > 0x7F, which is the
only gate here capable of catching this class of regression.
Verified against the v9.3.4 release image: all eight bundles rebuild with zero
non-ASCII bytes and zero bare astral keys, `x2t -create-js-cache` exits 0 with
all four caches written (previously aborted leaving a 0-byte cache), and
documentserver-generate-allfonts.sh completes cleanly.
Fixes #80
Assisted-by: ClaudeCode:claude-opus-5
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Christoph Schaefer <christoph.schaefer@nextcloud.com>
ReviewTL;DR: Approve with reservations. The shipped fix is correct — I reproduced the build with the exact pinned toolchain ( Findings1. The code comment has the mechanism backwards.
2. The regression test that's supposed to catch this can never fail. 3. CI's grep gate is blind to this exact residual mode. 4. A second, untouched Terser pass exists. 5. Minor CI hardening. Checked and not problems: Suggestion before merge: correct the comment at |
… coverage gaps Review of #81 (thanks @moodyjmz) found the mechanism written into the config comment was backwards, and that both new gates were consequently unable to detect the failure they exist for. Verified each point against the real pipeline before changing anything. quote_keys is the load-bearing option, not the redundant one. Built with the pinned toolchain and only ascii_only, this pipeline emits 131 keys of the form \u{1d552}:"\\doublea" which contains no byte > 0x7F yet is still a bare astral identifier, and the no-ICU V8 rejects it exactly as it rejects the raw UTF-8 spelling. Measured against the v9.3.4 image: ascii_only alone still aborts x2t and leaves a 0-byte sdk-all.cache; ascii_only + quote_keys writes a valid 5.3 MB cache. Both options stay, with the comment now describing what actually happens. - The behavioral test called terser.minify() without `ecma`, a path the real build never takes: Terser's default is conservative and quotes astral keys regardless, so the test passed with or without quote_keys. It now pins ecma to what terser-webpack-plugin injects, and a companion test asserts the bare key *does* reappear without quote_keys, so the guard cannot go blind again. Confirmed by deleting quote_keys from the config: 4 tests now fail, including the behavioral one, where previously none did. - The CI gate greped for bytes > 0x7F and so was blind to the escaped form. Replaced with build/scripts/check-bundle-ascii.cjs, which checks both invariants (no raw non-ASCII, no bare astral key in raw or \u{...} form), reports GitHub annotations, and fails when the glob matches nothing instead of silently passing. The test suite imports its regexes so the two cannot drift apart. - deploy-assets.cjs ran a second, separate Terser pass over Native/*.js and libfont/engine/fonts_*.js -- files doctrenderer concatenates into the same script it compiles -- with neither option set and outside the gate's reach. ASCII-clean today by luck; given the same options so it stays that way. Full suite: 44/44. Rebuilt word/sdk-all.js with the final config and confirmed x2t writes a valid cache. Assisted-by: ClaudeCode:claude-opus-5 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Christoph Schaefer <christoph.schaefer@nextcloud.com>
|
Verified the follow-up commit (991547e) directly rather than taking the commit message on faith — pulled the diff, re-derived Terser's actual identifier-printing rule ( All four points from the review above are correctly addressed:
Nice catch-and-fix cycle. Approving. |
moodyjmz
left a comment
There was a problem hiding this comment.
Approving — see review comments above; the follow-up commit correctly addressed all findings.
Terser was configured with neither format.ascii_only nor format.quote_keys, so the webpack pipeline emitted the LaTeX symbol table's supplementary-plane ("astral") keys as bare identifiers -- 131 of them per bundle in word/ and slide/ sdk-all.js, alongside 43348 raw non-ASCII bytes.
doctrenderer/x2t execute these bundles on a V8 built with v8_enable_i18n_support=false (core: Common/3dParty/v8/tools/8.9/*/nc-build.sh). Without ICU its reduced Unicode tables do not classify astral characters as ID_Start, so it rejects the entire bundle with "SyntaxError: Invalid or unexpected token". ScriptCompiler::Compile then returns an empty MaybeLocal and an unguarded ToLocalChecked() aborts the process -- the "Fatal error in v8::ToLocalChecked / Empty MaybeLocal" and "Illegal instruction (core dumped)" that make Euro-Office 9.3.4 fail on first start and on every document save.
The Closure Compiler build this replaced escaped all non-ASCII to \uXXXX, so the ASCII-only invariant held implicitly. The webpack migration dropped it silently, and no gate noticed: every JS check in check-build.yml runs on Node and headless Chromium, both full-ICU, which parse these bundles happily.
format; Terser rejects a top-level quote_keys outright with "quote_keysis not a supported option".Verified against the v9.3.4 release image: all eight bundles rebuild with zero non-ASCII bytes and zero bare astral keys,
x2t -create-js-cacheexits 0 with all four caches written (previously aborted leaving a 0-byte cache), and documentserver-generate-allfonts.sh completes cleanly.Fixes #80
Assisted-by: ClaudeCode:claude-opus-5