Terser emits bare astral object keys in sdk-all.js; doctrenderer's no-ICU V8 rejects them and x2t aborts (9.3.4 regression)

[chrip]

Terser emits bare astral object keys in sdk-all.js; doctrenderer's no-ICU V8 rejects them and x2t aborts (9.3.4 regression)

Target repo: Euro-Office/sdkjs
Labels (suggested): bug, regression, build, priority:critical

---

Summary

Since the Grunt → webpack/Terser migration (#65, shipped in Euro-Office 9.3.4), the
built sdk-all.js bundles contain 131 unquoted supplementary-plane ("astral")
object keys, e.g. 𝕒:"\\doublea" in the LaTeX symbol table. Terser treats those
characters as valid identifiers and drops the quotes.

doctrenderer/x2t embed V8 8.9 built with v8_enable_i18n_support=false.
Without ICU, V8's reduced Unicode tables do not classify astral characters as
ID_Start, so parsing the bundle fails with SyntaxError: Invalid or unexpected token. ScriptCompiler::Compile then returns an empty MaybeLocal, and an
unguarded .ToLocalChecked() turns that into a CHECK failure →
__builtin_trap() → Illegal instruction (core dumped).

The previous Closure Compiler build emitted pure ASCII (everything escaped to
\uXXXX), so this could never happen. The 9.3.2 bundle contains zero non-ASCII
bytes; the 9.3.4 bundle contains 43,348.

User-facing report: Euro-Office/DocumentServer#339 and
https://help.nextcloud.com/t/euro-office-upgrade-to-9-3-4-from-9-3-3-failed/248589

⚠️ Euro-Office/DocumentServer#339 attributes the crash to Intel MPX. That is
incorrect — see "Ruled out" below. The Illegal instruction is V8 trapping
itself, not an unsupported CPU instruction.

Impact

• Euro-Office 9.3.4 is unusable: .deb postinst, standalone image and
  cluster-docs all fail during documentserver-generate-allfonts.sh, and every
  document save/convert fails (callback status 7).
• Affects amd64 and arm64 alike. Downgrading to 9.3.3 resolves it.
• Affected bundles: word/sdk-all.js and slide/sdk-all.js (131 bare astral keys,
  1,256 astral characters each).

Root cause chain

1. Terser config in build/webpack.sdk.factory.mjs sets neither
   format.ascii_only nor quote_keys. Defaults are raw UTF-8 output and
   identifier-style (unquoted) object keys.
2. Output therefore contains 𝕒:"\\doublea", 𝔸:"\\doubleA", … while BMP
   characters stay quoted ("∙":"\\bullet").
3. V8 is built without ICU:
   core/Common/3dParty/v8/tools/8.9/x64-linux-dynamic/nc-build.sh#L329
   (arm64-linux-dynamic is identical).
4. doctrenderer concatenates sdk-all-min.js + fonts_*.js + sdk-all.js and
   compiles the result as one script:
   core/DesktopEditor/doctrenderer/editors.cpp#L69-L135
5. The compile fails and is not checked:
   core/DesktopEditor/doctrenderer/js_internal/v8/v8_base.cpp#L95
   — the only unguarded ToLocalChecked() on that path; its siblings at L61, L83
   and L112 all test IsEmpty() first.

Minimal reproduction

Take the shipped 9.3.4 word/sdk-all.js, replace every non-ASCII byte with x
(known-good, identical byte length), and append one line:

var __k={"𝕒":1};   // astral char inside a string literal → compiles fine
var __k={𝕒:1};     // same char as a bare identifier key  → V8 aborts

Oracle — inside ghcr.io/euro-office/documentserver:v9.3.4, a non-zero
sdk-all.cache means the script compiled:

DIR=/var/www/euro-office/documentserver
export LD_LIBRARY_PATH=$DIR/server/FileConverter/bin:$LD_LIBRARY_PATH
rm -f $DIR/sdkjs/*/sdk-all.cache
cd $DIR/server/FileConverter/bin && ./x2t -create-js-cache
stat -c %s $DIR/sdkjs/word/sdk-all.cache

Evidence

#	Variant	word/sdk-all.cache
1	stock 9.3.4 word bundle	0 — abort
2	9.3.2 word bundle, same 9.3.4 x2t binary	6,752,720 — OK
3	9.3.4 sdk-all-min.js + 9.3.2 sdk-all.js	6,752,720 — OK
4	9.3.2 sdk-all-min.js + 9.3.4 sdk-all.js	0 — abort
5	9.3.4 bundle, all non-ASCII → x, identical byte length	5,331,872 — OK
6	restore 25 / 50 / 75 / 87 / 94 % of non-ASCII bytes	all OK
7	restore 99 %	0 — abort
8	restore only the math-table region (2,168 bytes)	0 — abort
9	ASCII bundle + var __k={"𝕒":1};	5,331,952 — OK
10	ASCII bundle + var __k={𝕒:1};	0 — abort

Rows 3/4 isolate the fault to the JS, not the binary. Row 5 rules out size.
Rows 9/10 are the 15-byte minimal case.

node --check passes on every variant above under Node 15 (V8 8.6), Node 16
(V8 9.4) and Node 22 (V8 12.4) — all ship full ICU. No Node- or Chromium-based
test can detect this.

Ruled out

• Intel MPX (as claimed in Cannot save documents: callback always returns status code 7 (force-save error) due to Node.js/V8 SIGILL (MPX) on cluster-docs v9.3.4_amd64 DocumentServer#339) — reproduced on
  arm64, where MPX does not exist.
• ES-version skew / dropped Babel ES5 downlevel — the bundles parse on V8 8.6,
  older than the embedded 8.9.
• Bundle bloat — 9.3.2's sdk-all.js is larger (19.7 MB vs 18.4 MB) and works.
• Parser stack depth — both parse at --stack-size=600.
• One-byte vs two-byte string representation — a single astral or Cyrillic
  character inside a string literal compiles fine.
• Invalid UTF-8 / lexical corruption — the file is valid UTF-8.
• Leaked @@license-banner@@ sentinel — it sits inside /* */.
• Stale code cache — fresh installs fail too; the 0-byte cache is a consequence.

Proposed fix (this repo)

In build/webpack.sdk.factory.mjs, terserOptions:

format: {
    comments: /@@license-banner@@/,
    ascii_only: true,     // escape all non-ASCII as \uXXXX (restores Closure's invariant)
},
quote_keys: true,         // force "𝕒": instead of 𝕒:

Either option alone is likely sufficient; together they are equivalent to the
configurations proven to compile (rows 5 and 9).

Regression test: assert the built bundles contain no byte > 0x7F. Cheap,
exact, and encodes the invariant the Closure build used to provide implicitly.

Follow-ups in other repos

• Euro-Office/core — defensive hardening, worth its own issue:
  • guard the ToLocalChecked() at v8_base.cpp#L95 and surface the TryCatch
    message; a JS syntax error must not abort the process;
  • do not create sdk-all.cache until CreateCodeCache() succeeds — an abort
    currently leaves a 0-byte cache that poisons every later run;
  • evaluate v8_enable_i18n_support=true (separate trade-off: binary size, build
    time). The fix above does not depend on it.
• CI gap — this repo's check-build.yml validates bundles with Node 20 and
  headless Chromium, both full-ICU, so it structurally cannot catch this class of
  bug. The bundle needs to be parsed by the actual embedded V8. The three
  doctrenderer suites tracked in Run existing test cases core#93 are exactly that gate;
  Enable osign gtest suite under CTest core#134 clears the last currently-unblocked suite.

Related

• Cannot save documents: callback always returns status code 7 (force-save error) due to Node.js/V8 SIGILL (MPX) on cluster-docs v9.3.4_amd64 DocumentServer#339 — user-facing report (title needs correcting)
• build(sdkjs): fix webpack migration bundle bloat, CI, and coverage gaps #65 — the webpack migration that introduced this
• webpack watch mode intermittently reads sdkjs source mid-write, breaking the concatenated bundle and increasing build times #78 — separate webpack-migration bundle-integrity bug
• chore(deps): update dependency terser to v5.51.2 #73 — open Terser bump; whatever lands here must keep ascii_only/quote_keys