CLDR-16765 Show un printables #4664
Conversation
…endpoint - add CodePointEscaper.NAMED_INVISIBLES set, also add unit test - add API endpoint to get escaper information
- move <> into CSS before/after for visible-mark - rewire startup sequence a little- could pre-load some things in parallel - new mechanism cldrEscaper
srl295
requested review from
macchiati,
btangmu,
conradarcturus,
pedberg-icu and
DraganBesevic
and removed request for
macchiati
tools/cldr-code/src/main/java/org/unicode/cldr/util/CodePointEscaper.java
Outdated
Show resolved
Hide resolved
tools/cldr-code/src/main/java/org/unicode/cldr/util/CodePointEscaper.java
Outdated
Show resolved
Hide resolved
- add JS unit test - refactor, split out cldrEscaperLoader.mjs to avoid vue dependency - use CodePointEscaper.FORCE_ESCAPE, delete other sets
- exclude space
- fixup broken regex output from UnicodeSet: use \u{...} form, also don't escape space
- escape U+XXXX as hex
|
@macchiati using FORCE_ESCAPE results in the regex:
|
now using FORCE_ESCAPE, minus space |
|
The vast majority of those are [[:Cn:][:Co:][:Cs:]] (unassigned, private use, surrogates). Our data will contain none of them. For completeness, CodePointEscaper has them, but if the regex is slow in JS, then you could subtract them as well as subtracting space. (As far as the name/description goes, a future enhancement of CodePointEscaper might give a short description of each of (unassigned, private use, surrogates), but for CLDR usage it isn't needed. |
It's a one time transfer and then the regex is usable globally. |
|
Looks like build stuck. Will check |
|
for a million strings it is a factor of 10x between this regex and the trivial one We don't have a million rows on a page. So I think the difference is containable. It's one-time during each page load. |
CLDR-16765
For now, only named invisibles (except SPACE U+0020!) are shown. But this mechanism could show all invisibles, substituting U+ numbers or something.
ALLOW_MANY_COMMITS=true
