Ignore degenerate identity ToUnicode ranges

[yzxcj797]

Summary

Some dompdf-generated CIDFontType2 subsets contain a ToUnicode CMap whose only mapping is the degenerate full-range identity:

<0000> <FFFF> <0000>

That assertion says every CID equals its Unicode codepoint. For a subset font whose CIDs are renumbered glyph IDs, the assertion is false and can decode some CJK characters as two Latin-1 bytes.

This change detects that exact structural shape and treats it as an absent ToUnicode map. Both CMap collection paths then use the existing Type0 fallback chain, including the embedded TrueType cmap table, which is authoritative for these fonts. Normal CMaps, non-zero bases, partial ranges, and CMaps with explicit bfchar entries are not affected.

Fixes #246.

Testing

• Added six regressions covering:
  • exact full-range identity detection;
  • treatment as absent when no fallback exists;
  • rejection of a non-zero base;
  • rejection of partial identity ranges;
  • rejection when explicit bfchar mappings exist;
  • rejection of a normal CJK ToUnicode map.
• cargo test — 993 unit, 3 CLI, 162 integration, and 2 documentation tests pass
• cargo clippy -- -D warnings — passes
• cargo fmt --all -- --check — passes
• git diff --check — passes

---

Summary by cubic

Ignores degenerate full-range identity ToUnicode CMaps and falls back to Type0/TrueType mappings to avoid mojibake in subset CIDFontType2 fonts. Previously we applied the identity mapping (<0000> <0000>) and some CJK decoded as two Latin‑1 bytes; now we treat that map as absent.

Notes for reviewers

• Triggers only when the CMap has exactly one bfrange <0000> <FFFF> <0000> and no bfchar entries; non‑zero bases, partial ranges, or any bfchar mappings are unaffected.
• Applied in both ToUnicode collection paths (build_cmap_entry_from_stream and FontCMaps): on detection we construct existing fallbacks (Type0/TrueType or simple) and still promote TrueType when it has more entries.
• Adds focused regression tests for detection and fallback behavior; no API or config changes.

^{Written for commit a15268d. Summary will update on new commits.}

[cubic-dev-ai]

2 issues found across 1 file

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/tounicode.rs">

<violation number="1" location="src/tounicode.rs:60">
P3: When a degenerate identity bfrange is detected and a fallback CMap exists, the new branch falls through to the trailing block whose debug log reads "ToUnicode CMap obj={} parse failed". The CMap did not fail to parse; it was deliberately rejected as degenerate, so the log misattributes the fallback and makes diagnostics confusing. Log the degenerate-detection reason here instead of reusing the parse-failed message.</violation>

<violation number="2" location="src/tounicode.rs:60">
P2: The degenerate detection rejects any full-range identity bfrange regardless of whether the font is actually a renumbered subset whose CIDs differ from Unicode codepoints. A producer that genuinely maps CIDs to equal Unicode codepoints (an Identity font with no embedded TrueType `cmap` and no CIDSystemInfo ordering to build a fallback) had a working extraction before this change, but is now discarded: build_cmap_entry_from_stream returns None and FontCMaps inserts no entry, dropping the text. Consider restricting the "treat as absent" behavior to the subset/renumbered case, or fall back to honoring the identity CMap when neither the TrueType `cmap` nor any other fallback is available.</violation>
</file>

<sub>Shadow auto-approve: would not auto-approve because issues were found.

Fix all with cubic | Re-trigger cubic</sub>

[cubic-dev-ai]

P2: The degenerate detection rejects any full-range identity bfrange regardless of whether the font is actually a renumbered subset whose CIDs differ from Unicode codepoints. A producer that genuinely maps CIDs to equal Unicode codepoints (an Identity font with no embedded TrueType cmap and no CIDSystemInfo ordering to build a fallback) had a working extraction before this change, but is now discarded: build_cmap_entry_from_stream returns None and FontCMaps inserts no entry, dropping the text. Consider restricting the "treat as absent" behavior to the subset/renumbered case, or fall back to honoring the identity CMap when neither the TrueType cmap nor any other fallback is available.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/tounicode.rs, line 60:

<comment>The degenerate detection rejects any full-range identity bfrange regardless of whether the font is actually a renumbered subset whose CIDs differ from Unicode codepoints. A producer that genuinely maps CIDs to equal Unicode codepoints (an Identity font with no embedded TrueType `cmap` and no CIDSystemInfo ordering to build a fallback) had a working extraction before this change, but is now discarded: build_cmap_entry_from_stream returns None and FontCMaps inserts no entry, dropping the text. Consider restricting the "treat as absent" behavior to the subset/renumbered case, or fall back to honoring the identity CMap when neither the TrueType `cmap` nor any other fallback is available.</comment>

<file context>
@@ -29,55 +29,83 @@ pub struct ToUnicodeCMap {
-            if let Some(ref fb) = fallback {
-                let fb_entries = fb.char_map.len() + fb.ranges.len();
-                if fb_entries > primary_entries {
+        if is_degenerate_identity_bfrange(&cmap) {
+            debug!(
+                "ToUnicode obj={}: full-range identity bfrange detected; treating as absent",
</file context>

[cubic-dev-ai]

P3: When a degenerate identity bfrange is detected and a fallback CMap exists, the new branch falls through to the trailing block whose debug log reads "ToUnicode CMap obj={} parse failed". The CMap did not fail to parse; it was deliberately rejected as degenerate, so the log misattributes the fallback and makes diagnostics confusing. Log the degenerate-detection reason here instead of reusing the parse-failed message.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/tounicode.rs, line 60:

<comment>When a degenerate identity bfrange is detected and a fallback CMap exists, the new branch falls through to the trailing block whose debug log reads "ToUnicode CMap obj={} parse failed". The CMap did not fail to parse; it was deliberately rejected as degenerate, so the log misattributes the fallback and makes diagnostics confusing. Log the degenerate-detection reason here instead of reusing the parse-failed message.</comment>

<file context>
@@ -29,55 +29,83 @@ pub struct ToUnicodeCMap {
-            if let Some(ref fb) = fallback {
-                let fb_entries = fb.char_map.len() + fb.ranges.len();
-                if fb_entries > primary_entries {
+        if is_degenerate_identity_bfrange(&cmap) {
+            debug!(
+                "ToUnicode obj={}: full-range identity bfrange detected; treating as absent",
</file context>