Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

font issue in typst tables with gt with strings containing numbers and letters #11683

Open
schochastics opened this issue Dec 13, 2024 · 8 comments · May be fixed by #11918
Open

font issue in typst tables with gt with strings containing numbers and letters #11683

schochastics opened this issue Dec 13, 2024 · 8 comments · May be fixed by #11918
Assignees
Labels
bug Something isn't working tables Issues with Tables including the gt integration typst
Milestone

Comments

@schochastics
Copy link

Bug description

When I render a gt table in typst that contains strings that mix letters and digits (e.g. "aaa111aaa") the font switches and the string renders weirdly.

Steps to reproduce

---
title: "Untitled"
format: typst
keep-typ: true
---

```{r}
df <- data.frame(a = c("aa111bbb", "abcdef"), b = c(1, 2))
gt::gt(df)
knitr::kable(df)
```

I get the following result:
Image

The produced typst code for the gt table looks like this:

#[
#set text(font: ("system-ui", "Segoe UI", "Roboto", "Helvetica", "Arial", "sans-serif", "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji") , size: 12pt); #table(
  columns: 2,
  align: (left,right,),
  table.header(table.cell(align: bottom + left, fill: rgb("#ffffff"))[#set text(size: 1.0em , fill: rgb("#333333")); a], table.cell(align: bottom + right, fill: rgb("#ffffff"))[#set text(size: 1.0em , fill: rgb("#333333")); b],),
  table.hline(),
  table.cell(align: horizon + left, stroke: (top: (paint: rgb("#d3d3d3"), thickness: 0.75pt)))[aa111bbb], table.cell(align: horizon + right, stroke: (top: (paint: rgb("#d3d3d3"), thickness: 0.75pt)))[1],
  table.cell(align: horizon + left, stroke: (top: (paint: rgb("#d3d3d3"), thickness: 0.75pt)))[abcdef], table.cell(align: horizon + right, stroke: (top: (paint: rgb("#d3d3d3"), thickness: 0.75pt)))[2],
)
]

Expected behavior

When I remove

#set text(font: ("system-ui", "Segoe UI", "Roboto", "Helvetica", "Arial", "sans-serif", "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji") , size: 12pt);

from the intermediate typ file, I get the correct table when I compile that file

Image

after playing around a bit, setting the font explicitly via gt::opt_table_font() also removes the weird behavio.

Actual behavior

No response

Your environment

Positron Version 2024.12.0 build 80

sessionInfo()

R version 4.4.2 (2024-10-31)
Platform: x86_64-pc-linux-gnu
Running under: Ubuntu 22.04.5 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so;  LAPACK version 3.10.0

locale:
 [1] LC_CTYPE=en_GB.UTF-8       LC_NUMERIC=C               LC_TIME=en_GB.UTF-8       
 [4] LC_COLLATE=en_GB.UTF-8     LC_MONETARY=en_GB.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=de_DE.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
[10] LC_TELEPHONE=C             LC_MEASUREMENT=de_DE.UTF-8 LC_IDENTIFICATION=C       

time zone: Europe/Berlin
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] gt_0.11.1       readxl_1.4.3    lubridate_1.9.3 forcats_1.0.0   stringr_1.5.1   dplyr_1.1.4    
 [7] purrr_1.0.2     readr_2.1.5     tidyr_1.3.1     tibble_3.2.1    ggplot2_3.5.1   tidyverse_2.0.0

loaded via a namespace (and not attached):
 [1] styler_1.10.3     sass_0.4.9        utf8_1.2.4        generics_0.1.3    xml2_1.3.6       
 [6] stringi_1.8.4     hms_1.1.3         digest_0.6.37     magrittr_2.0.3    evaluate_1.0.1   
[11] grid_4.4.2        timechange_0.3.0  fastmap_1.2.0     R.oo_1.27.0       cellranger_1.1.0 
[16] R.cache_0.16.0    R.utils_2.12.3    fansi_1.0.6       scales_1.3.0      cli_3.6.3        
[21] rlang_1.1.4       R.methodsS3_1.8.2 munsell_0.5.1     commonmark_1.9.2  withr_3.0.2      
[26] tools_4.4.2       tzdb_0.4.0        colorspace_2.1-1  vctrs_0.6.5       R6_2.5.1         
[31] lifecycle_1.0.4   snakecase_0.11.1  janitor_2.2.0     pkgconfig_2.0.3   pillar_1.9.0     
[36] gtable_0.3.6      data.table_1.16.2 glue_1.8.0        xfun_0.49         tidyselect_1.2.1 
[41] knitr_1.49        htmltools_0.5.8.1 compiler_4.4.2    markdown_1.13

Quarto check output

Quarto 1.6.37
[✓] Checking environment information...
      Quarto cache location: /home/david/.cache/quarto
[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.4.0: OK
      Dart Sass version 1.70.0: OK
      Deno version 1.46.3: OK
      Typst version 0.11.0: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.6.37
      Path: /opt/quarto/bin

[✓] Checking tools....................OK
      TinyTeX: v2023.10
      Chromium: (not installed)

[✓] Checking LaTeX....................OK
      Using: TinyTex
      Path: /home/david/.TinyTeX/bin/x86_64-linux
      Version: 2023

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.10.12
      Path: /bin/python3
      Jupyter: (None)

      Jupyter is not available in this Python installation.
      Install with python3 -m pip install jupyter

[✓] Checking R installation...........OK
      Version: 4.4.2
      Path: /usr/lib/R
      LibPaths:
        - /home/david/R/x86_64-pc-linux-gnu-library/4.4
        - /usr/local/lib/R/site-library
        - /usr/lib/R/site-library
        - /usr/lib/R/library
      knitr: 1.49
      rmarkdown: 2.29

[✓] Checking Knitr engine render......OK
@schochastics schochastics added the bug Something isn't working label Dec 13, 2024
@mcanouil
Copy link
Collaborator

mcanouil commented Dec 13, 2024

I can't reproduce the issue on macOS at least.
Image

Note that since gt cannot emit Typst code, here it emits HTML table which is parsed by Quarto into AST which is then converted to the proper format, in this case Typst.
gt is the one setting the fonts you see in the Typst output, you can see this with gt::as_raw_html(gt::gt(df))) in R. (also why when you used gt::opt_table_font(), the behaviour changed)

I believe the font issue is more on your system which don't have the font.
I don't know if "system-ui" is recognised everywhere.

Anyhow, I don't see any Quarto issue here.

@schochastics
Copy link
Author

Thanks. I was also not sure where to post this issue but just wanted to be sure that this is not on the quarto side. I will check what my system is doing.

@cscheid cscheid reopened this Dec 16, 2024
@cscheid
Copy link
Collaborator

cscheid commented Dec 16, 2024

Let's keep this open at least for now - the behavior does feel unintuitive to me, so I'd like to understand it better.

@cscheid cscheid added typst tables Issues with Tables including the gt integration labels Dec 16, 2024
@gordonwoodhull gordonwoodhull self-assigned this Dec 16, 2024
@gordonwoodhull
Copy link
Contributor

gordonwoodhull commented Dec 17, 2024

Repro

Okay, I have a simple repro in Typst on macOS.

#[
#set text(font: ("Apple Color Emoji") , size: 12pt); #table(
  columns: 2,
  align: (left,right,),
  table.header(table.cell(align: bottom + left, fill: rgb("#ffffff"))[#set text(size: 1.0em , fill: rgb("#333333")); a], table.cell(align: bottom + right, fill: rgb("#ffffff"))[#set text(size: 1.0em , fill: rgb("#333333")); b],),
  table.hline(),
  table.cell(align: horizon + left, stroke: (top: (paint: rgb("#d3d3d3"), thickness: 0.75pt)))[aa111bbb], table.cell(align: horizon + right, stroke: (top: (paint: rgb("#d3d3d3"), thickness: 0.75pt)))[1],
  table.cell(align: horizon + left, stroke: (top: (paint: rgb("#d3d3d3"), thickness: 0.75pt)))[abcdef], table.cell(align: horizon + right, stroke: (top: (paint: rgb("#d3d3d3"), thickness: 0.75pt)))[2],
)
]

Root cause

The Apple Color Emoji font, installed on Macs, supports numbers but not letters. This is probably true of other emoji fonts.

If the first available font in the list is an emoji font, then the numbers will be bigger than the rest of the text. because emoji fonts have large glyph sizes.

Image

The rest of the text will have a fallback font provided by Typst, unless disabled with

#set text(fallback: false)

If my theory is correct, then none of the fonts

"Segoe UI", "Roboto", "Helvetica", "Arial"

are installed on @schochastics's machine, or they are not available to Typst. But at least one of the emoji fonts

"Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"

is installed and available.

(quarto typst fonts will list fonts available to Typst.)

Font stacks

If there is a Quarto issue here, it is we translate CSS font lists into Typst font lists verbatim.

But "font stacks" like system-ui and sans-serif will not be recognized by Typst and may give a false sense of security.

Conceivably, we could substitute lists of appropriate fonts for "font stacks", at the mild maintenance burden of having to maintain such lists.

Our current behavior is surprising because the intent of the font list

"system-ui", "Segoe UI", "Roboto", "Helvetica", "Arial", "sans-serif", "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"

is clearly "choose a nice looking or at least sans serif font for text, and one of these emoji fonts for emoji". Not "use an emoji font for the numbers and a fallback font for the text".

For now I can add a note about this to the advanced docs on troubleshooting Typst fonts tracked in #11278. Similar to that issue, we again see that the letters have an unexpected serif font (the Typst default font).

@gordonwoodhull gordonwoodhull added this to the v1.7 milestone Dec 31, 2024
gordonwoodhull added a commit to quarto-dev/quarto-web that referenced this issue Jan 2, 2025
gordonwoodhull added a commit to quarto-dev/quarto-web that referenced this issue Jan 6, 2025
github-actions bot pushed a commit to quarto-dev/quarto-web that referenced this issue Jan 6, 2025
@gordonwoodhull
Copy link
Contributor

gordonwoodhull commented Jan 6, 2025

Correction: it's called

  • "generic font family" when a browser implements it
  • "font stack" when implemented as a library like system-ui No, system-ui is also a generic font family.

Comprehensive and helpful blog post on the subject:

https://blog.jim-nielsen.com/2020/system-fonts-on-the-web/

Corresponding Typst issue: typst/typst#3342

Sounds like they don't want to implement it because it hurts reproducibility, but fonts always introduce a dependency external to the document. (There is even an OS dependency on the web, because you want to choose a system font that the user has.)

I'm tempted just to replace generic font families and font stacks with names of fonts that Typst ships, if possible.

gordonwoodhull added a commit to quarto-dev/quarto-web that referenced this issue Jan 6, 2025
fixes quarto-dev/quarto-cli#11083
fixes quarto-dev/quarto-cli#11278
addresses quarto-dev/quarto-cli#11683
addresses quarto-dev/quarto-cli#11616 (reply in thread)

just a draft - some of this stuff might belong better elsewhere

(cherry picked from commit 1f34e20)
@cscheid
Copy link
Collaborator

cscheid commented Jan 6, 2025

I'm tempted just to replace generic font families and font stacks with names of fonts that Typst ships, if possible.

I think that's a great idea. Alternatively, we can ship a few chosen open-source fonts with Quarto itself, so that these are more clearly "Quarto's defaults" (I happen to not like the default typefaces from Typst, but that's a super personal thing).

@gordonwoodhull
Copy link
Contributor

I like the idea of shipping some fonts.

From that blog post, the generic font families according to CSS spec:

    serif *
    sans-serif *
    monospace *
    system-ui
    cursive
    fantasy
    emoji
    math
    fangsong
    ui-serif †
    ui-sans-serif †
    ui-monospace †
    ui-rounded †

I count about 8 or 9 distinct, since we don't care about ui-, the OS-dependent thing.

Some will have sufficient defaults shipped with Typst, so we don't need to choose all of them.

@gordonwoodhull
Copy link
Contributor

gordonwoodhull commented Jan 8, 2025

Okay, Typst ships two serif fonts and a monospace sans-serif font, but no variable-width sans-serif font.

So we need to ship at least one font in order to fix the bug.

Libertinus Serif, New Computer Modern, New Computer Modern Math, and DejaVu Sans Mono
https://typst.app/docs/reference/text/text/#parameters-font

This is going to be really easy, since we just have to pass Typst another another --font-path, and define the mapping.

Is there anywhere to put static configuration like the font mapping? (i.e. that gets checked in and distributed with Quarto.) No practical difference from defining it in Lua, just seems icky to do that.

Making them default

In order for Quarto's Typst output to use default fonts, we would have brand defaults like

brand:
  typography:
    base: serif
    headings: sans-serif
    monospace: monospace

(or base: sans-serif, whatever)

Then these would get picked up by the Typst CSS layer, which is the main topic of this issue.

Currently brand doesn't have default font families, although there is some ugly bleed of Linux Libertine / Libertinus Serif in Typst due to the way parameters to functions can't be left unset.

gordonwoodhull added a commit that referenced this issue Jan 22, 2025
i can't figure out a way to automate this test
but we should have it for visual testing
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working tables Issues with Tables including the gt integration typst
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants