CurlShot API docs rendered with marked: async walkTokens for highlighting, TOC from the heading renderer

[SergiuPogor]

Sharing how we use marked in production, in case the async walkTokens pattern helps someone.

The docs of CurlShot, a screenshot API (URL, HTML or Markdown in; PNG, JPEG, WebP, PDF or video out), are plain Markdown files rendered server-side in Next.js with marked.

What worked well:

• Async syntax highlighting in walkTokens. With new Marked({ async: true, gfm: true, async walkTokens(token) { ... } }), every code token is highlighted (Shiki) before rendering and the HTML is stored on the token; the code renderer just returns it. No sync highlighter needed.
• Table of contents from the heading renderer. The renderer slugifies the heading text, adds a number when the same heading repeats on a page (for example "Common mistakes"), pushes h2/h3 into a TOC array and adds an anchor link. A # in the Markdown is rendered as <h2> because the page title is the only <h1>.
• Small renderer overrides instead of post-processing: external links get rel="noopener noreferrer"; images become <figure> with known width/height so the page does not jump, and an example image that does not exist yet is left out instead of rendering broken; tables are wrapped for horizontal scrolling; a paragraph that only held a figure is not wrapped in <p>.
• Custom blocks around marked. Code tabs (one sample per language) and callouts are split out of the Markdown first, and only their inner text goes through marked.

Thanks for marked. The token/renderer API made all of this a few dozen lines.