Skip to content

Commit d28a60b

Browse files
committed
fix(content): track content-specific revisions
Written by an AI agent on behalf of @atinux; not yet human-reviewed.
1 parent 9be95bf commit d28a60b

11 files changed

Lines changed: 156 additions & 64 deletions

File tree

docs/cold-page-request.md

Lines changed: 18 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -17,30 +17,30 @@ sequenceDiagram
1717
1818
SSR->>ContentRoute: $fetch (navigation)
1919
ContentRoute->>Content: getProdContent()
20-
Content->>Refs: resolveSha(targetBranch)
20+
Content->>Refs: resolveContentSha(targetBranch, contentDir)
2121
alt cache hit (within 60s TTL)
22-
Refs-->>Content: cached sha
22+
Refs-->>Content: cached content sha
2323
else cache miss
24-
Refs->>GH: commits/<branch>
25-
GH-->>Refs: sha
26-
Refs-->>Content: sha
24+
Refs->>GH: commits?sha=<branch>&path=<contentDir>
25+
GH-->>Refs: latest content sha
26+
Refs-->>Content: content sha
2727
end
28-
Content->>Content: rebuild if sha advanced
29-
Content->>GH: init partial (~36 files, at <sha>)
28+
Content->>Content: rebuild if content sha advanced
29+
Content->>GH: init partial (~36 files, at <content-sha>)
3030
ContentRoute-->>SSR: nav tree
3131
3232
SSR->>ContentRoute: $fetch (page)
3333
ContentRoute->>Content: getProdContent() (same sha → no rebuild)
34-
Content->>GH: fetch + parse 1 page (at <sha>)
34+
Content->>GH: fetch + parse 1 page (at <content-sha>)
3535
ContentRoute-->>SSR: parsed page
3636
3737
SSR-->>Edge: HTML
3838
Edge-->>Browser: HTML (cached for next visitor)
3939
```
4040

41-
**Cost:** one shared-cache lookup for the branch tip + the instance builds its index
42-
from GitHub once per head, then one page parse. All reads pinned to the immutable
43-
`<sha>`.
41+
**Cost:** one shared-cache lookup for the latest commit touching the content directory + the
42+
instance builds its index from GitHub once per content revision, then one page parse. All reads
43+
are pinned to the immutable `<content-sha>`. Code-only commits do not rebuild the content instance.
4444

4545
The ref cache is shared across *instances*, so GitHub is hit once per 60s TTL window
4646
rather than once per cold start. It is **not** shared across regions — Vercel's
@@ -49,14 +49,18 @@ Runtime Cache is regional (see the note on `refCacheDriver()` in
4949
This project runs single-region, which is what makes that distinction academic today.
5050

5151
A ref that doesn't resolve is cached too, for the same window, but **only** when the
52-
caller asks for it (`resolveSha(ref, { cacheMisses: true })`) — the public
52+
caller asks for it (`resolveContentSha(ref, contentDir, { cacheMisses: true })`) — the public
5353
`/tree/:branch` route does, so a nonexistent branch can't be replayed into one
5454
GitHub API call per request. The production branch above deliberately does not:
5555
GitHub answers 404 when a token loses access to a private repo, and caching that
5656
would turn an expired token into a site-wide outage for the window rather than one
5757
failed request.
5858

59-
**On a content push**, `server/api/revalidate.post.ts` writes the new SHA
60-
directly into the same shared ref cache (`cacheSha()`) before fanning out ISR
59+
**On a content push**, `server/api/revalidate.post.ts` forces a fresh `resolveContentSha()` lookup,
60+
which writes the latest content SHA into the same shared ref cache before fanning out ISR
6161
purges for the affected pages, so a freshly-purged page's next render already
6262
sees the new SHA instead of waiting out the 60s TTL.
63+
64+
Parsed manifests and bodies live under a deployment-revision + content-SHA namespace. Vercel
65+
Runtime Cache persists across deployments within an environment, so the deployment component keeps
66+
new parser or plugin code from restoring artifacts produced by an older deployment.

playground/content/2.concepts/1.architecture.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,9 +8,9 @@ description: The serving modes and caching tiers behind the layer.
88
| Mode | URL | Content |
99
| --- | --- | --- |
1010
| prod | `/getting-started/introduction` | pinned production SHA |
11-
| tree | `/tree/main/...` | branch tip preview |
11+
| tree | `/tree/main/...` | latest content commit on the branch |
1212
| blob | `/blob/<sha>/...` | immutable commit preview |
1313

1414
## Caching
1515

16-
Two tiers: ISR-cached page HTML at the edge, and a per-SHA runtime cache for parsed Markdown bodies.
16+
Two tiers: ISR-cached page HTML at the edge, and a per-deployment, per-content-SHA runtime cache for parsed Markdown bodies.

playground/skills/preview-versions/SKILL.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,12 +13,12 @@ Content is served at request time. Production is pinned to a commit SHA; any bra
1313
| Mode | URL | Content |
1414
| --- | --- | --- |
1515
| prod | `/getting-started/introduction` | pinned production SHA |
16-
| tree | `/tree/main/getting-started/introduction` | branch tip |
16+
| tree | `/tree/main/getting-started/introduction` | latest commit touching the content directory |
1717
| blob | `/blob/<sha>/getting-started/introduction` | immutable commit |
1818

1919
Raw markdown mirrors exist at `/raw/**` (and under `/tree/.../raw/` / `/blob/.../raw/`).
2020

21-
Two cache tiers: ISR-cached page HTML at the edge, and a per-SHA runtime cache for parsed Markdown bodies. A GitHub push to the production branch hits `/api/revalidate` and purges ISR.
21+
Two cache tiers: ISR-cached page HTML at the edge, and a per-deployment, per-content-SHA runtime cache for parsed Markdown bodies. A GitHub push to the production branch hits `/api/revalidate` and purges ISR.
2222

2323
Keyboard shortcut `g` `h` toggles the version-history panel on a docs page.
2424

server/api/content/tree/[branch]/[...path].get.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ export default defineEventHandler(async (event) => {
1313
}
1414

1515
// `cacheMisses`: the ref comes from the URL, so a miss must not re-cost a GitHub call each time.
16-
const sha = await resolveSha(branch, { cacheMisses: true })
16+
const sha = await resolveContentSha(branch, useRuntimeConfig(event).docs.contentDir, { cacheMisses: true })
1717
const content = await getPreviewContent(sha, `/api/content/tree/${encodeURIComponent(branch)}`)
1818

1919
return await content.handler(toWebRequest(event))

server/api/revalidate.post.ts

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@ export default defineEventHandler(async (event) => {
4343

4444
const payload = JSON.parse(raw) as GitHubPushPayload
4545
const branch = targetBranch()
46+
const contentDir = docs.contentDir
4647
const expectedRef = `refs/heads/${branch}`
4748

4849
if (payload.ref !== expectedRef) {
@@ -103,10 +104,11 @@ export default defineEventHandler(async (event) => {
103104
throw createError({ statusCode: 400, statusMessage: 'Missing head commit SHA' })
104105
}
105106

106-
// Ahead of the purge fan-out, so a freshly-purged page can't re-render against the stale SHA.
107-
await cacheSha(branch, headSha)
107+
// Bypass the short ref cache and write the canonical path-filtered revision before the purge fan-out,
108+
// so a freshly-purged page cannot re-render against a stale or payload-order-dependent content SHA.
109+
const contentSha = await resolveContentSha(branch, contentDir, { refresh: true })
108110

109-
console.log(`[content] revalidate push headSha=${headSha ?? '<none>'}`)
111+
console.log(`[content] revalidate push headSha=${headSha} contentSha=${contentSha}`)
110112

111113
const requestId = getHeader(event, 'x-vercel-id') ?? getHeader(event, 'x-request-id') ?? 'local'
112114
const tag = `[revalidate:${requestId}]`
@@ -127,7 +129,9 @@ export default defineEventHandler(async (event) => {
127129
}
128130
}
129131

130-
const headContent = await createSourceContent(headSha, { cache: { driver: cacheDriver(headSha) } })
132+
// The head snapshot has the same content directory as `contentSha`; populate the namespace that
133+
// production instances will read even when later commits in this push only changed code.
134+
const headContent = await createSourceContent(headSha, { cache: { driver: cacheDriver(contentSha) } })
131135
await headContent.init()
132136
const newItems = headContent.manifest.items
133137

server/utils/cache.ts

Lines changed: 17 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -5,19 +5,28 @@ import vercelRuntimeCache from 'unstorage/drivers/vercel-runtime-cache'
55
/** SHA-pinned content is immutable, so it can be cached for a long time. */
66
const TTL = 60 * 60 * 24
77

8-
/** Branch tips move, so the ref pointer cache uses a short TTL. */
8+
/** Content refs move with their branches, so the pointer cache uses a short TTL. */
99
const REF_TTL = 60
1010

1111
/** Whether the Vercel Runtime Cache is available (i.e. running on Vercel). */
1212
function cacheAvailable(): boolean {
1313
return !import.meta.dev && Boolean(process.env.VERCEL)
1414
}
1515

16-
/** Per-SHA driver backing comark's content cache (parsed bodies). */
16+
/**
17+
* Runtime Cache persists across deployments within one Vercel environment. Include the deployment's
18+
* code revision so a parser/plugin change cannot restore artifacts produced by older code.
19+
*/
20+
export function contentCacheBase(sha: string): string {
21+
const deploymentRef = process.env.VERCEL_GIT_COMMIT_SHA || process.env.VERCEL_DEPLOYMENT_ID
22+
return deploymentRef ? `content:${deploymentRef}:${sha}` : `content:${sha}`
23+
}
24+
25+
/** Per-deployment, per-content-SHA driver backing comark's manifest and parsed bodies. */
1726
export function cacheDriver(sha: string): Driver {
1827
if (!cacheAvailable()) return memoryDriver()
1928
return vercelRuntimeCache({
20-
base: `content:${sha}`,
29+
base: contentCacheBase(sha),
2130
ttl: TTL,
2231
})
2332
}
@@ -32,13 +41,14 @@ export function shaCacheStorage(sha: string): Storage {
3241
}
3342

3443
/**
35-
* Shared driver backing the branch → commit SHA pointer (`resolveSha`/`cacheSha` in `github.ts`),
36-
* in its own namespace so every instance reads one pointer instead of keeping its own timer.
44+
* Shared driver backing the branch + content directory → content commit pointer
45+
* (`resolveContentSha` in `github.ts`), in its own namespace so every instance reads one pointer
46+
* instead of keeping its own timer.
3747
*
3848
* Vercel Runtime Cache is **regional**, not global (https://vercel.com/docs/caching/runtime-cache):
3949
* this assumes Functions run in a single region (no `regions` in `vercel.json`/`nuxt.config.ts`).
40-
* Multi-region would confine `cacheSha()`'s write-through to the webhook's region — others self-heal
41-
* on TTL, so reach for a globally replicated store (e.g. Edge Config) only if that day comes.
50+
* Multi-region would confine the webhook's forced refresh to its region — others self-heal on TTL,
51+
* so reach for a globally replicated store (e.g. Edge Config) only if that day comes.
4252
*/
4353
export function refCacheDriver(): Driver {
4454
if (!cacheAvailable()) return memoryDriver()

server/utils/content.ts

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -86,18 +86,19 @@ export function targetBranch(): string {
8686
let headRef: string | undefined
8787

8888
export function getHeadRef(): string {
89-
headRef ??= process.env.VERCEL_GIT_COMMIT_SHA || targetBranch()
89+
headRef ??= targetBranch()
9090
return headRef
9191
}
9292

9393
/**
9494
* Shared content instance for the lifetime of this server instance, pinned to `headRef`. In production every
95-
* call resolves the tip of `targetBranch()` via `resolveSha()` — a shared, short-TTL cache, not a
96-
* per-instance timer — and rebuilds when it advances. Previews stay pinned to their build commit.
95+
* call resolves the latest commit touching the content directory via `resolveContentSha()` — a shared,
96+
* short-TTL cache, not a per-instance timer — and rebuilds when that advances. Previews stay pinned.
9797
*/
9898
export async function getProdContent(): Promise<ComarkContent> {
9999
if (['production', 'preview'].includes(process.env.VERCEL_ENV || '')) {
100-
const sha = await resolveSha(targetBranch())
100+
const { contentDir } = useRuntimeConfig().docs
101+
const sha = await resolveContentSha(targetBranch(), contentDir)
101102
if (sha !== getHeadRef()) {
102103
console.log(`[content] head ${getHeadRef()} -> ${sha}`)
103104
headRef = sha

server/utils/github.ts

Lines changed: 37 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -39,62 +39,74 @@ export function githubToken(): string | undefined {
3939
return docs.githubToken || process.env.GITHUB_TOKEN || undefined
4040
}
4141

42-
// Branch commit SHA pointer, shared across every instance so only one pays for the GitHub API
43-
// call per TTL window. See `refCacheDriver()` for the single-region assumption this relies on.
42+
// Branch + content directory → content commit SHA pointer, shared across every instance so only one
43+
// pays for the GitHub API call per TTL window. See `refCacheDriver()` for the single-region assumption.
4444
const refStorage = createStorage({ driver: refCacheDriver() })
45-
const refKey = (branch: string) => `branch:${branch}`
45+
const normalizeContentDir = (contentDir: string) => contentDir.replace(/^\/+|\/+$/g, '')
46+
const refKey = (branch: string, contentDir: string) =>
47+
`branch:${encodeURIComponent(branch)}:path:${encodeURIComponent(normalizeContentDir(contentDir))}`
4648

47-
/** Sentinel for "this ref doesn't resolve" — see the negative caching in `resolveSha`. */
49+
/** Sentinel for "this ref doesn't resolve" — see the negative caching in `resolveContentSha`. */
4850
const UNRESOLVED = '\0unresolved'
4951

5052
/**
51-
* Resolve a branch name to its tip commit SHA.
53+
* Resolve a branch to the latest commit that touched `contentDir`.
5254
*
5355
* Callers serving attacker-supplied refs must set `cacheMisses`: `/tree/:branch` is public, so with
5456
* no negative entry every missing-branch request costs an authenticated GitHub call — an
5557
* unauthenticated way to burn the token's rate limit. Off by default because the production branch
5658
* must not be negative-cached: GitHub answers 404, not 403, for a repo a token can't see, so a
5759
* rotated token looks like a missing ref and caching that downs the site for the TTL.
5860
*/
59-
export async function resolveSha(branch: string, opts: { cacheMisses?: boolean } = {}): Promise<string> {
61+
export async function resolveContentSha(
62+
branch: string,
63+
contentDir: string,
64+
opts: { cacheMisses?: boolean; refresh?: boolean } = {}
65+
): Promise<string> {
6066
if (import.meta.dev) return branch
6167

62-
const cached = await refStorage.getItem<string>(refKey(branch))
63-
if (cached === UNRESOLVED) {
64-
throw createError({ statusCode: 404, statusMessage: `Ref not found: ${branch}` })
68+
const key = refKey(branch, contentDir)
69+
if (!opts.refresh) {
70+
const cached = await refStorage.getItem<string>(key)
71+
if (cached === UNRESOLVED) {
72+
throw createError({ statusCode: 404, statusMessage: `Ref not found: ${branch}` })
73+
}
74+
if (cached) return cached
6575
}
66-
if (cached) return cached
6776

6877
const token = githubToken()
69-
let commit: { sha: string }
78+
let commits: Array<{ sha: string }>
7079
try {
71-
commit = await $fetch<{ sha: string }>(`https://api.github.com/repos/${githubRepo()}/commits/${branch}`, {
80+
commits = await $fetch<Array<{ sha: string }>>(`https://api.github.com/repos/${githubRepo()}/commits`, {
7281
headers: {
7382
Accept: 'application/vnd.github+json',
7483
...(token ? { Authorization: `Bearer ${token}` } : {}),
7584
},
85+
query: {
86+
sha: branch,
87+
path: normalizeContentDir(contentDir),
88+
per_page: 1,
89+
},
7690
})
77-
} catch (error: any) {
91+
} catch (error: unknown) {
7892
// Only a definitive 404 is cacheable; a 5xx, rate-limit 403 or network blip stays retryable.
79-
const status = error?.statusCode ?? error?.response?.status
93+
const failure = error as { statusCode?: number; response?: { status?: number } }
94+
const status = failure.statusCode ?? failure.response?.status
8095
if (status === 404) {
81-
if (opts.cacheMisses) await refStorage.setItem(refKey(branch), UNRESOLVED)
96+
if (opts.cacheMisses) await refStorage.setItem(key, UNRESOLVED)
8297
throw createError({ statusCode: 404, statusMessage: `Ref not found: ${branch}` })
8398
}
8499
throw error
85100
}
86101

87-
await refStorage.setItem(refKey(branch), commit.sha)
88-
return commit.sha
89-
}
102+
const sha = commits[0]?.sha
103+
if (!sha) {
104+
if (opts.cacheMisses) await refStorage.setItem(key, UNRESOLVED)
105+
throw createError({ statusCode: 404, statusMessage: `Content not found at ref: ${branch}` })
106+
}
90107

91-
/**
92-
* Write-through, so the revalidate webhook needn't wait for the next `resolveSha` TTL window — this
93-
* stops freshly-purged ISR pages re-rendering against a stale SHA. Reaches only the region running
94-
* it (see `refCacheDriver()`); other regions self-heal via TTL.
95-
*/
96-
export async function cacheSha(branch: string, sha: string): Promise<void> {
97-
await refStorage.setItem(refKey(branch), sha)
108+
await refStorage.setItem(key, sha)
109+
return sha
98110
}
99111

100112
export interface PageCommit {

server/utils/search.ts

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -9,11 +9,11 @@ interface SearchSection {
99
}
1010

1111
// Building the index parses every document — the most expensive read in the app. The built sections
12-
// are persisted in `content.cache`, whose driver is namespaced per SHA (`content:${sha}`), so the
13-
// index survives cold starts, is shared across lambda instances in the region, and a stale index is
14-
// unreachable: a new head or preview SHA reads from a fresh namespace. Colon-free so it can't collide
15-
// with `<source>:<path>` content keys, the `manifest` key, or the shared `gh:` namespace, and the SWR
16-
// fallback in `cache.get` (`key.split(':')`) can't map it to a real source.
12+
// are persisted in `content.cache`, whose driver is namespaced per deployment and content SHA, so the
13+
// index survives cold starts, is shared across lambda instances in the region, and stale parser output
14+
// is unreachable. Colon-free so it can't collide with `<source>:<path>` content keys, the `manifest`
15+
// key, or the shared `gh:` namespace, and the SWR fallback in `cache.get` (`key.split(':')`) can't map
16+
// it to a real source.
1717
const SEARCH_SECTIONS_KEY = 'search-sections'
1818

1919
/**

test/github.test.ts

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
import { afterEach, describe, expect, it, vi } from 'vitest'
2+
import { contentCacheBase } from '../server/utils/cache'
3+
import { resolveContentSha } from '../server/utils/github'
4+
5+
afterEach(() => {
6+
vi.unstubAllEnvs()
7+
vi.unstubAllGlobals()
8+
})
9+
10+
describe('resolveContentSha', () => {
11+
it('resolves the latest commit touching the configured content directory', async () => {
12+
const fetch = vi.fn().mockResolvedValue([{ sha: 'content-sha' }])
13+
vi.stubGlobal('$fetch', fetch)
14+
15+
await expect(resolveContentSha('feat/docs', '/docs/content/')).resolves.toBe('content-sha')
16+
expect(fetch).toHaveBeenCalledWith(
17+
'https://api.github.com/repos/comarkdown/comark-docs/commits',
18+
expect.objectContaining({
19+
query: { sha: 'feat/docs', path: 'docs/content', per_page: 1 },
20+
})
21+
)
22+
})
23+
24+
it('caches each branch and content directory independently', async () => {
25+
const fetch = vi.fn().mockResolvedValueOnce([{ sha: 'docs-sha' }]).mockResolvedValueOnce([{ sha: 'api-sha' }])
26+
vi.stubGlobal('$fetch', fetch)
27+
28+
await expect(resolveContentSha('test/cache-key', 'docs/content')).resolves.toBe('docs-sha')
29+
await expect(resolveContentSha('test/cache-key', 'docs/content')).resolves.toBe('docs-sha')
30+
await expect(resolveContentSha('test/cache-key', 'api/content')).resolves.toBe('api-sha')
31+
expect(fetch).toHaveBeenCalledTimes(2)
32+
})
33+
34+
it('can refresh a cached content revision for the push webhook', async () => {
35+
const fetch = vi.fn().mockResolvedValueOnce([{ sha: 'before' }]).mockResolvedValueOnce([{ sha: 'after' }])
36+
vi.stubGlobal('$fetch', fetch)
37+
38+
await expect(resolveContentSha('test/refresh', 'docs/content')).resolves.toBe('before')
39+
await expect(resolveContentSha('test/refresh', 'docs/content')).resolves.toBe('before')
40+
await expect(resolveContentSha('test/refresh', 'docs/content', { refresh: true })).resolves.toBe('after')
41+
await expect(resolveContentSha('test/refresh', 'docs/content')).resolves.toBe('after')
42+
expect(fetch).toHaveBeenCalledTimes(2)
43+
})
44+
})
45+
46+
describe('contentCacheBase', () => {
47+
it('isolates parsed artifacts by deployment code revision', () => {
48+
vi.stubEnv('VERCEL_GIT_COMMIT_SHA', 'deployment-sha')
49+
expect(contentCacheBase('content-sha')).toBe('content:deployment-sha:content-sha')
50+
})
51+
52+
it('falls back to the deployment id outside Git deployments', () => {
53+
vi.stubEnv('VERCEL_GIT_COMMIT_SHA', '')
54+
vi.stubEnv('VERCEL_DEPLOYMENT_ID', 'deployment-id')
55+
expect(contentCacheBase('content-sha')).toBe('content:deployment-id:content-sha')
56+
})
57+
})

0 commit comments

Comments
 (0)