-
Notifications
You must be signed in to change notification settings - Fork 8
feat(site): serve downloads via Cloudflare R2 mirror with GitHub fallback #1000
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
58f5901
feat(site): serve downloads via Cloudflare R2 mirror with GitHub fall…
Astro-Han 935165d
fix(site): address review feedback on R2 mirror
Astro-Han 03f3a98
test(desktop): lock R2 mirror upload order and manifest shape
Astro-Han 9310680
fix(ci): pass release tag via env to prevent shell injection in R2 mi…
Astro-Han File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| name: mirror-release-to-r2 | ||
|
|
||
| # Mirror a published release's installers to Cloudflare R2 so the China landing | ||
| # page (site/) serves fast CDN-cached direct downloads. Gated by verify-release: | ||
| # if the release is incomplete the mirror fails closed and the site keeps | ||
| # pointing at the previous good release. | ||
|
|
||
| on: | ||
| release: | ||
| types: [published] | ||
| workflow_dispatch: | ||
| inputs: | ||
| tag: | ||
| description: "Release tag to mirror (e.g. v2026.5.29)" | ||
| required: true | ||
|
|
||
| concurrency: | ||
| group: mirror-r2-${{ github.event.release.tag_name || github.event.inputs.tag }} | ||
| cancel-in-progress: false | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| env: | ||
| R2_BUCKET: pawwork-downloads | ||
| DOWNLOAD_PUBLIC_BASE: https://dl.pawwork.ai | ||
|
|
||
| jobs: | ||
| mirror: | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 15 | ||
| steps: | ||
| - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 | ||
| with: | ||
| persist-credentials: false | ||
|
|
||
| - uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2 | ||
| with: | ||
| # Load-bearing for `bun audit` exit semantics; do not bump without re-verifying advisory exit codes. | ||
| bun-version: "1.3.14" | ||
|
|
||
| - name: Resolve tag | ||
| id: tag | ||
| run: echo "tag=${{ github.event.release.tag_name || github.event.inputs.tag }}" >> "$GITHUB_OUTPUT" | ||
|
|
||
| - name: Verify release is complete (fail closed) | ||
| run: bun packages/desktop-electron/scripts/verify-release.ts "${{ steps.tag.outputs.tag }}" | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Mirror release assets to R2 | ||
| run: bun packages/desktop-electron/scripts/mirror-release-to-r2.ts "${{ steps.tag.outputs.tag }}" | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| R2_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} | ||
| AWS_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }} | ||
| AWS_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }} | ||
| AWS_DEFAULT_REGION: auto | ||
55 changes: 55 additions & 0 deletions
55
packages/desktop-electron/scripts/mirror-release-to-r2.test.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| import { describe, expect, test } from "bun:test" | ||
|
|
||
| import { releaseAssetNames } from "./verify-release.ts" | ||
| import { buildManifest, uploadPlan } from "./mirror-release-to-r2.ts" | ||
|
|
||
| describe("buildManifest", () => { | ||
| test("locks the manifest shape and per-platform installer URLs", () => { | ||
| expect(buildManifest("2026.5.29", "https://dl.pawwork.ai")).toEqual({ | ||
| version: "2026.5.29", | ||
| macArm64: "https://dl.pawwork.ai/pawwork-mac-arm64-2026.5.29.dmg", | ||
| macX64: "https://dl.pawwork.ai/pawwork-mac-x64-2026.5.29.dmg", | ||
| winX64: "https://dl.pawwork.ai/pawwork-win-x64-2026.5.29.exe", | ||
| }) | ||
| }) | ||
|
|
||
| test("normalizes a trailing slash in the public base", () => { | ||
| expect(buildManifest("2026.5.29", "https://dl.pawwork.ai/").macArm64).toBe( | ||
| "https://dl.pawwork.ai/pawwork-mac-arm64-2026.5.29.dmg", | ||
| ) | ||
| }) | ||
| }) | ||
|
|
||
| describe("uploadPlan", () => { | ||
| const plan = uploadPlan(releaseAssetNames("2026.5.29")) | ||
| const names = plan.map((step) => step.name) | ||
|
|
||
| test("ends with the landing-page manifest as the single live switch", () => { | ||
| expect(names.at(-1)).toBe("latest.json") | ||
| expect(plan.at(-1)).toMatchObject({ manifest: true, cacheControl: "no-cache, must-revalidate" }) | ||
| }) | ||
|
|
||
| test("orders immutable versioned artifacts before the mutable updater pointers", () => { | ||
| const lastVersioned = Math.max( | ||
| names.indexOf("pawwork-mac-arm64-2026.5.29.dmg"), | ||
| names.indexOf("pawwork-win-x64-2026.5.29.exe"), | ||
| names.indexOf("pawwork-mac-arm64-2026.5.29.zip.blockmap"), | ||
| ) | ||
| const firstPointer = Math.min(names.indexOf("latest.yml"), names.indexOf("latest-mac.yml")) | ||
| expect(lastVersioned).toBeLessThan(firstPointer) | ||
| expect(firstPointer).toBeLessThan(names.indexOf("latest.json")) | ||
| }) | ||
|
|
||
| test("marks versioned artifacts immutable and pointers no-cache", () => { | ||
| const cacheOf = (name: string) => plan.find((step) => step.name === name)?.cacheControl | ||
| expect(cacheOf("pawwork-mac-arm64-2026.5.29.dmg")).toBe("public, max-age=31536000, immutable") | ||
| expect(cacheOf("latest.yml")).toBe("no-cache, must-revalidate") | ||
| expect(cacheOf("latest-mac.yml")).toBe("no-cache, must-revalidate") | ||
| }) | ||
|
|
||
| test("uploads every released asset exactly once plus the manifest", () => { | ||
| const assets = releaseAssetNames("2026.5.29") | ||
| expect(names.slice(0, -1).sort()).toEqual([...assets].sort()) | ||
| expect(new Set(names).size).toBe(names.length) | ||
| }) | ||
| }) |
171 changes: 171 additions & 0 deletions
171
packages/desktop-electron/scripts/mirror-release-to-r2.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,171 @@ | ||
| // Mirror a published GitHub Release's installers to Cloudflare R2 so the China | ||
| // landing page (site/) can serve fast, CDN-cached direct downloads. | ||
| // | ||
| // Ordering matters (see PR discussion): versioned installer objects are | ||
| // immutable and uploaded first; the mutable pointers — the electron-updater | ||
| // latest*.yml and finally the landing page's latest.json — are written last, | ||
| // so a failure leaves the site pointing at the previous good release rather | ||
| // than a half-mirrored one. Run AFTER verify-release.ts has confirmed the | ||
| // release is complete. | ||
| // | ||
| // Usage: bun packages/desktop-electron/scripts/mirror-release-to-r2.ts <tag> [owner/repo] | ||
| // Env: | ||
| // R2_ACCOUNT_ID Cloudflare account id (S3 endpoint host) | ||
| // R2_BUCKET target bucket, e.g. pawwork-downloads | ||
| // DOWNLOAD_PUBLIC_BASE public base URL, e.g. https://dl.pawwork.ai | ||
| // AWS_ACCESS_KEY_ID R2 token access key (S3-compatible) | ||
| // AWS_SECRET_ACCESS_KEY R2 token secret | ||
| // GH_TOKEN GitHub token for `gh release download` | ||
|
|
||
| import { mkdtemp, readdir, rm, stat } from "node:fs/promises" | ||
| import { tmpdir } from "node:os" | ||
| import { join } from "node:path" | ||
| import { releaseAssetNames } from "./verify-release.ts" | ||
|
|
||
| const MUTABLE_POINTERS = new Set(["latest.yml", "latest-mac.yml"]) | ||
| const IMMUTABLE_CACHE = "public, max-age=31536000, immutable" | ||
| const POINTER_CACHE = "no-cache, must-revalidate" | ||
| const MANIFEST_NAME = "latest.json" | ||
|
|
||
| export type UploadStep = { name: string; cacheControl: string; manifest?: boolean } | ||
|
|
||
| // Pointer object the landing page reads to swap download buttons to R2 links. | ||
| // Keys match the data-dl attributes on the buttons in site/src/pages/index.astro. | ||
| export function buildManifest(version: string, publicBase: string) { | ||
| const base = publicBase.replace(/\/$/, "") | ||
| return { | ||
| version, | ||
| macArm64: `${base}/pawwork-mac-arm64-${version}.dmg`, | ||
| macX64: `${base}/pawwork-mac-x64-${version}.dmg`, | ||
| winX64: `${base}/pawwork-win-x64-${version}.exe`, | ||
| } | ||
| } | ||
|
|
||
| // Ordered upload plan: immutable versioned artifacts first, then the mutable | ||
| // electron-updater pointers, then the landing-page manifest LAST — the single | ||
| // switch that makes a release live. The order is load-bearing (a half-mirror | ||
| // must never point the site at incomplete artifacts); locked by the test. | ||
| export function uploadPlan(assets: string[]): UploadStep[] { | ||
| const versioned = assets | ||
| .filter((name) => !MUTABLE_POINTERS.has(name)) | ||
| .map((name) => ({ name, cacheControl: IMMUTABLE_CACHE })) | ||
| const pointers = assets | ||
| .filter((name) => MUTABLE_POINTERS.has(name)) | ||
| .map((name) => ({ name, cacheControl: POINTER_CACHE })) | ||
| return [...versioned, ...pointers, { name: MANIFEST_NAME, cacheControl: POINTER_CACHE, manifest: true }] | ||
| } | ||
|
|
||
| const CONTENT_TYPES: Record<string, string> = { | ||
| dmg: "application/x-apple-diskimage", | ||
| exe: "application/octet-stream", | ||
| zip: "application/zip", | ||
| yml: "text/yaml", | ||
| json: "application/json", | ||
| blockmap: "application/octet-stream", | ||
| } | ||
|
|
||
| function contentTypeFor(name: string) { | ||
| const ext = name.split(".").pop() ?? "" | ||
| return CONTENT_TYPES[ext] ?? "application/octet-stream" | ||
| } | ||
|
|
||
| function requireEnv(key: string): string { | ||
| const value = process.env[key] | ||
| if (!value) throw new Error(`Missing required env: ${key}`) | ||
| return value | ||
| } | ||
|
|
||
| async function run(cmd: string[]): Promise<string> { | ||
| const proc = Bun.spawn(cmd, { stdout: "pipe", stderr: "pipe" }) | ||
| const [stdout, stderr, code] = await Promise.all([ | ||
| new Response(proc.stdout).text(), | ||
| new Response(proc.stderr).text(), | ||
| proc.exited, | ||
| ]) | ||
| if (code !== 0) throw new Error(`Command failed (${code}): ${cmd.join(" ")}\n${stderr}`) | ||
| return stdout | ||
| } | ||
|
|
||
| async function main() { | ||
| const tag = process.argv[2] | ||
| if (!tag) { | ||
| console.error("Usage: bun packages/desktop-electron/scripts/mirror-release-to-r2.ts <tag> [owner/repo]") | ||
| process.exit(1) | ||
| } | ||
| const repo = process.argv[3] ?? "Astro-Han/pawwork" | ||
| const version = tag.replace(/^v/, "") | ||
|
|
||
| const accountId = requireEnv("R2_ACCOUNT_ID") | ||
| const bucket = requireEnv("R2_BUCKET") | ||
| const publicBase = requireEnv("DOWNLOAD_PUBLIC_BASE").replace(/\/$/, "") | ||
| requireEnv("AWS_ACCESS_KEY_ID") | ||
| requireEnv("AWS_SECRET_ACCESS_KEY") | ||
| const endpoint = `https://${accountId}.r2.cloudflarestorage.com` | ||
|
|
||
| const assets = releaseAssetNames(version) | ||
|
|
||
| const dir = await mkdtemp(join(tmpdir(), "pawwork-r2-")) | ||
| try { | ||
| await mirror({ assets, tag, repo, dir, bucket, endpoint, publicBase, version }) | ||
| } finally { | ||
| await rm(dir, { recursive: true, force: true }) | ||
| } | ||
| } | ||
|
|
||
| type MirrorArgs = { | ||
| assets: string[] | ||
| tag: string | ||
| repo: string | ||
| dir: string | ||
| bucket: string | ||
| endpoint: string | ||
| publicBase: string | ||
| version: string | ||
| } | ||
|
|
||
| async function mirror({ assets, tag, repo, dir, bucket, endpoint, publicBase, version }: MirrorArgs) { | ||
| console.log(`Downloading ${assets.length} assets of ${tag} from ${repo} ...`) | ||
| for (const name of assets) { | ||
| await run(["gh", "release", "download", tag, "--repo", repo, "--pattern", name, "--dir", dir]) | ||
| } | ||
| const present = new Set(await readdir(dir)) | ||
| const missing = assets.filter((name) => !present.has(name)) | ||
| if (missing.length) throw new Error(`Assets missing after download: ${missing.join(", ")}`) | ||
|
|
||
| const upload = async (name: string, cacheControl: string) => { | ||
| const local = join(dir, name) | ||
| await run([ | ||
| "aws", "s3", "cp", local, `s3://${bucket}/${name}`, | ||
| "--endpoint-url", endpoint, | ||
| "--content-type", contentTypeFor(name), | ||
| "--cache-control", cacheControl, | ||
| "--no-progress", | ||
| ]) | ||
| const head = JSON.parse( | ||
| await run(["aws", "s3api", "head-object", "--bucket", bucket, "--key", name, "--endpoint-url", endpoint]), | ||
| ) | ||
| const localSize = (await stat(local)).size | ||
| if (head.ContentLength !== localSize) { | ||
| throw new Error(`Size mismatch for ${name}: local ${localSize} vs R2 ${head.ContentLength}`) | ||
| } | ||
| console.log(` ✓ ${name} (${localSize} bytes)`) | ||
| } | ||
|
|
||
| // Upload in the load-bearing order (versioned -> updater pointers -> manifest | ||
| // last). The manifest is generated just before its upload. | ||
| for (const step of uploadPlan(assets)) { | ||
| if (step.manifest) { | ||
| await Bun.write(join(dir, step.name), JSON.stringify(buildManifest(version, publicBase), null, 2)) | ||
| } | ||
| await upload(step.name, step.cacheControl) | ||
| } | ||
|
|
||
| console.log(`Mirrored ${tag} to ${publicBase} (latest.json -> ${version}).`) | ||
| } | ||
|
Astro-Han marked this conversation as resolved.
|
||
|
|
||
| if (import.meta.main) { | ||
| main().catch((err) => { | ||
| console.error(err instanceof Error ? err.message : err) | ||
| process.exit(1) | ||
| }) | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.