Skip to content
This repository was archived by the owner on Jun 3, 2026. It is now read-only.

Commit f0b53a8

Browse files
pgrayypgrayy
committed
docs: add design proposal for managing forked WASM dependencies
1 parent f0b2e81 commit f0b53a8

1 file changed

Lines changed: 186 additions & 0 deletions

File tree

‎designs/0011-fork-management.md‎

Lines changed: 186 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,186 @@
1+
# Managing Forked WASM Dependencies
2+
3+
**Status**: Proposed
4+
5+
**Date**: 2025-05-20
6+
7+
## Context
8+
9+
The WASM integration in sdk-typescript depends on modified versions of several upstream repositories:
10+
11+
- **wasmtime** (Rust) — the WASM runtime
12+
- **wasmtime-py** (Python) — Python bindings for wasmtime
13+
- **componentize-js** (JavaScript) — JS component model tooling
14+
- **jco** (JavaScript/Rust) — JS toolchain for WebAssembly Components
15+
16+
Upstream WASM tooling does not yet support all the features we need for the Python-via-WASM bindings. Certain capabilities (e.g., async streaming) require workarounds that live in forked versions of these dependencies. Today, those forks exist as separate repositories, which creates friction:
17+
18+
- Changes that span sdk-typescript and a fork require coordinating across multiple repos, branches, and PRs.
19+
- CI/CD pipelines must clone and build from multiple sources, adding complexity and fragility to the build.
20+
- Contributors need to discover and set up the correct fork versions manually.
21+
- There is no single place to see the full picture of what's modified and why.
22+
23+
Bringing the forks into sdk-typescript simplifies the development loop: one clone, one set of commits, one CI pipeline.
24+
25+
## Decision
26+
27+
Use **git subtrees** to import each fork's source directly into sdk-typescript.
28+
29+
Subtrees copy a remote repo's file tree into a subdirectory of the host repo. After import, the files are regular tracked content. No special tooling is required beyond the `git subtree` command, which ships with git.
30+
31+
Benefits of this approach:
32+
33+
- **Single clone** — contributors get everything with a standard `git clone`. No extra steps.
34+
- **Atomic commits** — a change to sdk-typescript and a fork workaround land in one commit/PR.
35+
- **Upstream sync** — `git subtree pull` merges new upstream changes; `git subtree push` extracts patches back when ready to upstream.
36+
- **Simpler CI** — one checkout, one pipeline. No submodule initialization, no cross-repo coordination.
37+
38+
The resulting layout:
39+
40+
```
41+
sdk-typescript/
42+
├── strands-ts/ # existing SDK package
43+
├── strands-wasm/ # existing WASM build tooling
44+
├── strands-py-wasm/ # existing Python bindings
45+
├── forks/
46+
│ ├── wasmtime/ # subtree
47+
│ ├── wasmtime-py/ # subtree
48+
│ ├── componentize-js/ # subtree
49+
│ └── jco/ # subtree
50+
├── package.json # workspace config
51+
```
52+
53+
### Metrics
54+
55+
sdk-typescript today clones at about 4 MB over the network (8.6 MB total on disk with full history, 408 files). After `npm ci`, disk usage grows to ~591 MB, almost entirely from `node_modules/` (~548 MB). The source code itself is small.
56+
57+
Each fork was measured via shallow clone (`--depth=1`) against the current main branch:
58+
59+
| Fork | Network transfer | Disk | Files | Notes |
60+
|------|-----------------|------|-------|-------|
61+
| wasmtime | 26 MB | 86 MB | 6,700 | Largest: `crates/` 41 MB, `cranelift/` 21 MB, `tests/` 17 MB |
62+
| wasmtime-py | 304 KB | 1 MB | 91 | Negligible |
63+
| componentize-js | 352 KB | 1.5 MB | 265 | Negligible |
64+
| jco | 80 MB | 413 MB | 1,060 | 401 MB is `.wasm` test fixtures; source is ~12 MB |
65+
66+
wasmtime and jco carry significant weight in directories we don't need (test fixtures, docs, benchmarks). Maintaining a **slim branch** in each fork strips these before import:
67+
68+
| Fork | After stripping | Files | Reduction |
69+
|------|----------------|-------|-----------|
70+
| wasmtime (drop tests, docs, benches, examples) | ~60 MB | ~4,000 | ~30% |
71+
| jco (drop test fixtures) | ~12 MB | ~650 | ~97% |
72+
| wasmtime-py | No change needed | 91 | Already tiny |
73+
| componentize-js | No change needed | 265 | Already tiny |
74+
75+
A slim branch is regenerated from main whenever upstream is updated:
76+
77+
```bash
78+
# In the fork repo:
79+
git checkout main
80+
git pull upstream main
81+
git checkout -B slim
82+
git rm -rf tests/ docs/ benches/ examples/
83+
git commit -m "chore: strip non-essential dirs for slim branch"
84+
git push origin slim --force
85+
```
86+
87+
Then pulled into sdk-typescript with `--squash` to collapse upstream history into a single merge commit:
88+
89+
```bash
90+
git subtree add --prefix=forks/wasmtime <fork-url> slim --squash # initial
91+
git subtree pull --prefix=forks/wasmtime <fork-url> slim --squash # updates
92+
```
93+
94+
With all four forks imported (slim where applicable, squashed), the projected clone size rises from ~4 MB to roughly 30-35 MB. This is still modest compared to the ~548 MB that `npm ci` adds.
95+
96+
## Developer Experience
97+
98+
### Making Changes to Fork Code
99+
100+
Edit files directly and commit normally:
101+
102+
```bash
103+
vim forks/wasmtime/crates/something/src/lib.rs
104+
git add forks/wasmtime/
105+
git commit -m "fix: workaround for async streaming in wasmtime"
106+
```
107+
108+
### Pulling Upstream Changes
109+
110+
Update the slim branch in the fork repo first (scripted), then pull into sdk-typescript:
111+
112+
```bash
113+
git subtree pull --prefix=forks/wasmtime <fork-url> slim --squash
114+
```
115+
116+
Conflicts with local patches are resolved as a normal merge.
117+
118+
### Pushing Changes Back Upstream
119+
120+
When ready to contribute fixes back:
121+
122+
```bash
123+
git subtree push --prefix=forks/wasmtime <fork-url> my-upstream-pr-branch
124+
```
125+
126+
This extracts only the commits that touched `forks/wasmtime/`, rewrites their paths (strips the prefix), and pushes them to the fork repo. From there, open a PR on the upstream project.
127+
128+
## Alternatives Considered
129+
130+
### Submodules
131+
132+
Each fork lives in its own GitHub repo. sdk-typescript references them at pinned commits via `.gitmodules`. Each submodule is a pointer (SHA) to a specific commit in the fork repo. Contributors must use `git clone --recurse-submodules` or run `git submodule update --init` after cloning.
133+
134+
**Pros:**
135+
136+
- Each fork retains full independence (own history, branches, tags, CI).
137+
- Pinned to exact commits for reproducible builds.
138+
- No increase to sdk-typescript's pack size.
139+
- Natural for upstreaming (fork repos are first-class).
140+
141+
**Cons:**
142+
143+
- Known developer experience pain: forgetting `--recurse-submodules` is common.
144+
- PRs that span sdk-typescript + a fork require coordinating across repos.
145+
- Detached HEAD state inside submodules is confusing.
146+
- Nested submodules (wasmtime has its own) compound complexity.
147+
- CI must be configured to initialize submodules.
148+
149+
### Git LFS
150+
151+
Replace large binary files (e.g., `.wasm` test fixtures) with small pointer files (~130 bytes each). Actual content lives on a separate LFS server and is fetched lazily on checkout. Supported on our GitHub Enterprise plan (5 GB storage, 5 GB/month bandwidth included).
152+
153+
**Pros:**
154+
155+
- Dramatically reduces clone size for repos with large binaries.
156+
- Transparent to most workflows once set up.
157+
158+
**Cons:**
159+
160+
- Only helps with binary files; doesn't reduce file count from source directories (tests, docs, etc.).
161+
- Adds infrastructure dependency (LFS server must be available).
162+
- Some tooling (mirrors, forks, CI runners) doesn't handle LFS transparently.
163+
- Switching files in/out of LFS after the fact is messy.
164+
- Doesn't address the core problem of cross-repo coordination.
165+
166+
## Consequences
167+
168+
- sdk-typescript clone size increases from ~4 MB to ~30-35 MB (still small).
169+
- Developers can modify fork code and sdk-typescript code in a single PR.
170+
- CI/CD pipelines simplify to a single checkout.
171+
- Slim branches require periodic regeneration (scriptable).
172+
- Upstream sync requires `git subtree pull/push` commands, which are verbose but infrequent.
173+
174+
## Willingness to Implement
175+
176+
Yes.
177+
178+
## References
179+
180+
- [Git Subtree Documentation](https://git-scm.com/book/en/v2/Git-Tools-Advanced-Merging#_subtree_merge)
181+
- [Git Submodule Documentation](https://git-scm.com/book/en/v2/Git-Tools-Submodules)
182+
- [Git LFS](https://git-lfs.com/)
183+
- [bytecodealliance/wasmtime](https://github.com/bytecodealliance/wasmtime)
184+
- [bytecodealliance/wasmtime-py](https://github.com/bytecodealliance/wasmtime-py)
185+
- [bytecodealliance/ComponentizeJS](https://github.com/bytecodealliance/ComponentizeJS)
186+
- [bytecodealliance/jco](https://github.com/bytecodealliance/jco)

0 commit comments

Comments
 (0)