Repository navigation
docs(exploration): database scale limits, virtualization, and materialization (0318) - #505
Merged
Merged
Conversation
… materialization Measured 10k/100k/1M/10M-row databases end to end (native better-sqlite3 tier through the real SQLiteNodeStorageAdapter, plus the live web app on OPFS via Playwright). Indexed first pages stay flat at 0.6-0.8ms to 10M; four O(N) query shapes are the real limits (property sort, after-cursor, exact count, deep OFFSET), the MV build OOMs at 10M, and the grid's 500-row window mislabels every larger database. Includes the env-gated benchmark harness used to collect the data. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: xNet Test <test@xnet.dev>
Contributor
|
✓ Changelog fragment found — thanks! |
Contributor
|
Preview removed for PR #505. |
crs48
added a commit
that referenced
this pull request
Jul 17, 2026
## Summary Adds exploration **0335** — a full-repo release-readiness audit answering *"what's in the way of releasing xNet, and what would a well-received release need?"* Grounded in a six-dimension parallel audit (release/packaging infra, security & authz, CI & test health, docs/DX/legal, product completeness, external launch bar). The four release-blockers were re-verified against live code. ## Headline Pipelines, protocol spec + golden vectors, and governance are all healthy — the gap is a short, specific list. **4 release-blockers (verified in code):** 1. Electron boots a deterministic test signing key — `apps/electron/src/renderer/main.tsx:833` (`makeTestKey`, unconditional; secure path in `secure-seed.ts` unused). 2. Wildcard UCAN neutralizes hub authz — `use-hub-auth-token.ts:10` + `hub/ws/authorize.ts:146` (0307 "Option B"). 3. Envelope signature omits ciphertext — `packages/crypto/src/envelope.ts:167` (0307 "Option A"). 4. No `SECURITY.md`. **Operational blocker:** release PR #526 has zero CI (changesets branch pushed with `GITHUB_TOKEN` → no workflows). Fix: provision `RELEASE_GITHUB_TOKEN`. **Well-received gap:** version incoherence, README advertising ~12 npm packages that 404, at-scale perf (0318) unmerged on PR #505, read-only AI assistant, rotted `validate:canvas-v2` gate. **Recommendation:** phased Option B — soft-launch SDK + hosted demo + hub (fixing the two multi-user security blockers + SECURITY.md + release-PR CI first), position desktop as beta, then cut `xnet 1.0`. Docs-only; no changeset needed. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
crs48
added a commit
that referenced
this pull request
Jul 30, 2026
…(0410) (#658) Implements exploration `0410_[_]_OPEN_PR_TRIAGE_AND_THE_STRANDED_BRANCH_PROBLEM.md`, written in the previous turn. ## What this closes out Eight PRs were open, the oldest from March. Checking each one's **actual diff against `main`** (rather than its description) found that four described work already in `main`, re-derived and landed weeks later by another route. | PR | Age | Action taken | | --- | --- | --- | | #400 | 24d | **Closed** — fix landed on main in a better form; branch predates 0391 and would have deleted `NodeLineRunner` | | #9 | 145d | **Closed** — draft; `apps/electron/src/renderer/workspace/` no longer exists | | #595 | 11d | **Closed** — byte-identical to main (`fbc2965a2`) | | #463 | 20d | **Closed** — main has a longer, `[x]` version of 0298 | | #13 | 141d | **Closed** — both fixes on main verbatim | | #505 | 17d | **Merged** — exploration 0318 + scale bench | | #389 | 25d | **Merged** — exploration 0270 | | #449 | 20d | **Rebuilt here** — see below | `#400` was the load-bearing one: `git diff origin/main pr400 -- packages/devkit/src/command-runner.ts` showed ~150 lines of *deletion* covering the whole 0391 streaming agent seam. Only its `CONFLICTING` state had prevented that. `#505`'s `build-and-smoke-test` went green on a rebase, confirming it was branch staleness rather than the diff. ## The #449 rebuild (0290 bugs #2 and #3) The original patched `apps/web/src/workbench/SyncStatus.tsx`, which 0406 deleted; the component now lives in `packages/workbench`, so the rebuild lands on **desktop and web at once**. - `WorkbenchHost` gains a `hub` capability (`configuredUrl` / `connect`) so the shell can offer a way out of the disconnected state without importing either app's `hub-url` module. `connect` returns a reason string on rejection — never a silent no-op. - `SyncStatus` exports `openSyncStatusPanel()`; the desktop popover and mobile sheet both listen. The panel shows an inline hub-URL form while there is no hub. - `ShareDialog`'s `!ready` branch explains why a hub is needed and offers **Connect a hub…**; private-hub links now confirm before copying. - Adds `normalizeHubUrl` to the electron renderer's `hub-url` mirror (+4 unit tests), so a malformed URL is rejected loudly rather than persisted. Verified by driving the real app against a local hub — the CTA closes the dialog and opens the panel; an invalid URL shows an error and persists nothing; `https://hub.xnet.fyi/` normalises to `wss://hub.xnet.fyi`; a `localhost` link's first Copy click asks "Copy local-only link?" and writes nothing, the second copies. No console errors. 0290 is now `[-]` at 11/20. ## Stopping the recurrence Time-to-merge for the last 20 merged PRs was **0 hours median, 1 hour max** — there is no review queue, so a PR that outlives its session is abandoned, not pending. - `.github/workflows/stale.yml` — warns at 14d, closes at 21d, `keep-open` exempts. Named consumer (the maintainer, weekly) and a decidable pass condition, per AGENTS.md. - Fixed the `/explore` next-number command, which read only the working tree and so handed out numbers already claimed on branches. Seven explorations were found stranded that way. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Exploration 0318 — measured database load/render behavior at 10k / 100k / 1M / 10M rows, native tier (real
SQLiteNodeStorageAdapterover better-sqlite3) plus the live web app on OPFS (Playwright).Headlines (all measured):
sortKeyorder — 20s@1M),after-cursor pages (29s@1M, hydrates the full result set),count:'exact'(24.6s@10M), deep OFFSET (3.5s@5M).ERR_WORKER_OUT_OF_MEMORY(~2.5KB heap/row).Recommends keyset pushdown + index-servable property sorts + estimate counts for effectively-infinite scroll (no materialization needed sequentially), and sequences the MV-table question against 0317's P5 (its "zero production opt-in" premise is stale —
useGridDatabaseshipsdb:<id>MV keys today).Also adds the env-gated benchmark harness (
packages/data/src/store/scale-limits.bench.test.ts, inert unlessXNET_SCALE_BENCHis set) so the validation checklist is re-runnable.Test plan
XNET_SCALE_BENCH(verified: 1 skipped)prettier --checkandtsc --noEmit -p packages/dataclean🤖 Generated with Claude Code