Repository navigation
feat: add recoverable desktop Library and 3D link browsing - #721
Merged
Merged
Conversation
added 19 commits
September 29, 2026 17:58
- Ground the daily library loop in the roadmap, code, and personal projects. - Prioritize protected desktop data, complete backups, and recoverable updates. - Record destructive startup behavior and phased implementation checks. Signed-off-by: xNet Test <test@xnet.dev>
- Ground the seed corpus in the existing Twitter, YouTube, and Instagram archives. - Add GitHub stars, collection fidelity, provenance, and cross-source graph views. - Prioritize resumable imports and include source evidence in backup validation. Signed-off-by: xNet Test <test@xnet.dev>
- Make metadata and offline thumbnails core requirements for every imported link. - Add caption discovery and local transcription fallbacks with explicit coverage. - Define persistent enrichment jobs, full-text indexing, and recovery checks. Signed-off-by: xNet Test <test@xnet.dev>
- Inspect isolated database copies and route unsupported stores to recovery. - Remove destructive startup resets and preserve inspection errors. - Isolate source launches from packaged data and retain existing daily paths. - Verify corrupt, old, future, and WAL-backed databases plus real app restart. Signed-off-by: xNet Test <test@xnet.dev>
- Serialize document snapshots and retain failed writes for retry. - Require durable SQLite commits before acknowledging desktop writes. - Verify complete native copies and recover interrupted restore renames. - Preserve replaced generations and pause sync until recovery review. - Add integrity, retention, WAL, failure, and document-write tests plus ADR-37. Signed-off-by: xNet Test <test@xnet.dev>
- Hold window close and app quit until text persistence acknowledges writes. - Add local recovery settings and startup restore before normal database open. - Stop background writers before update checkpoints and retain newer work on restore. - Keep restored workspaces offline until an explicit reviewed reconnect. - Record isolated desktop save, restart, restore, and recovery UI verification. Signed-off-by: xNet Test <test@xnet.dev>
- Parse nested Instagram collections and the liked-comments export wrapper. - Share post resources across Instagram likes, saves, and collection membership. - Preserve repeated YouTube memberships and report ambiguous filename matches. - Report unavailable Twitter bookmarks and native like timestamps explicitly. - Add a read-only inventory command and run source-shaped fixtures in CI. - Reconcile real archive counts without writing a personal workspace. Signed-off-by: xNet Test <test@xnet.dev>
- Read saved GitHub JSON snapshots through the archive pipeline. - Preserve repository IDs, native star dates, and private visibility across reimports. - Report unknown timestamps and pagination coverage explicitly. - Validate schema fields and correct imported Instagram creator handles. Signed-off-by: xNet Test <test@xnet.dev>
- Verify and privately retain source bytes before committing imported nodes. - Accept saved GitHub snapshots and expose import through the command palette. - Keep cancellation active until the outstanding write batch acknowledges. - Add repeatable star capture, four-source inventory, and restore evidence. Signed-off-by: xNet Test <test@xnet.dev>
- Schedule changed-data recovery and retain recent, daily, weekly, and pinned copies. - Surface backup failures and refuse replacement identities for existing workspaces. - Migrate supported historical storage on verified copies with preserved originals. - Validate restart, immediate text saves, visible failures, and native upgrade recovery. Signed-off-by: xNet Test <test@xnet.dev>
- Document native data, key locations, and the browser-state recovery gaps. - Record scheduled-copy failure checks and preservation through a real Electron upgrade. - Define tiered retention and candidate migration in ADR-39. Signed-off-by: xNet Test <test@xnet.dev>
- Verify encrypted chunks and recover the signing seed with a new key store. - Restore through a disposable candidate and preserve the current workspace. - Add desktop controls, corruption tests, and an original-profile-loss exercise. - Document excluded settings and the limits of off-device protection. Signed-off-by: xNet Test <test@xnet.dev>
- Persist reviewed selections and acknowledged batch cursors without private keys. - Reopen interrupted jobs paused and resume against the same importer version. - Preserve source-relative recovery paths and make journal failures visible. - Verify a 1500-repository import across restart with its original export removed. Signed-off-by: xNet Test <test@xnet.dev>
- Queue metadata, thumbnails, captions, and indexing with durable per-provider backoff. - Retain complete source evidence and timestamped captions outside bounded card fields. - Validate and cache image bytes, and pin public fetches to checked network addresses. - Freeze enrichment during imports and recovery; include its database in verified copies. - Verify 69 storage/provider tests and a real offline thumbnail and late-caption restart. Signed-off-by: xNet Test <test@xnet.dev>
- Open Library from the desktop shell and command menu. - Browse saved images and source details, filter providers, and search timestamped captions. - Show separate enrichment gaps and retain full evidence outside card responses. - Record real desktop restart and offline-search evidence without claiming full corpus coverage. Signed-off-by: xNet Test <test@xnet.dev>
- Accept garden JSON snapshots through the archive picker and resumable import path. - Reuse native video and post identities while retaining separate source commentary. - Preserve category and tag memberships and index authored notes in the Library. - Verify 242 social tests, eight Library tests, and a real desktop import and recovery. - Record a read-only reconciliation of all eight entries in the existing garden. Signed-off-by: xNet Test <test@xnet.dev>
- Journal each capture before native writes and replay interrupted saves with stable IDs. - Preserve source evidence and subsequent Page edits across duplicate requests and retries. - Index saved note bodies alongside source metadata and expose capture through desktop IPC. - Validate offline Page rendering and restart search in Electron; add six recovery tests. Signed-off-by: xNet Test <test@xnet.dev>
- Advance each local change from the persisted Lamport clock to preserve post-import edits. - Refresh subscribers from committed native writes without replaying or broadcasting them. - Separate IPC data types from native implementations and retain failed refresh work. - Verify 1915 focused data, storage, and library tests; confirm edited captures after restart. Signed-off-by: xNet Test <test@xnet.dev>
- Add offline link capture with excerpts, duplicate lookup, retained drafts, and explicit retries. - Open captured Pages beyond the recent list and refresh native changes in the shell. - Show matching transcript passages with timestamp links and reject unsupported note bodies. - Record desktop evidence, preview instructions, and the partially implemented exploration. Signed-off-by: xNet Test <test@xnet.dev>
|
Important Review skippedToo many files! This PR contains 207 files, which is 107 over the limit of 100. To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch. Upgrade to a paid plan to raise the limit. This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry. ⚙️ Run configuration
⛔ Files ignored due to path filters (1)
📒 Files selected for processing (207)
You can disable this status message by setting the
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
|
Preview removed for PR #721. |
…copies - Report unreadable manifests separately and keep their files outside retention deletion. - Let verified new copies and safe quit proceed while showing the damage in Settings. - Reject symlinked or unsafe manifest paths and keep strict listing for legacy callers. - Verify fresh copies, warning display, quit, and restart in an isolated Electron profile. - Give production-cost encrypted recovery tests a bounded CI execution budget. Signed-off-by: xNet Test <test@xnet.dev>
Contributor
🖼️ UI changes in this PRScreens✏️ Data _(SSIM 0.996)_
✏️ Discover _(SSIM 0.983)_
Desktop (Electron)✏️ Desktop · Canvas home _(SSIM 0.997)_
✏️ Desktop · Canvas with objects _(SSIM 0.996)_
Auto-captured by CI · run. Informational — not a blocking check. |
added 3 commits
September 29, 2026 23:13
- Pin the official Mac helper by size, SHA-256, and executable version. - Keep installation private, atomic, cancellable, and separate from enrichment. - Bound source requests and retry validated addresses without accepting private redirects. - Verify native cancellation, cleanup, retry, and restart; retain the live download gap. - Cover partial bytes, failed repair, malformed downloads, and network deadlines. Signed-off-by: xNet Test <test@xnet.dev>
- Update generated data declarations for Page source resources and NodeStore refresh. - Refresh the React API report after the document write barrier changes. - Keep API Extractor output unchanged so the drift check can verify committed reports. Signed-off-by: xNet Test <test@xnet.dev>
- Materialize the Electron sync fixture before acquiring its durable document. - Wait for the native NodeStore before creating the fixture. - Pin patched undici and brace-expansion versions without changing audit baselines. - Verify frozen installation, the dependency audit, and the Electron-web gate. Signed-off-by: xNet Test <test@xnet.dev>
- Describe canonical node data and device-local Library payloads. - Plan versioned enrichment records and resumable migration from saved content. - Define offline reconstruction, privacy, recovery, and performance checks. Signed-off-by: xNet Test <test@xnet.dev>
- Compare WebXR, native visionOS, and Mac-assisted viewing paths. - Specify tracked controller flight, spatial search, and metadata inspection. - Define read-only Library access, performance trials, and hardware validation. Signed-off-by: xNet Test <test@xnet.dev>
- Treat Retry-After as a minimum alongside exponential retry delays. - Keep throttled work resumable and preserve longer waits across restarts. - Cover repeated failures, provider isolation, and eventual recovery in tests. Signed-off-by: xNet Test <test@xnet.dev>
added 2 commits
October 4, 2026 10:23
- Increase the default readiness timeout from three to fifteen minutes. - Keep workspace inspection and native scope verification enabled during startup. Signed-off-by: xNet Test <test@xnet.dev>
- Use native macOS file clones instead of the unsupported Node clone path. - Fold WAL writes into isolated copies without rewriting unchanged database pages. - Preserve backup retention, integrity checks, and independent recovery snapshots. - Verify live-write isolation, restore and export, and the full 12,717-test suite. Signed-off-by: xNet Test <test@xnet.dev>
added 2 commits
October 4, 2026 23:38
- Wait up to ten minutes for startup checks and search reconciliation. - Keep the thirty-second deadline for ordinary requests. - Verify six-minute startup and immediate integrity-error propagation. - Confirm the old timeout fails the new regression test. Signed-off-by: xNet Test <test@xnet.dev>
- Treat empty version output as a transient probe failure instead of a missing installation. - Keep pinned-version checks, verified fallbacks, and blocking for absent or mismatched helpers. - Cover the regression with real process probes and discovery tests, including two negative controls. - Validate 38 focused Library tests and document the recovery behavior. Signed-off-by: xNet Test <test@xnet.dev>
1 task done
crs48
marked this pull request as ready for review
October 6, 2026 19:41
added 2 commits
October 6, 2026 12:45
- Keep the 15 newly overdue proposals open for a November 6 review. - Preserve their implementation checklists and the 41-item fallow baseline. - Regenerate the backlog report for the current review date. Signed-off-by: xNet Test <test@xnet.dev>
- Use Sharp 0.35.5 with the patched bundled librsvg dependency. - Replace the malformed PNG fixture rejected by the updated decoder. - Verify all 16 Library service tests and decoding under Electron 33. Signed-off-by: xNet Test <test@xnet.dev>
- Include the native Library and recovery decisions already in the source docs. - Verify generated text matches all 63 published documentation pages. Signed-off-by: xNet Test <test@xnet.dev>
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem and behavior
The desktop could discard unreadable storage during startup, and saved exports had no durable path from import to search. This adds a private desktop Library with retained imports, resumable enrichment, cached thumbnails and captions, editable source notes, a 3D graph of saved links, and verified native recovery copies.
Preserve unsupported or damaged workspaces. Commit native record batches atomically, flush before quit/update, and restore password-encrypted backups with known desktop settings and the provider key. Restore editing if Library resume acknowledgement fails after a recovery copy, keep the failure visible, and hold the recovery barrier through cleanup.
Import social activity, GitHub stars, garden notes, and AI conversations through the native archive screen. Retain original exports, deterministic identities, source relationships, and resumable job journals. Preserve richer Reddit post text across overlapping save/vote records.
Fetch YouTube and Instagram details, TikTok posters and subtitle tracks, GitHub repository READMEs and structured public details, Reddit previews, and visible article text. Route cited links by URL. Store complete retrieved text and image bytes locally; show partial results and access failures honestly.
Use indexed queue claims and targeted search updates for large imports. Pace ordinary citations by destination host, bound active work per source, and keep provider throttling retryable. A separate thumbnail CDN cooldown no longer pauses source metadata; source-host and unattributed throttles still pause the provider. Revisit older GitHub/web previews once while preserving fetched media.
Retrieve available captions with language and timing. Try alternate tracks and refresh YouTube tracks through the pinned helper when direct caption responses fail. Prioritize fresh caption indexing and selected retries without bypassing provider rate limits.
Search private AI conversations locally, including text beyond preview limits. Browse collections and save URL-plus-note captures as editable private Pages. Batch native reads so final import scans do not issue one hydration request per record.
Browse every saved web link in a full-window 3D force graph. Follow imported collections, topics, tags, categories, and creator names; navigate through ranked keyboard autocomplete, filter by any/all selected categories, tags, playlists, or creators, isolate neighborhoods, and hover for cached thumbnails and complete saved metadata. Keep orphan links visible and source counts explicit. The view is read-only and runs its layout locally. AI categories are not implemented.
Readiness
Exploration 0466 remains partially implemented; the implemented Library is requested for merge. The selected exports have been exercised in the real Electron development profile. Direct messages, billing, and account/security categories are excluded from graph import; complete retained exports still contain those categories. Earlier YouTube playlist and Instagram saves/likes selections are unchanged.
The Library holds 55,973 resources. All 120,985 expected graph records from GitHub, TikTok, Reddit, X, Claude, ChatGPT, Grok, and the garden are present; all 37,518 nonempty text comparisons match the selected exports. Reddit reimport repairs missing/shortened text without duplicating content. A verified recovery copy covers 28 workspace files.
The October 3 live snapshot held 39,760 fetched titles, 32,731 nonempty descriptions, 22,494 cached thumbnails, and 4,788 transcripts. Those counts include partial metadata; 10,381 metadata jobs remained queued or retrying at that instant. All 55,973 resources had local index entries. Public-page retrieval has processed the initial GitHub, YouTube, Reddit, and general-web queues; platform limits and remaining retries are still outstanding. Instagram, TikTok, X, caption retrieval, and thumbnail work continue. Earlier native checks verified searchable GitHub README and article text, cached images, and YouTube/TikTok transcript passages.
Whole-corpus enrichment is still running. Public access limits remain explicit gaps. Instagram written captions are separate from speech; local ASR and generated posters are not connected. The signed installed-Mac upgrade, off-device retention, complete settings/key coverage, successful managed-helper download, full website/overlapping-export reconciliation, guide publishing, and the founder trial remain open. A compatible existing helper works on this Mac; the managed install previously timed out.
Verification
Resource-list follow-up: native sampling found SQLite sorting full saved payloads while the Library refreshed every five seconds. On a disposable copy of all 55,973 resources, the unfiltered query took 7,125 ms and a filtered page took 4,320 ms. Indexes reduced those reads to 1.04 ms and 14.47 ms with identical ordered IDs. Periodic refreshes now coalesce while one is pending. All 123 focused checks passed, including ordering, platform pagination, and preserved long text across an older-cache upgrade. Real Electron returned three 40-row pages in 79–291 ms including IPC, remained editable, and stored 95 fresh metadata payloads plus 53 transcripts in about three minutes after resuming. These are local measurements, not provider throughput guarantees.
Recovery follow-up: a live
library:thawtimeout left the renderer inert even though enrichment resumed. Two regression checks failed before the fix. The revised workflow restores editing on failure, reports the error, retains the barrier until cleanup completes, and serializes waiting copies. All 122 focused Library/recovery/quit tests passed. On the updated native app, a fresh 28-file recovery copy completed, the Resources button remained usable, recovery reported no error, and eight enrichment jobs resumed. Fresh metadata, thumbnails, and transcripts were subsequently stored. The resume acknowledgement now has the same bounded two-minute deadline as freezing the Library.Full pre-push checks at
089ab2f32: 12,715 tests passed, four skipped; all 101 workspace typecheck tasks passed. Earlier frozen lockfile install and repository formatting checks passed.Full social package: 244 tests passed across 22 files.
Library enrichment: 100 focused tests passed, covering host isolation, migrations, search replacement, repeated throttling, public parsers, and captions. The latest tests verify redirected CDN error attribution and persisted capability cooldowns across restart, while preserving source-wide throttling for same-host or unknown-host failures. On a disposable copy of the actual corpus, sampled metadata claims fell from 421 ms to 0.13–0.64 ms; these are local queue timings, not an end-to-end throughput guarantee.
Earlier Library and 3D graph: 92 tests passed across 13 files, including pagination past 1,000 records, relation deduplication, orphan retention, filters, deterministic layout seeds, and the recovery read barrier. Earlier native batch/storage checks also passed.
Startup follow-up: the real multi-gigabyte Library exceeded the ordinary 30-second initialization deadline twice. Initialization now gets two minutes, with a three-minute development readiness probe; ordinary RPC deadlines and integrity checks are preserved. The regression test failed before the fix and passes afterward. Native startup and graph navigation succeeded with the change. Three actual-manager tests cover delayed success, bounded timeout, and immediate integrity-error propagation. Recovery-copy work can still block the main thread on this corpus and remains a responsiveness limitation.
Autocomplete and group-filter follow-up: 11 focused model/navigation tests passed. Native verification covered all 54,233 links, ranked keyboard suggestions, empty results, preview opening with the inspector hidden, group unions/intersections, source filters, chip removal, and restoring every link.
Desktop production build and main-process typecheck passed. Changed graph files pass ESLint. Repository ESLint reported zero errors and 469 existing warnings.
Real Electron: public provider results, caption retrieval and search, full conversation tail search, local image reads, offline thumbnail/transcript display, and verified recovery copies.
A scan of 55,973 Library resources took 7,976 ms after batched hydration.
Real Electron 3D graph: all 54,233 web links, 21,735 relationship groups, and 95,367 edges; no dangling edges or unreadable metadata. A 1,514-repository filter and a 45-link neighborhood were exercised. Hover, a saved 1280-pixel thumbnail, complete metadata, orbit/zoom, layout settling, Escape cleanup, and reopen passed. A local 90-frame sample during layout measured 8 ms median and 9 ms p95; this is not a cross-device benchmark.
Earlier isolated Electron checks cover native/encrypted restore, interrupted imports after removing the original export, offline restart search and thumbnails, capture edits across restart, and settings/key/draft recovery after deleting the test profile. Deterministic tests cover corrupt/future storage, incomplete recovery copies, rollback, interrupted writes, and stale clocks.
The standalone renderer typecheck still reports six existing errors involving deep-link inclusion, FormView, a readonly ref, plugin cleanup, and IPC interfaces. Root Turbo does not include that check.
Merge follow-up: 15 overdue proposals retain their open checklists and receive a November 6 review date; the exploration-age check passes at its unchanged 41-item baseline. The generated documentation export is refreshed from all 63 source pages. Electron now uses Sharp 0.35.5 with patched librsvg; all 16 Library service tests pass, and the decoder loads and decodes a PNG under Electron 33 / Node 20.18.3. The unrelated dependency versions flagged by the informational Audit already exist on main, and its baseline is unchanged. All required hosted checks pass on
749974845: real CI lint, typecheck, three test shards, editor UX, and changelog-section. The full local suite on that commit passed 12,722 tests with four skipped. The informational dependency Audit still reports pre-existing dependencies; its baseline remains unchanged.See
docs/reference/personal-library.mdand exploration 0466 for exact scope and remaining acceptance checks. Source archives, credentials, and live private content are excluded from this PR.