Skip to content

fix: release Chroma handles before migrate swap on Windows#890

Open
fuzzymoomoo wants to merge 1 commit intoMemPalace:developfrom
fuzzymoomoo:fix/windows-migrate-swap-release
Open

fix: release Chroma handles before migrate swap on Windows#890
fuzzymoomoo wants to merge 1 commit intoMemPalace:developfrom
fuzzymoomoo:fix/windows-migrate-swap-release

Conversation

@fuzzymoomoo
Copy link
Copy Markdown

Summary

Fix a Windows-specific mempalace migrate failure where the final palace swap can raise PermissionError: [WinError 32] on chroma.sqlite3 even after the import finished successfully.

Root cause

migrate() was importing drawers into a fresh palace and then immediately calling shutil.rmtree(palace_path).

On Windows, Chroma-backed clients can keep SQLite handles alive longer than plain object deletion suggests. In the current codebase, those handles can sit behind:

  • the temporary ChromaCollection
  • the backend wrapper's _clients cache
  • the underlying Chroma shared system cache

That means the migrate path can successfully import everything and still fail at the last filesystem step.

Changes

  • add _release_chroma_handles() to stop/clear best-effort Chroma client state before directory replacement
  • teach the helper to walk through backend-cached clients (_clients) as well as wrapped collection/client objects
  • add _replace_palace_dir() with bounded retry on PermissionError
  • release handles after the initial readability probe and again before the destructive swap
  • update tests/test_migrate.py to match the current ChromaBackend path
  • add a regression test that simulates a first-attempt Windows lock during the palace swap

Why this is scoped this way

This keeps the fix local to migrate.py rather than changing backend lifecycle semantics for the whole app.

That felt like the safest way to land a real Windows fix quickly while staying aligned with the current architecture.

Validation

Local targeted checks:

  • python -m pytest tests/test_migrate.py -q
  • python -m ruff check mempalace/migrate.py tests/test_migrate.py

Real Windows reproduction / rerun:

  • restored a copied 16,468-drawer palace that was unreadable without migration under the current runtime
  • ran:
    • python -m mempalace.cli --palace C:\Users\fuzzy\.mempalace\palace-upgrade-eval migrate --yes
  • confirmed migration completed successfully without manual directory swap
  • confirmed follow-up status on the migrated copy reports all 16,468 drawers correctly

Related context

This is in the same family as other Windows Chroma handle issues already seen in the repo, but I did not find an open PR specifically fixing the migrate swap path.

Related prior Windows handle discussions / fixes:

@fuzzymoomoo fuzzymoomoo marked this pull request as ready for review April 14, 2026 21:20
@igorls igorls added bug Something isn't working area/windows Windows-specific bugs and compatibility storage labels Apr 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/windows Windows-specific bugs and compatibility bug Something isn't working storage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants