Skip to content

feat: auto-fallback to SQLite when embedded SeekDB is unavailable - #993

Merged
lightzt99 merged 4 commits into
oceanbase:mainfrom
knqiufan:feat/cross-platform-storage-default-fallback
Jul 1, 2026
Merged

feat: auto-fallback to SQLite when embedded SeekDB is unavailable#993
lightzt99 merged 4 commits into
oceanbase:mainfrom
knqiufan:feat/cross-platform-storage-default-fallback

Conversation

@knqiufan

@knqiufan knqiufan commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add powermem.platform_defaults with choose_default_database_provider() to centralize zero-config storage selection.
  • When DATABASE_PROVIDER and OCEANBASE_HOST are unset:
    • Non-Linux platforms default to SQLite (./data/powermem_dev.db).
    • Linux probes embedded SeekDB (pyobvector, pyseekdb, pylibseekdb); uses OceanBase embedded mode when available, otherwise falls back to SQLite with a WARNING.
  • Wire the helper into config loading, Memory/CLI setup, OceanBase embedded init, and server startup.
  • Expose storage_capabilities on /api/v1/system/status so clients can see SQLite limitations (no Graph Store, sub_stores, sparse vectors, SkillStore).
  • Update .env.example / .env.example.full comments to document platform-aware defaults.

Closes #992

Test plan

  • pytest tests/unit/test_storage_default_provider.py
  • pytest tests/unit/test_seekdb_default_storage.py
  • Windows zero-config: powermem-server starts, memory APIs return 200 (not 503)
  • Linux + powermem[seekdb]: zero-config still uses embedded SeekDB at ./seekdb_data
  • Explicit DATABASE_PROVIDER=oceanbase on non-Linux still surfaces a clear validation error (no silent override)

@knqiufan knqiufan changed the title fix(storage): fall back to sqlite when seekdb is unavailable feat: auto-fallback to SQLite when embedded SeekDB is unavailable Jun 8, 2026
@wayyoungboy

Copy link
Copy Markdown
Member

#1031

Copy link
Copy Markdown
Member

Thanks for the update. I rechecked this after the latest commits. The checks are green, but GitHub currently reports this branch as conflicting with main, so I cannot mark it LGTM yet.

Please rebase or merge the latest main and ping me; I will review the rebased diff again.

knqiufan added 2 commits June 22, 2026 22:43
Centralize platform-aware storage default selection so zero-config startup uses embedded SeekDB only when available, and falls back to SQLite otherwise.

Expose memory service readiness and SQLite capability limitations through server status and the dashboard.

Closes oceanbase#992
@knqiufan
knqiufan force-pushed the feat/cross-platform-storage-default-fallback branch from 8328d4c to 50a9b5c Compare June 22, 2026 14:50
@knqiufan

Copy link
Copy Markdown
Contributor Author

Thanks @wayyoungboy. Rebased on the latest main (ffefdb2).

The only conflict was a non-semantic import ordering in src/powermem/config_loader.py — kept both the new NoopConfig import (from main) and the platform_defaults imports (from this PR). Verified noop is still registered in BaseLLMConfig._registry after the rebase.

Test plan still green:

  • pytest tests/unit/test_storage_default_provider.py — 7/7 passed
  • pytest tests/unit/test_seekdb_default_storage.py — 3/3 passed

Ready for re-review whenever you have a moment.

@wayyoungboy wayyoungboy left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for rebasing. I rechecked the latest head (50a9b5c). The previous merge conflict is resolved and the GitHub checks are green, but I still see a few items to fix before LGTM:

  1. The new default-provider test is not hermetic, and it exposes an explicit-vs-default mismatch for env-file based config. DatabaseSettings reads the auto-detected settings env file through settings_config() / _DEFAULT_ENV_FILE, but default_database_provider() and database_provider_explicitly_configured() only inspect os.environ. With a temporary HOME containing .powermem/.env and DATABASE_PROVIDER=oceanbase, DatabaseSettings().provider resolves to oceanbase while default_database_provider() resolves to sqlite, so tests/unit/test_seekdb_default_storage.py::test_database_settings_default_provider_matches_platform_helper fails. Please either make the test isolate settings env-file loading, or make the platform default/defaulted helpers use the same source of truth as DatabaseSettings.

  2. /api/v1/system/health is public, but it now returns raw startup_error text from service initialization. Those exceptions can include deployment details such as dependency errors, local paths, or database connection information. Please keep the public health response coarse/sanitized, and expose detailed startup errors only via authenticated status endpoints or logs.

  3. git diff --check origin/main...HEAD currently fails with src/server/utils/service_errors.py:19: new blank line at EOF.

Validation I ran:

  • Activated .venv and ran python -m pytest tests/unit/test_storage_default_provider.py tests/unit/test_seekdb_default_storage.py -q: 1 failed, 9 passed.
  • Activated .venv and ran the same tests with a temporary clean HOME: 10 passed.
  • Activated .venv and ran python -m pytest tests/unit/test_config_loader.py tests/unit/test_cli_config_show.py -q with a temporary clean HOME: 9 passed.
  • Activated .venv and ran python -m pytest tests/unit/server -q with a temporary clean HOME: 44 passed.
  • Activated .venv and ran python -m pip check: no broken requirements.

@knqiufan

Copy link
Copy Markdown
Contributor Author

Thanks @wayyoungboy. I addressed the three review items in the latest push (2a70a0f).

  1. The default-provider helpers now use the same configuration sources as DatabaseSettings: process env first, then the configured env files (POWERMEM_ENV_FILE / _DEFAULT_ENV_FILE). I also updated the defaulted calculation to use the same helper, so env-file based DATABASE_PROVIDER / OCEANBASE_HOST no longer diverges from DatabaseSettings.

  2. /api/v1/system/health no longer exposes raw startup exceptions. It only returns coarse readiness information; detailed startup errors remain available from the authenticated /api/v1/system/status endpoint and server logs.

  3. Fixed the trailing blank line reported by git diff --check.

Validation:

  • python -m pytest tests/unit/test_storage_default_provider.py tests/unit/test_seekdb_default_storage.py -q
  • python -m pytest tests/unit/server/test_system_health.py -q
  • python -m pytest tests/unit/test_config_loader.py tests/unit/test_cli_config_show.py -q
  • git diff --check origin/main

Please re-review when you have a chance.

@wayyoungboy

Copy link
Copy Markdown
Member

Rechecked the current PR state after the latest author update. GitHub currently reports this branch as conflicting with main, so final review/LGTM is still blocked until the branch is rebased or merged with the latest base branch.

The existing requested-changes review remains the active review state for now. After the conflict is resolved, please ping for a fresh review of the rebased diff and updated test results.

Resolve the setup documentation conflict and keep the storage fallback behavior aligned with the latest server and documentation changes.
@knqiufan

Copy link
Copy Markdown
Contributor Author

@wayyoungboy Thank you for the careful recheck and review.

I merged the latest main into this branch and pushed the conflict resolution in 6d4375c.

What changed in this update:

  • Resolved the apps/claude-code-plugin/SETUP.md conflict by keeping the latest server/MCP extras wording while preserving this PR's SQLite fallback note for platforms where embedded SeekDB is unavailable.
  • Rechecked the previous review items after the merge: env-file based default-provider resolution still uses the same source of truth as DatabaseSettings, and public /api/v1/system/health still returns only coarse readiness without exposing raw startup errors.
  • Fixed a Windows-specific server bind probe issue found while rerunning the server tests: the probe no longer sets SO_REUSEADDR on Windows, where it can incorrectly allow binding to an actively listened port.
  • Cleaned the trailing whitespace caught by git diff --check in the rebased/merged diff.

Validation:

  • python -m pytest tests/unit/test_storage_default_provider.py tests/unit/test_seekdb_default_storage.py -q -> 14 passed
  • python -m pytest tests/unit/server/test_system_health.py -q -> 1 passed
  • python -m pytest tests/unit/test_config_loader.py tests/unit/test_cli_config_show.py -q -> 9 passed
  • python -m pytest tests/unit/server -q -> 74 passed, 1 skipped
  • python -m pytest tests/unit/intelligence/test_importance_evaluator.py -q -> 17 passed
  • python -m pip check -> no broken requirements
  • git diff --check origin/main...HEAD -> clean

Please take another fresh look when convenient.

@wayyoungboy wayyoungboy left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rechecked the current head after the latest merge/rebase. The previously requested items appear addressed in the diff:

  • The platform default helper now reads the same explicit provider sources as DatabaseSettings (process env first, then configured env files), so env-file based DATABASE_PROVIDER / OCEANBASE_HOST no longer diverges from the default-provider decision.
  • Public /api/v1/system/health no longer exposes raw startup errors; the added test checks that sensitive-looking startup details are absent from the public health response.
  • The native dependency probe now handles modules with broken / missing specs as unavailable, which covers the CI stability issue around storage probes.
  • The latest status checks are green and GitHub reports the branch as mergeable.

I did not find a new blocking issue in this static re-review. I am not turning this into an approval in this pass because I did not run the PR test plan locally.

@lightzt99

Copy link
Copy Markdown
Collaborator

LGTM

@lightzt99
lightzt99 merged commit 3b282aa into oceanbase:main Jul 1, 2026
26 checks passed
@knqiufan
knqiufan deleted the feat/cross-platform-storage-default-fallback branch July 5, 2026 16:50
222twotwotwo pushed a commit to 222twotwotwo/powercontext that referenced this pull request Aug 23, 2026
…eanbase#993)

* fix(storage): fall back to sqlite when seekdb is unavailable

Centralize platform-aware storage default selection so zero-config startup uses embedded SeekDB only when available, and falls back to SQLite otherwise.

Expose memory service readiness and SQLite capability limitations through server status and the dashboard.

Closes oceanbase#992

* fix(ci): stabilize storage probes and extension build

* fix(storage): align defaults with env-file config
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Enhancement]: Auto-fallback to SQLite for zero-config startup when embedded seekdb is unavailable (non-Linux / missing pylibseekdb)

3 participants