Skip to content

fix(storage): enable FTS fallback for NoopEmbedding across core and adapter layers - #1091

Closed
lightzt99 wants to merge 3 commits into
oceanbase:mainfrom
lightzt99:fix/embedding-none-fts-fallback
Closed

fix(storage): enable FTS fallback for NoopEmbedding across core and adapter layers#1091
lightzt99 wants to merge 3 commits into
oceanbase:mainfrom
lightzt99:fix/embedding-none-fts-fallback

Conversation

@lightzt99

@lightzt99 lightzt99 commented Jun 25, 2026

Copy link
Copy Markdown
Collaborator

Summary

When EMBEDDING_PROVIDER=none (NoopEmbedding, added in #1058), memory search returned [] unconditionally. Two layers blocked the FTS fallback:

  1. Core layerMemory.search() and AsyncMemory.search() short-circuited to {"results": [], "relations": []} as soon as _is_embedding_disabled() was true, before the adapter ever saw the request.
  2. Adapter layerStorageAdapter.search_memories() returned [] when query_embedding was falsy, and _supports_text_search_without_vector() gated OceanBase FTS fallback behind the hybrid_search flag, so deployments with hybrid_search=False had no text-only path even when query was present.

This PR removes both blockers so the request flows through to the storage layer's FTS path when embedding is disabled, giving users keyword-based recall instead of silent empty results.

Changes

Core layer

  • src/powermem/core/memory.py — drop the _is_embedding_disabled() early-return in Memory.search(); keep query_embedding = None and let the adapter handle the fallback. Logs Embedding disabled; falling back to FTS for query for visibility.
  • src/powermem/core/async_memory.py — same change in AsyncMemory.search().

Adapter layer

  • src/powermem/storage/adapter.py_supports_text_search_without_vector() now returns True for OceanBase unconditionally (no longer gated by hybrid_search). The existing FTS fallback path in search_memories() (added in Fix memory search fallback when embeddings are unavailable #1052) already forwards query_vector=None when query_embedding is empty and query is present, so SQLite and OceanBase both route to full-text search.
  • src/powermem/storage/oceanbase/oceanbase.pysearch() now branches to _fulltext_search() when vectors is None and query is present, so OceanBase deployments with hybrid_search=False also get FTS fallback (previously the not self.hybrid_search branch would call _vector_search with no vectors and return []).

Tests

  • tests/integration/test_noop_embedding_mode.py — renamed test_noop_embedding_search_returns_emptytest_noop_embedding_search_returns_fts_hits; asserts the FTS hit for both sync Memory and async AsyncMemory.
  • tests/unit/test_adapter_fts_fallback.py — 5 unit tests covering:
    • Empty list embedding + query text → FTS hits
    • None embedding + query text → FTS hits
    • Empty embedding + no query → []
    • Unrelated query → []
    • FTS respects user_id scoping

Validation

  • pytest tests/integration/test_noop_embedding_mode.py tests/unit/test_noop_embedding.py tests/unit/test_adapter_fts_fallback.py tests/unit/test_oceanbase_fts_quality_score.py tests/unit/test_sqlite_fts.py tests/unit/test_memory.py -q — 130 passed
  • No regressions in test_list_memory_filters.py, test_search_score_threshold.py, etc. (138 related unit tests pass)

Behavior matrix

Configuration Before After
SQLite + EMBEDDING_PROVIDER=none + query text [] (core short-circuit) FTS hits
OceanBase + hybrid_search=True + none + query text [] (core short-circuit) FTS hits
OceanBase + hybrid_search=False + none + query text [] (core + adapter + store all block) FTS hits
Any backend + none + no query text [] [] (degenerate, unchanged)
Real embedding + query text Vector / hybrid search Vector / hybrid search (unchanged)

Notes

  • OceanBase hybrid_search=False deployments now also get FTS fallback. Previously the not self.hybrid_search branch in search() would call _vector_search(query, vectors=None, ...) and return []; the new vectors is None and query branch routes to _fulltext_search first.
  • The skill store path (add_skill / search_skills) was already correct and is not touched here — Memory._embed() returns None (not []) when _is_embedding_disabled() is true, so OceanBaseSkillStore.search(query_embedding=None, query_text=...) already skipped vector search.
  • Rebased onto upstream/main (71003f5) to pick up feat(cli): add profile, batch, import/export, quality, optimize commands #1092 and fix(config): honor disabled graph store flag #1130.

@lightzt99
lightzt99 force-pushed the fix/embedding-none-fts-fallback branch from 479bb19 to 1882db3 Compare June 26, 2026 07:58
@wayyoungboy

Copy link
Copy Markdown
Member

Thanks for the PR. I cannot mark this ready yet because both CI test jobs are currently failing:

  • test (3.11)
  • test (3.12)

Both fail in tests/integration/test_noop_embedding_mode.py::test_noop_embedding_search_returns_empty: the test still expects an empty result in noop embedding mode, while this change now returns an FTS hit for the added memory. Please reconcile the intended noop/EMBEDDING_PROVIDER=none search contract before merge, either by updating the test and documenting automatic FTS fallback as the new default behavior, or by keeping default search empty and requiring an explicit FTS retrieval mode. This also overlaps with #1052, which adds explicit FTS retrieval controls, so it would be good to align the two PRs before one lands.

@wayyoungboy wayyoungboy left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes because the current CI failures line up with the behavior change in this PR.

Both test (3.11) and test (3.12) fail in tests/integration/test_noop_embedding_mode.py::test_noop_embedding_search_returns_empty: the existing integration contract expects default search to return no results when embeddings are explicitly disabled, but this PR now returns an FTS hit.

Please settle the intended contract before merge: either document/update tests so automatic FTS fallback is the new default for EMBEDDING_PROVIDER=none, or keep default no-embedding search empty and require an explicit FTS retrieval mode. This should also be aligned with #1052 before either path lands.

lightzt99 added a commit to lightzt99/powermem that referenced this pull request Jun 27, 2026
The integration test asserted search() returns [] when EMBEDDING_PROVIDER=none,
but PR oceanbase#1091 changes that contract to fall back to FTS5 keyword search when
query_embedding is empty and query text is present. Update the test to expect
the FTS hit and reflect the new behavior in the module docstring.

@wayyoungboy wayyoungboy left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rechecked the current head. The Python checks are green now, but the original contract issue is still unresolved: this PR changes default search with EMBEDDING_PROVIDER=none to automatic FTS fallback and updates the integration test to expect a hit.

That still conflicts with the direction in #1052, where explicit no-embedding mode stays empty by default and full-text lookup is exposed through an explicit retrieval mode such as retrieval_mode="fts". Please align these two PRs before merge. A clean path would be to drop the automatic default fallback here, wait for #1052 to land, and add the complementary explicit retrieval-mode exposure where this PR needs it.

Keeping changes requested until there is one consistent public search contract for no-embedding mode.

lightzt99 added 3 commits July 2, 2026 15:35
Cover the search_memories() paths when NoopEmbedding returns an empty
vector: FTS results returned when query text is present, empty result
when both embedding and query are missing, unrelated-query filtering,
and user_id scoping.
The integration test asserted search() returns [] when EMBEDDING_PROVIDER=none,
but PR oceanbase#1091 changes that contract to fall back to FTS5 keyword search when
query_embedding is empty and query text is present. Update the test to expect
the FTS hit and reflect the new behavior in the module docstring.
When EMBEDDING_PROVIDER=none (NoopEmbedding), the core layer short-circuited
search() to return [] before the adapter's FTS fallback could run, so
queries on stored memories silently returned no hits.

Remove the early-return in Memory.search() and AsyncMemory.search() so
query_embedding stays None and the request flows through to the storage
adapter. The adapter already falls back to text-only search when the
target store supports it.

On the OceanBase side, _supports_text_search_without_vector() no longer
gates on the hybrid_search flag, and OceanBaseVectorStore.search() now
routes to _fulltext_search() when vectors is None and a query is
present, so deployments with hybrid_search=False also get FTS fallback.

The integration test that固化ed the old "returns empty" contract now
asserts the FTS hit for both sync and async paths.
@lightzt99
lightzt99 force-pushed the fix/embedding-none-fts-fallback branch from 01c9ec8 to d7b51de Compare July 2, 2026 12:25
@lightzt99 lightzt99 changed the title fix(storage): fall back to FTS when EMBEDDING_PROVIDER=none fix(storage): enable FTS fallback for NoopEmbedding across core and adapter layers Jul 3, 2026

Copy link
Copy Markdown
Member

Thanks for continuing on the NoopEmbedding fallback work. I do not think this is ready to merge yet because the public search contract is still unresolved.

This PR changes EMBEDDING_PROVIDER=none default search behavior to automatically return FTS hits. That conflicts with the direction discussed in #1052, where no-embedding search stays empty by default and FTS is exposed through an explicit retrieval mode such as retrieval_mode="fts".

Please align this PR with that contract before merge: either drop the automatic default fallback here and wait for the explicit retrieval-mode path, or get maintainer agreement to change and document the default behavior.

@Teingi

Teingi commented Aug 10, 2026

Copy link
Copy Markdown
Member

Thank you for your contribution! PowerMem has evolved into PowerContext as part of a major project upgrade, so we’re closing this PR for now. We truly appreciate your work and would be happy to revisit it in PowerContext if it is still relevant.

@Teingi Teingi closed this Aug 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants