Vectorize semantic projection with embedding-profile and visibility gates

[burakdede]

Parent epic: #8 (post-v0.1 backlog)

Context

§17.1 and §24.12 are emphatic: a remote semantic target is allowed only when it can generate query embeddings matching the build's complete EmbeddingProfile, proven by fixed-string compatibility fixtures within a documented cosine tolerance. §1.5 additionally records that Vectorize acknowledges durable writes before they become query-visible (ref [27]), so activation must wait for visibility.

Post-v0.1 and gated on both the semantic package and hard compatibility proof.

Spec references

§17.1 (post-v0.1 semantic extension), §17.4/§17.5 projection and activation, §24.12, §1.5 audit findings, refs [22][27].

Scope

In: the Vectorize projection adapter: embedding-profile compatibility gate, build-scoped namespaces, batched writes, query-visibility polling with a deadline, and capability advertisement rules.
Out: changing the v0.1 Cloudflare target's lexical-only behavior.

Design notes

• Compatibility is proven, not assumed: fixed-string fixtures are embedded locally and remotely and compared within a documented cosine tolerance before any projection is allowed (§17.1).
• Vectors are written into build-scoped namespaces in large batches; verification polls a deterministic candidate query through the runtime query path until all expected records are visible or a deadline expires, a timeout leaves the candidate inactive (§17.5 step 5).
• Capability loss is never silent: a semantic build deployed to a lexical-only target still fails by default (§16.3).

Acceptance criteria

• Compatibility gate: model ID, immutable revision, tokenizer, pooling, normalization, value type, and dimensions must all match before projection proceeds.
• Fixed-string compatibility fixtures pass within the documented cosine tolerance; failure blocks projection with a precise report.
• Vectors written under build-scoped namespaces in batched operations with progress reporting.
• Query-visibility polling through the real runtime query path; the deadline leaves the candidate inactive and emits a resumable receipt.
• The target advertises semantic-search only after visibility verification succeeds.
• Retention cleanup removes candidate namespaces for discarded builds without touching retained ones.
• Adapter runs the §20.6 contract suite plus the delayed-write visibility tests.

Tests / verification

• Compatibility fixture tests including a deliberate profile mismatch that must be refused.
• Visibility-polling tests with simulated write lag and a timeout case.
• Contract suite against the Vectorize-enabled target.
• End-to-end deploy/verify/activate/rollback in the integration environment.

Dependencies

Blocked by the semantic package issue and Cloudflare Phase 6 completion.

Definition of Done

Remote semantic retrieval is possible only when it is provably the same vector space, and never activated before it is actually queryable.