feat(search): make the embedding model configurable (#83)

Co-authored-by: Harlan Wilton <harlan@harlanzw.com>

Author: mrrobertkent

README.md

@@ -239,6 +239,59 @@ Generation runs locally: free, offline, no API key. Unlike the CLI and API backe
 
 The large default context can exceed memory for big models on constrained hardware (Ollama returns a 500). Lower `OLLAMA_NUM_CTX` or pick a smaller model if generation fails to load.
 
+### Embedding Model
+
+`skilld search` uses a local embedding model. It runs offline through transformers.js. It needs no API key or network after the first download. Pick one under **Embedding model** in `skilld config`:
+
+| Model | Dimensions | Notes |
+|-------|-----------:|-------|
+| `bge-small-en-v1.5` | 384 | Default. Fastest to index, smallest download. |
+| `bge-base-en-v1.5` | 768 | Balanced accuracy and speed. |
+| `bge-m3` | 1024 | Multilingual, 8192-token context. |
+
+Larger models retrieve more accurately but cost more time and memory when indexing. Set `SKILLD_EMBED_MODEL` to override the saved setting:
+
+```bash
+export SKILLD_EMBED_MODEL=bge-m3
+skilld update --force
+```
+
+Each search index belongs to one model and device. Keep environment overrides set for both indexing and querying. Rebuild indexes after either setting changes:
+
+```bash
+skilld update --force
+```
+
+### Embedding Device
+
+The embedding model runs on the CPU by default. **Embedding device** in `skilld config` moves it onto a GPU backend, which can be substantially faster:
+
+| Device | Notes |
+|--------|-------|
+| `auto` | Default. Lets transformers.js choose, CPU under Node. |
+| `cpu` | Always available, predictable. |
+| `webgpu` | Fastest on Apple Silicon in testing. |
+| `coreml` | Apple Neural Engine. Measured slower than CPU for these models. |
+
+Measured on an Apple M5 Max, 120 documents, best of 3 after warm-up (docs/sec):
+
+| Model | `cpu` | `coreml` | `webgpu` |
+|-------|------:|---------:|---------:|
+| `bge-small-en-v1.5` | 664 | 198 | **1713** |
+| `bge-base-en-v1.5` | 198 | 68 | **580** |
+| `Xenova/bge-large-en-v1.5` | 71 | 9 | **201** |
+
+WebGPU was 2.6 to 2.9 times faster than CPU at every size. `bge-large` on WebGPU indexed faster than `bge-base` on CPU. CoreML was consistently slower.
+
+The ranking is hardware-specific, so benchmark before trusting a device on other machines. Set `SKILLD_EMBED_DEVICE` to override the saved setting:
+
+```bash
+export SKILLD_EMBED_DEVICE=cpu
+skilld update --force
+```
+
+If a backend is unavailable, indexing fails to start. Switch back to `auto`.
+
 ### Eject
 
 Export a skill as a portable, self-contained directory for sharing via git repos:

pnpm-lock.yaml

@@ -2,6 +2,7 @@ catalogMode: prefer
 minimumReleaseAgeExclude:
   - verkit@0.2.0
   - '@mdream/rust-wasm32-wasi@1.5.12'
+  - retriv@0.15.0
 shellEmulator: true
 
 trustPolicy: no-downgrade
@@ -23,7 +24,7 @@ catalog:
   ofetch: ^1.5.1
   pathe: ^2.0.3
   publint: ^0.3.23
-  retriv: ^0.14.7
+  retriv: ^0.15.0
   std-env: ^4.2.0
   tsx: ^4.23.8
   typebox: ^1.3.10