Skip to content

Commit

Permalink
docs: update README with optimal context values
Browse files Browse the repository at this point in the history
  • Loading branch information
lsorber committed Dec 4, 2024
1 parent daa1f37 commit f660991
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ my_config = RAGLiteConfig(
my_config = RAGLiteConfig(
db_url="sqlite:///raglite.sqlite",
llm="llama-cpp-python/bartowski/Meta-Llama-3.1-8B-Instruct-GGUF/*Q4_K_M.gguf@8192",
embedder="llama-cpp-python/lm-kit/bge-m3-gguf/*F16.gguf@4096",
embedder="llama-cpp-python/lm-kit/bge-m3-gguf/*F16.gguf@1024", # A context size of 1024 tokens is the sweet spot for bge-m3.
)
```

Expand Down Expand Up @@ -281,7 +281,7 @@ You can specify the database URL, LLM, and embedder directly in the Chainlit fro
raglite chainlit \
--db_url sqlite:///raglite.sqlite \
--llm llama-cpp-python/bartowski/Llama-3.2-3B-Instruct-GGUF/*Q4_K_M.gguf@4096 \
--embedder llama-cpp-python/lm-kit/bge-m3-gguf/*F16.gguf@4096
--embedder llama-cpp-python/lm-kit/bge-m3-gguf/*F16.gguf@1024
```

To use an API-based LLM, make sure to include your credentials in a `.env` file or supply them inline:
Expand Down

0 comments on commit f660991

Please sign in to comment.