TonAI RAG

A simple Anime QA Chatbot powered by DeepSeek-R1 and Ollama

Overview

A minimal RAG application to answer anime related questions.

Tech stack:

• Local LLM with Ollama.
• Hybrid search with BM25 and FAISS.
• Huggingface's text-embedding-inference for text embeddings and reranker.
• FastAPI as API interface.
• Docker and Docker Compose for containerization.
• Terraform to setup infrastructure

Dataset

mkdir data/

curl -L -o ./data/myanimelist-dataset.zip https://www.kaggle.com/api/v1/datasets/download/svanoo/myanimelist-dataset

Running the app

Preparation

We use uv for dependency management:

pip install uv

Install the app dependencies:

uv pip install -r pyproject.toml

Create embeddings for sementic search

First, run the server for embedding inference:

docker compose up vectorizer

Then run the ingest.py script:

cd rag/

python ingest.py

Running the app with docker-compose

docker compose up --build

Using the application

curl -X 'POST' \
  'http://localhost:8000/query' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "query": "Tell me about Hyouka"
}'