diff --git a/readme.md b/readme.md index 2603569..caf7b39 100644 --- a/readme.md +++ b/readme.md @@ -1,7 +1,44 @@ -run: -cp .env.example .env +# LLM Infrastructure on Oracle Cloud -edit .env and fill values -set -a; source .env; set +a -terraform init -terraform apply -auto-approve +**Terraform infrastructure-as-code to deploy a free-tier LLM server on Oracle Cloud Infrastructure (OCI)** + +This project automatically provisions and configures an ARM64 virtual machine on OCI's Always Free tier, running the [llama.cpp](https://github.com/ggml-org/llama.cpp) server with the **TinyLlama-1.1B-Chat-v1.0** model (GGUF format, Q4_K_M quantization). + +## 🚀 What It Does + +- Provisions a complete OCI infrastructure (VCN, subnet, security groups, compute instance) +- Uses OCI's free-tier **VM.Standard.A1.Flex** ARM64 shape (4 OCPUs, 24GB RAM) +- Automatically installs and configures llama.cpp +- Downloads and deploys the TinyLlama-1.1B model +- Starts an LLM inference server on port 8000 +- Ready for REST API requests and chat interactions + +## 🛠️ Technologies Used + +- **Terraform** - Infrastructure as Code +- **Oracle Cloud Infrastructure (OCI)** - Cloud platform (free tier) +- **llama.cpp** - High-performance LLM inference engine +- **TinyLlama-1.1B-Chat-v1.0** - Lightweight chat model (GGUF Q4_K_M) +- **cloud-init** - Automated instance configuration + +## 📋 Quick Start + +1. **Copy and configure environment variables:** + ```bash + cp .env.example .env + ``` + +2. **Edit `.env` and fill in your OCI credentials and settings** + +3. **Load environment variables:** + ```bash + set -a; source .env; set +a + ``` + +4. **Deploy infrastructure:** + ```bash + terraform init + terraform apply -auto-approve + ``` + +After deployment, the LLM server will be accessible at `http://:8000`