Skip to content
Draft
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 43 additions & 6 deletions readme.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,44 @@
run:
cp .env.example .env
# LLM Infrastructure on Oracle Cloud

edit .env and fill values
set -a; source .env; set +a
terraform init
terraform apply -auto-approve
**Terraform infrastructure-as-code to deploy a free-tier LLM server on Oracle Cloud Infrastructure (OCI)**

This project automatically provisions and configures an ARM64 virtual machine on OCI's Always Free tier, running the [llama.cpp](https://github.com/ggml-org/llama.cpp) server with the **TinyLlama-1.1B-Chat-v1.0** model (GGUF format, Q4_K_M quantization).

## 🚀 What It Does

- Provisions a complete OCI infrastructure (VCN, subnet, security groups, compute instance)
- Uses OCI's free-tier **VM.Standard.A1.Flex** ARM64 shape (4 OCPUs, 24GB RAM)
- Automatically installs and configures llama.cpp
- Downloads and deploys the TinyLlama-1.1B model
- Starts an LLM inference server on port 8000
- Ready for REST API requests and chat interactions

## 🛠️ Technologies Used

- **Terraform** - Infrastructure as Code
- **Oracle Cloud Infrastructure (OCI)** - Cloud platform (free tier)
- **llama.cpp** - High-performance LLM inference engine
- **TinyLlama-1.1B-Chat-v1.0** - Lightweight chat model (GGUF Q4_K_M)
- **cloud-init** - Automated instance configuration

## 📋 Quick Start

1. **Copy and configure environment variables:**
```bash
cp .env.example .env
```

2. **Edit `.env` and fill in your OCI credentials and settings**

3. **Load environment variables:**
```bash
set -a; source .env; set +a
```

4. **Deploy infrastructure:**
```bash
terraform init
terraform apply -auto-approve
```

After deployment, the LLM server will be accessible at `http://<instance-public-ip>:8000`