Streaming-AKS-Kubernetes-Logs-to-Azure-Blob-Storage-using-Vector

Step1: Login to your Azure portal and Provision an AKS cluster

Highlights of this project:

🚀Provisioned an AKS cluster using Azure portal or Azure CLI, enabling deployment and management of containerized applications in a highly available and scalable environment.

🚀Deployed an Nginx deployment in a test namespace of the AKS cluster, allowing testing of containerized applications and validating cluster functionality.

🚀Created an Azure blob storage account and container named "akslogs" to store application logs securely and reliably, facilitating log analysis and troubleshooting.

🚀Configured Vector, a high-performance log collector, to access the Azure blob storage container and stream application logs using a configuration file called "vector-config.yaml", ensuring proper storage and analysis of logs.

Azure CLI commands

az group create --name MyResourceGroup --location centralindia
az aks create -g MyResourceGroup -n myAKSCluster  --node-count 1 --generate-ssh-keys

Images:

Step 2: Log in to the cluster and create a deployment

Step 3: Create Azure blob storage to store the logs

commands

az storage account create \
   --name storageromesh2002 \
   --resource-group MyResourceGroup \
   --location centralindia \
   --sku Standard_ZRS \
   --encryption-services blob

az storage container create \
  --account-name storageromesh2002 \
  --name akslogs \

Step4: Storage Account Connection String

overview:

Step 5: Prepare vector-config.yaml

Step 6: Install Vector using Helm

command to install the vector

helm repo add vector https://helm.vector.dev
helm repo update
helm upgrade --install vector vector/vector -f vector-config.yaml -n test-ns --version 0.17.1

Step 7: The moment of truth!

After a few minutes see the logs streaming to the “akslogs” container in your Azure Blob Storage.

Overall, this project can help organizations effectively manage and leverage their AKS logs for various use cases, such as troubleshooting, performance optimization, security, compliance, and business intelligence.

Thank you!