PC_Algorithm_Microservice_Root_Cause_Analysis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "22ea4214",
   "metadata": {},
   "source": [
    "# PC Algorithm for Root Cause Analysis of Microservice Failure"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73b13b56",
   "metadata": {},
   "source": [
    "The Peter-Clark (PC) algorithm is one of the most general purpose algorithms for causal discovery that can be used for both tabular and time series data, of both continuous and discrete types. As proposed in CD-NOD [1], PC algorithm can be tailored for root cause analysis by treating the failure as an intervention on the root cause, and PC can use conditional independence tests to quickly detect it. Let us see how PC algorithm, with slight modifications on the PriorKnowledge sets, can be adapted for Root Cause Analysis for continous, microservice monitoring metrics data.\n",
    "\n",
    "References:\n",
    "\n",
    "[1] Huang, Biwei, Kun Zhang, Jiji Zhang, Joseph Ramsey, Ruben Sanchez-Romero, Clark Glymour, and Bernhard Schölkopf. \"Causal discovery from heterogeneous/nonstationary data.\" The Journal of Machine Learning Research 21, no. 1 (2020): 3482-3534.\n",
    "\n",
    "[2] Ikram, Azam, Sarthak Chakraborty, Subrata Mitra, Shiv Saini, Saurabh Bagchi, and Murat Kocaoglu. \"Root Cause Analysis of Failures in Microservices through Causal Discovery.\" Advances in Neural Information Processing Systems 35 (2022): 31158-31170."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "0422b961",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from scipy.stats import truncexpon, halfnorm\n",
    "from causalai.application import RootCauseDetector\n",
    "from causalai.application.common import rca_preprocess"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c17a1152",
   "metadata": {},
   "source": [
    "### Generate cloud monitoring metrics data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b71fd9b1",
   "metadata": {},
   "source": [
    "We create distribution shifts on the marginals/external noises of caching service, in which the anomaly will get propagated downwards to product service because of the causal graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "dad5751f",
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_observed_latency_data(unobserved_intrinsic_latencies):\n",
    "    observed_latencies = {}\n",
    "    observed_latencies['Product DB'] = unobserved_intrinsic_latencies['Product DB']\n",
    "    observed_latencies['Customer DB'] = unobserved_intrinsic_latencies['Customer DB']\n",
    "    observed_latencies['Order DB'] = unobserved_intrinsic_latencies['Order DB']\n",
    "    observed_latencies['Shipping Cost Service'] = unobserved_intrinsic_latencies['Shipping Cost Service']\n",
    "    observed_latencies['Caching Service'] = np.random.choice([0, 1], size=(len(observed_latencies['Product DB']),),\n",
    "                                                             p=[.5, .5]) * \\\n",
    "                                            observed_latencies['Product DB'] \\\n",
    "                                            + unobserved_intrinsic_latencies['Caching Service']\n",
    "    observed_latencies['Product Service'] = np.maximum(np.maximum(observed_latencies['Shipping Cost Service'],\n",
    "                                                                  observed_latencies['Caching Service']),\n",
    "                                                       observed_latencies['Customer DB']) \\\n",
    "                                            + unobserved_intrinsic_latencies['Product Service']\n",
    "\n",
    "    return pd.DataFrame(observed_latencies)\n",
    "\n",
    "\n",
    "def unobserved_intrinsic_latencies_normal(num_samples):\n",
    "    return {\n",
    "        'Product Service': halfnorm.rvs(size=num_samples, loc=0.1, scale=0.2),\n",
    "        'Shipping Cost Service': halfnorm.rvs(size=num_samples, loc=0.1, scale=0.2),\n",
    "        'Caching Service': halfnorm.rvs(size=num_samples, loc=0.1, scale=0.1),\n",
    "        'Order DB': truncexpon.rvs(size=num_samples, b=5, scale=0.2),\n",
    "        'Customer DB': truncexpon.rvs(size=num_samples, b=6, scale=0.2),\n",
    "        'Product DB': truncexpon.rvs(size=num_samples, b=10, scale=0.2)\n",
    "    }\n",
    "\n",
    "def unobserved_intrinsic_latencies_anomalous(num_samples):\n",
    "    return {\n",
    "        'Product Service': halfnorm.rvs(size=num_samples, loc=0.1, scale=0.2),\n",
    "        'Shipping Cost Service': halfnorm.rvs(size=num_samples, loc=0.1, scale=0.2),\n",
    "        'Caching Service': 2 + halfnorm.rvs(size=num_samples, loc=0.1, scale=0.1),\n",
    "        'Order DB': truncexpon.rvs(size=num_samples, b=5, scale=0.2),\n",
    "        'Customer DB': truncexpon.rvs(size=num_samples, b=6, scale=0.2),\n",
    "        'Product DB': truncexpon.rvs(size=num_samples, b=10, scale=0.2)\n",
    "    }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "8f105d1c",
   "metadata": {},
   "outputs": [],
   "source": [
    "normal_data = create_observed_latency_data(unobserved_intrinsic_latencies_normal(1000))\n",
    "outlier_data = create_observed_latency_data(unobserved_intrinsic_latencies_anomalous(1000))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "7204d48c",
   "metadata": {},
   "outputs": [],
   "source": [
    "lower_level_columns = ['Customer DB', 'Shipping Cost Service', 'Caching Service', 'Product DB']\n",
    "upper_level_metric = normal_data['Product Service'].tolist() + outlier_data['Product Service'].tolist()\n",
    "outlier_data = outlier_data[lower_level_columns]\n",
    "normal_data = normal_data[lower_level_columns]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "09afe737",
   "metadata": {},
   "outputs": [],
   "source": [
    "data_obj, var_names = rca_preprocess(\n",
    "    data=[normal_data, outlier_data],\n",
    "    time_metric=upper_level_metric,\n",
    "    time_metric_name='time'\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9456c5b",
   "metadata": {},
   "source": [
    "### Run root cause analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "45542449",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = RootCauseDetector(\n",
    "    data_obj = data_obj,\n",
    "    var_names=var_names,\n",
    "    time_metric_name='time',\n",
    "    prior_knowledge=None\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "d543cdbe",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The root cause(s) of the incident are: {'Caching Service'}\n"
     ]
    }
   ],
   "source": [
    "root_causes, graph = model.run(\n",
    "    pvalue_thres=0.001,\n",
    "    max_condition_set_size=4,\n",
    "    return_graph=True\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "592aa297",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'Caching Service'}\n"
     ]
    }
   ],
   "source": [
    "print(root_causes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "6898eb41",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'Customer DB': set(), 'Shipping Cost Service': set(), 'Caching Service': set(), 'Product DB': set(), 'time': {'Caching Service'}}\n"
     ]
    }
   ],
   "source": [
    "print(graph)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "59747165",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "thesis",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.19"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}