demo.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## testing the code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "## maciej's code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/maciejkozubal/miniconda3/envs/chat/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n"
     ]
    }
   ],
   "source": [
    "import openai\n",
    "import os\n",
    "from dotenv import load_dotenv, find_dotenv\n",
    "from IPython.display import display, Markdown\n",
    "from langchain.chat_models import ChatOpenAI\n",
    "from langchain.document_loaders import PyPDFLoader\n",
    "from langchain.embeddings.openai import OpenAIEmbeddings\n",
    "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
    "from langchain.vectorstores import DocArrayInMemorySearch\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "_ = load_dotenv(find_dotenv()) # read local .env file\n",
    "The line `openai.api_key = os.environ['OPENAI_API_KEY']` is setting the API key for the OpenAI library. It is retrieving the API key from the environment variable `OPENAI_API_KEY` and assigning it to the `api_key` attribute of the `openai` object. This allows the code to authenticate and make API calls to the OpenAI service using the provided API key.\n",
    "openai.api_key = os.environ['OPENAI_API_KEY']\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Load file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Document(page_content='90 3. Linear Regression\\n0 50 100 150200 600 1000 1400IncomeBalance0 50 100 150200 600 1000 1400IncomeBalancestudentnon−student\\nFIGURE 3.7.For theCreditdata, the least squares lines are shown for pre-diction ofbalancefromincomefor students and non-students.Left:The model(3.34) was ﬁt. There is no interaction betweenincomeandstudent.Right:Themodel (3.35) was ﬁt. There is an interaction term betweenincomeandstudent.takes the formbalancei≈β0+β1×incomei+/braceleft⎪iggβ2ifith person is a student0 ifith person is not a student=β1×incomei+/braceleft⎪iggβ0+β2ifith person is a studentβ0ifith person is not a student.(3.34)Notice that this amounts to ﬁtting two parallel lines to the data, one forstudents and one for non-students. The lines for students and non-studentshave diﬀerent intercepts,β0+β2versusβ0, but the same slope,β1. Thisis illustrated in the left-hand panel of Figure 3.7. The fact that the linesare parallel means that the average eﬀect onbalanceof a one-unit increaseinincomedoes not depend on whether or not the individual is a student.This represents a potentially serious limitation of the model, since in fact achange inincomemay have a very diﬀerent eﬀect on the credit card balanceof a student versus a non-student.This limitation can be addressed by adding an interaction variable, cre-ated by multiplyingincomewith the dummy variable forstudent. Ourmodel now becomesbalancei≈β0+β1×incomei+/braceleft⎪iggβ2+β3×incomeiif student0 if not student=/braceleft⎪igg(β0+β2)+(β1+β3)×incomeiif studentβ0+β1×incomeiif not student.(3.35)', metadata={'source': 'data/pdfs/ISLRv2_website.pdf', 'page': 99})"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "file = 'data/pdfs/ISLRv2_website.pdf'\n",
    "\n",
    "loader = PyPDFLoader(file)\n",
    "document = loader.load()\n",
    "document[99]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Document(page_content='FIGURE 2.9.Left:Data simulated fromf, shown in black. Three estimates offare shown: the linear regression line (orange curve), and two smoothing splineﬁts (blue and green curves).Right:Training MSE (grey curve), test MSE (redcurve), and minimum possible test MSE over all methods (dashed line). Squaresrepresent the training and test MSEs for the three ﬁts shown in the left-handpanel.since the training MSE and the test MSE appear to be closely related.Unfortunately, there is a fundamental problem with this strategy: thereis no guarantee that the method with the lowest training MSE will alsohave the lowest test MSE. Roughly speaking, the problem is that manystatistical methods speciﬁcally estimate coeﬃcients so as to minimize thetraining set MSE. For these methods, the training set MSE can be quitesmall, but the test MSE is often much larger.Figure 2.9 illustrates this phenomenon on a simple example. In the left-hand panel of Figure 2.9, we have generated observations from (2.1) withthe truefgiven by the black curve. The orange, blue and green curves illus-trate three possible estimates forfobtained using methods with increasinglevels of ﬂexibility. The orange line is the linear regression ﬁt, which is rela-tively inﬂexible. The blue and green curves were produced usingsmoothingsplines, discussed in Chapter 7, with diﬀerent levels of smoothness. It issmoothingsplineclear that as the level of ﬂexibility increases, the curves ﬁt the observeddata more closely. The green curve is', metadata={'source': 'data/pdfs/ISLRv2_website.pdf', 'page': 41})"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text_splitter = RecursiveCharacterTextSplitter(chunk_size=1500, chunk_overlap=150)\n",
    "splits = text_splitter.split_documents(document)\n",
    "splits[99]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Convert the pages into embeddings and store in vector store"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "embeddings = OpenAIEmbeddings()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-08-03 17:27:45.127552: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F AVX512_VNNI FMA\n",
      "To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n",
      "2023-08-03 17:27:48.544472: I tensorflow/core/util/port.cc:104] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.\n",
      "2023-08-03 17:27:54.823896: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory\n",
      "2023-08-03 17:27:54.824441: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory\n",
      "2023-08-03 17:27:54.824471: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.\n"
     ]
    }
   ],
   "source": [
    "db = DocArrayInMemorySearch.from_documents(\n",
    "    splits, \n",
    "    embeddings\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Query a store"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pose a query, find pages similar to query"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "query = \"\"\"What's the most complicated ML model\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(page_content='426 10. Deep LearningOℓ. We use the ﬁnal valueOLto predict the response: the sentiment of thereview.This is a simple RNN, and has relatively few parameters. If there areKhidden units, the common weight matrixWhasK×(m+ 1) parameters,the matrixUhasK×Kparameters, andBhas 2(K+ 1) for the two-classlogistic regression as in (10.15). These are used repeatedly as we processthe sequenceX={Xℓ}L1from left to right, much like we use a singleconvolution ﬁlter to process each patch in an image (Section 10.3.1). If theembedding layerEis learned, that adds an additionalm×Dparameters(D= 10,000 here), and is by far the biggest cost.We ﬁt the RNN as described in Figure 10.12 and the accompaying text totheIMDbdata. The model had an embedding matrixEwithm= 32 (whichwas learned in training as opposed to precomputed), followed by a singlerecurrent layer withK= 32 hidden units. The model was trained withdropout regularization on the 25,000 reviews in the designated trainingset, and achieved a disappointing 76% accuracy on theIMDbtest data. Anetwork using theGloVepretrained embedding matrixEperformed slightlyworse.For ease of exposition we have presented a very simple RNN. More elab-orate versions uselong termandshort termmemory (LSTM). Two tracksof hidden-layer activations are maintained, so that when the activationAℓis computed, it gets input from hidden units both further back in time,and closer in time — a so-calledLSTM RNN. With long sequences, thisLSTM RNNovercomes the problem of early signals', metadata={'source': 'data/pdfs/ISLRv2_website.pdf', 'page': 432}),\n",
       " Document(page_content='408 10. Deep Learningkind are relatively simple for humans. Our visual system occupies a largefraction of our brains, and good recognition is an evolutionary force forsurvival. These tasks are not so simple for machines, and it has taken morethan 30 years to reﬁne the neural-network architectures to match humanperformance.Figure 10.4 shows a multilayer network architecture that works well forsolving the digit-classiﬁcation task. It diﬀers from Figure 10.1 in severalways:•It has two hidden layersL1(256 units) andL2(128 units) ratherthan one. Later we will see a network with seven hidden layers.•It has ten output variables, rather than one. In this case the ten vari-ables really represent a single qualitative variable and so are quitedependent. (We have indexed them by the digit class 0–9 rather than1–10, for clarity.) More generally, inmulti-task learningone can pre-multi-tasklearningdict diﬀerent responses simultaneously with a single network; they allhave a say in the formation of the hidden layers.•The loss function used for training the network is tailored for themulticlass classiﬁcation task.The ﬁrst hidden layer is as in (10.2), withA(1)k=h(1)k(X)=g(w(1)k0+/summationtextpj=1w(1)kjXj)(10.10)fork=1,...,K1. The second hidden layer treats the activationsA(1)kofthe ﬁrst hidden layer as inputs and computes new activationsA(2)ℓ=h(2)ℓ(X)=g(w(2)ℓ0+/summationtextK1k=1w(2)ℓkA(1)k)(10.11)forℓ=1,...,K2.Notice that each of the activations in the second layerA(2)ℓ=h(2)ℓ(X) is a', metadata={'source': 'data/pdfs/ISLRv2_website.pdf', 'page': 414}),\n",
       " Document(page_content='10Deep Learning', metadata={'source': 'data/pdfs/ISLRv2_website.pdf', 'page': 409}),\n",
       " Document(page_content='since the model was selected on the training data, therewould beselection bias. Instead, we reﬁt the model on the test data, whichwas not used in the selection. Table 10.3 shows the results.We have a number of very powerful tools at our disposal, including neu-ral networks, random forests and boosting, support vector machines andgeneralized additive models, to name a few. And then we have linear mod-els, and simple variants of these. When faced with new data modeling andprediction problems, its tempting to always go for the trendy new methods.Often they give extremely impressive results, especially when the datasetsare very large and can support the ﬁtting of high-dimensional nonlinearmodels. However,ifwe can produce models with the simpler tools thatperform as well, they are likely to be easier to ﬁt and understand, and po-tentially less fragile than the more complex approaches. Wherever possible,it makes sense to try the simpler models as well, and then make a choicebased on the performance/complexity tradeoﬀ.Typically we expect deep learning to be an attractive choice when thesample size of the training set is extremely large, and when interpretabilityof the model is not a high priority.', metadata={'source': 'data/pdfs/ISLRv2_website.pdf', 'page': 439})]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "similar_pages = db.similarity_search(query, limit=10)\n",
    "similar_pages"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Retriever"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "retriever = db.as_retriever()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Query the stored pages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "llm = ChatOpenAI(temperature = 0.0)\n",
    "\n",
    "qpages = \"\".join([similar_pages[i].page_content for i in range(len(similar_pages))])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = llm.call_as_llm(f\"{qpages} Question: {query}\") \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "The most complicated machine learning model is difficult to determine as it depends on various factors such as the problem being solved, the size of the dataset, and the complexity of the features. However, deep learning models, such as deep neural networks with multiple hidden layers, are generally considered to be more complex compared to other machine learning models. These models have a large number of parameters and require significant computational resources for training. Other complex models include recurrent neural networks (RNNs), convolutional neural networks (CNNs), and generative adversarial networks (GANs)."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(Markdown(response))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}