docs/langchain-retrieval-agent.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "TXC2wBpCU9f7"
   },
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pinecone-io/examples/blob/master/docs/langchain-retrieval-agent.ipynb) [![Open nbviewer](https://raw.githubusercontent.com/pinecone-io/examples/master/assets/nbviewer-shield.svg)](https://nbviewer.org/github/pinecone-io/examples/blob/master/docs/langchain-retrieval-agent.ipynb)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "bhWwrfbbVGOA"
   },
   "source": [
    "#### [LangChain Handbook](https://www.pinecone.io/learn/series/langchain/)\n",
    "\n",
    "# Retrieval Agents\n",
    "\n",
    "We've seen in previous chapters how powerful [retrieval augmentation](https://www.pinecone.io/learn/series/langchain/langchain-retrieval-augmentation/) and [conversational agents](https://www.pinecone.io/learn/series/langchain/langchain-agents/) can be. They become even more impressive when we begin using them together.\n",
    "\n",
    "Conversational agents can struggle with data freshness, knowledge about specific domains, or accessing internal documentation. By coupling agents with retrieval augmentation tools we no longer have these problems.\n",
    "\n",
    "One the other side, using \"naive\" retrieval augmentation without the use of an agent means we will retrieve contexts with *every* query. Again, this isn't always ideal as not every query requires access to external knowledge.\n",
    "\n",
    "Merging these methods gives us the best of both worlds. In this notebook we'll learn how to do this.\n",
    "\n",
    "[![Open full notebook](https://raw.githubusercontent.com/pinecone-io/examples/master/assets/full-link.svg)](https://colab.research.google.com/github/pinecone-io/examples/blob/master/learn/generation/langchain/handbook/08-langchain-retrieval-agent.ipynb)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Prerequisites\n",
    "\n",
    "To begin, we must install several libraries that we will be using in this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "pva9ehKXUpU2",
    "outputId": "d74ebf63-a115-48fb-f950-71d8150a691c"
   },
   "outputs": [],
   "source": [
    "!pip install -qU \\\n",
    "  pinecone==5.4.2 \\\n",
    "  pinecone-datasets==1.0.2 \\\n",
    "  pinecone-notebooks==0.1.1 \\\n",
    "  langchain==0.3.20 \\\n",
    "  langchain-openai==0.3.9 \\\n",
    "  langchain-pinecone==0.2.3 \\\n",
    "  langgraph==0.3.14 \\\n",
    "  tqdm"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "ZTgrOQziXUto"
   },
   "source": [
    "## Building the Knowledge Base"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "qNyRsz0ZXXaq"
   },
   "source": [
    "For this demonstration, we will download a pre-embedded dataset using `pinecone-datasets`. This will allow us to skip the data preparation steps, if you'd rather work through those steps you can find the [full notebook here](https://colab.research.google.com/github/pinecone-io/examples/blob/master/learn/generation/langchain/handbook/08-langchain-retrieval-agent.ipynb).\n",
    "\n",
    "We will be using embeddings prepared from a subset of the [Stanford Question Answering Dataset (SQuAD)](https://huggingface.co/datasets/rajpurkar/squad). SQuAD is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.\n",
    "\n",
    "In this demo, we will use context about each topic from the dataset and incorporate that into a chat retrieval agent's knowledge base.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 250
    },
    "id": "laSDMjqQXuj-",
    "outputId": "2a05a716-73db-4da4-e19b-4378290ef2a9"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Loading documents parquet files: 100%|██████████| 1/1 [00:35<00:00, 35.11s/it]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>values</th>\n",
       "      <th>sparse_values</th>\n",
       "      <th>metadata</th>\n",
       "      <th>blob</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5733be284776f41900661182</td>\n",
       "      <td>[-0.010262451963272523, 0.02222637996192584, -...</td>\n",
       "      <td>None</td>\n",
       "      <td>{'text': 'Architecturally, the school has a Ca...</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5733bf84d058e614000b61be</td>\n",
       "      <td>[-0.009786712423983223, -0.013988726438873078,...</td>\n",
       "      <td>None</td>\n",
       "      <td>{'text': 'As at most other universities, Notre...</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5733bed24776f41900661188</td>\n",
       "      <td>[0.013343917696606181, -0.0007001232846109822,...</td>\n",
       "      <td>None</td>\n",
       "      <td>{'text': 'The university is the major seat of ...</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5733a6424776f41900660f51</td>\n",
       "      <td>[-0.0085222901071539, 0.004399558219521822, -0...</td>\n",
       "      <td>None</td>\n",
       "      <td>{'text': 'The College of Engineering was estab...</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5733a70c4776f41900660f64</td>\n",
       "      <td>[-0.006695996885869355, -0.02067068565761649, ...</td>\n",
       "      <td>None</td>\n",
       "      <td>{'text': 'All of Notre Dame's undergraduate st...</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                         id  \\\n",
       "0  5733be284776f41900661182   \n",
       "1  5733bf84d058e614000b61be   \n",
       "2  5733bed24776f41900661188   \n",
       "3  5733a6424776f41900660f51   \n",
       "4  5733a70c4776f41900660f64   \n",
       "\n",
       "                                              values sparse_values  \\\n",
       "0  [-0.010262451963272523, 0.02222637996192584, -...          None   \n",
       "1  [-0.009786712423983223, -0.013988726438873078,...          None   \n",
       "2  [0.013343917696606181, -0.0007001232846109822,...          None   \n",
       "3  [-0.0085222901071539, 0.004399558219521822, -0...          None   \n",
       "4  [-0.006695996885869355, -0.02067068565761649, ...          None   \n",
       "\n",
       "                                            metadata  blob  \n",
       "0  {'text': 'Architecturally, the school has a Ca...  None  \n",
       "1  {'text': 'As at most other universities, Notre...  None  \n",
       "2  {'text': 'The university is the major seat of ...  None  \n",
       "3  {'text': 'The College of Engineering was estab...  None  \n",
       "4  {'text': 'All of Notre Dame's undergraduate st...  None  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from pinecone_datasets import load_dataset\n",
    "\n",
    "dataset = load_dataset(\"squad-text-embedding-ada-002\")\n",
    "dataset.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "K5Q16wRH9SmO",
    "outputId": "ccad2ed3-335b-4a39-ad22-61404907f062"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "18891"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(dataset)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "c3-Plec39SmO"
   },
   "source": [
    "We'll format the dataset ready for upsert and reduce what we use to a subset of the full dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 206
    },
    "id": "4CW5mNi89SmO",
    "outputId": "03588b45-9028-43e0-83a3-cd2b93dc9d63"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>values</th>\n",
       "      <th>metadata</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5733be284776f41900661182</td>\n",
       "      <td>[-0.010262451963272523, 0.02222637996192584, -...</td>\n",
       "      <td>{'text': 'Architecturally, the school has a Ca...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5733bf84d058e614000b61be</td>\n",
       "      <td>[-0.009786712423983223, -0.013988726438873078,...</td>\n",
       "      <td>{'text': 'As at most other universities, Notre...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5733bed24776f41900661188</td>\n",
       "      <td>[0.013343917696606181, -0.0007001232846109822,...</td>\n",
       "      <td>{'text': 'The university is the major seat of ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5733a6424776f41900660f51</td>\n",
       "      <td>[-0.0085222901071539, 0.004399558219521822, -0...</td>\n",
       "      <td>{'text': 'The College of Engineering was estab...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5733a70c4776f41900660f64</td>\n",
       "      <td>[-0.006695996885869355, -0.02067068565761649, ...</td>\n",
       "      <td>{'text': 'All of Notre Dame's undergraduate st...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                         id  \\\n",
       "0  5733be284776f41900661182   \n",
       "1  5733bf84d058e614000b61be   \n",
       "2  5733bed24776f41900661188   \n",
       "3  5733a6424776f41900660f51   \n",
       "4  5733a70c4776f41900660f64   \n",
       "\n",
       "                                              values  \\\n",
       "0  [-0.010262451963272523, 0.02222637996192584, -...   \n",
       "1  [-0.009786712423983223, -0.013988726438873078,...   \n",
       "2  [0.013343917696606181, -0.0007001232846109822,...   \n",
       "3  [-0.0085222901071539, 0.004399558219521822, -0...   \n",
       "4  [-0.006695996885869355, -0.02067068565761649, ...   \n",
       "\n",
       "                                            metadata  \n",
       "0  {'text': 'Architecturally, the school has a Ca...  \n",
       "1  {'text': 'As at most other universities, Notre...  \n",
       "2  {'text': 'The university is the major seat of ...  \n",
       "3  {'text': 'The College of Engineering was estab...  \n",
       "4  {'text': 'All of Notre Dame's undergraduate st...  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# we drop sparse_values as they are not needed for this example\n",
    "dataset.documents.drop(['sparse_values', 'blob'], axis=1, inplace=True)\n",
    "\n",
    "dataset.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "topics = set()\n",
    "\n",
    "print(\"Here are some example topics in our Knowledge Base:\\n\")\n",
    "for r in dataset.documents.iloc[:].to_dict(orient=\"records\"):\n",
    "    topics.add(r['metadata']['title'])\n",
    "\n",
    "for topic in sorted(topics)[50:75]:\n",
    "    print(f\"- {topic}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Initializing the Pinecone client\n",
    "\n",
    "Now the data is ready, we can set up our index to store it.\n",
    "\n",
    "We begin by instantiating a Pinecone client. To do this we need a [free API key](https://app.pinecone.io)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "if not os.environ.get(\"PINECONE_API_KEY\"):\n",
    "    from pinecone_notebooks.colab import Authenticate\n",
    "    Authenticate()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pinecone import Pinecone\n",
    "\n",
    "# Instantiate client\n",
    "pc = Pinecone(api_key=os.environ.get(\"PINECONE_API_KEY\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating a Pinecone Index\n",
    "\n",
    "When creating the index we need to define several configuration properties. \n",
    "\n",
    "- `name` can be anything we like. The name is used as an identifier for the index when performing other operations such as `describe_index`, `delete_index`, and so on. \n",
    "- `metric` specifies the similarity metric that will be used later when you make queries to the index.\n",
    "- `dimension` should correspond to the dimension of the dense vectors produced by your embedding model. In this quick start, we are using made-up data so a small value is simplest.\n",
    "- `spec` holds a specification which tells Pinecone how you would like to deploy our index. You can find a list of all [available providers and regions here](https://docs.pinecone.io/troubleshooting/available-cloud-regions).\n",
    "\n",
    "There are more configurations available, but this minimal set will get us started."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "D5WT4PAN9SmP"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{\n",
       "    \"name\": \"langchain-retrieval-agent-fast\",\n",
       "    \"dimension\": 1536,\n",
       "    \"metric\": \"dotproduct\",\n",
       "    \"host\": \"langchain-retrieval-agent-fast-dojoi3u.svc.aped-4627-b74a.pinecone.io\",\n",
       "    \"spec\": {\n",
       "        \"serverless\": {\n",
       "            \"cloud\": \"aws\",\n",
       "            \"region\": \"us-east-1\"\n",
       "        }\n",
       "    },\n",
       "    \"status\": {\n",
       "        \"ready\": true,\n",
       "        \"state\": \"Ready\"\n",
       "    },\n",
       "    \"deletion_protection\": \"disabled\"\n",
       "}"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from pinecone import ServerlessSpec\n",
    "\n",
    "index_name = 'langchain-retrieval-agent-fast'\n",
    "\n",
    "if not pc.has_index(name=index_name):\n",
    "    # Create a new index\n",
    "    pc.create_index(\n",
    "        name=index_name,\n",
    "        dimension=1536,  # dimensionality of text-embedding-ada-002\n",
    "        metric='dotproduct',\n",
    "        spec=ServerlessSpec(\n",
    "            cloud='aws',\n",
    "            region='us-east-1'\n",
    "        )\n",
    "    )\n",
    "\n",
    "pc.describe_index(name=index_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Upserting data into your Pinecone Index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "bfsfuFmqaS4G",
    "outputId": "893ac72e-675f-4f51-cbee-9e3edb35e0bf"
   },
   "outputs": [],
   "source": [
    "# Instantiate an Index client\n",
    "index = pc.Index(name=index_name)\n",
    "\n",
    "index.describe_index_stats()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "QbDTrvvm9SmP"
   },
   "source": [
    "We should see that the new Pinecone index has a `total_vector_count` of `0`, as we haven't added any vectors yet.\n",
    "\n",
    "Now we upsert the data to Pinecone:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 99,
     "referenced_widgets": [
      "f209238acc1e4f1ab2d96ffb11be8bbf",
      "9a2f24d0fb9c46f8acc06b5cbab26b0f",
      "6b62eba67ca242c0aab2d89db8ecfe8e",
      "d3c5b4e6dd3f42b28ab2aa31467c225a",
      "85411136569942de8517f0fb87c6c219",
      "c8eb7fd7e1054081b787957f7629db93",
      "34defdaf2a0546c3adadc75f85826807",
      "0ada7be8205d4f5fb60e8f168461a98f",
      "1e766970eec4407dadc3d893bd183d10",
      "d7a718b5074941d7869e6e757341baa5",
      "48ff01c7ae0f4ddc977b3312b37f110c",
      "0b730c02e45d4c458ce5f13d17d077d6",
      "ca5e4518d60544198e6c61b4e6f474c6",
      "6f4b8a7d4d3845b9babd09a8c4e532aa",
      "3c958cf09ae44e56bf0b3b2fc874d5e6",
      "d8afd524312a491287cae59a4362cf9a",
      "f4b5f98a271d4910bc371968efd89bba",
      "6c147bb1b4ff4ce5b23581e6247d1dae",
      "0ed671c27e454f29a2932579287d9d65",
      "ae72fe004bf3446e93684f2e740710e1",
      "4003e6e3af06435a86632dc5ee372771",
      "00a123d105884d8a9d9bc54e7c041a42"
     ]
    },
    "id": "AhDcbRGTaWPi",
    "outputId": "16be1b62-0477-485f-ed9c-0020d8d0be11"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "sending upsert requests: 100%|██████████| 18891/18891 [02:39<00:00, 118.24it/s]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'upserted_count': 18891}"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "index.upsert_from_dataframe(dataset.documents, batch_size=100)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "jDUnLdy1b7G1"
   },
   "source": [
    "We've indexed everything, now we can check the number of vectors in our index. We may see `total_vector_count` is slightly less than the total vectors in our dataset but this is expected as Pinecone is eventually consistent. If you check back again a few moments later you should see the expected total."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "SiccGZKAb_Qo",
    "outputId": "80dcd3d8-c0cc-46c8-f00f-15129a262523"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'dimension': 1536,\n",
       " 'index_fullness': 0.0,\n",
       " 'namespaces': {'': {'vector_count': 18891}},\n",
       " 'total_vector_count': 18891}"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "index.describe_index_stats()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "b-3oolT5cCR8"
   },
   "source": [
    "## Working with Langchain"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "DcZ12U06cCH5"
   },
   "source": [
    "Now that we've built our index we can switch over to LangChain. LangChain defines standard interfaces that are helpful for using Pinecone with other components in your AI stack.\n",
    "\n",
    "We start by initializing `PineconeVectorStore` which implements LangChain's standard interface for vector stores. We configure it to interact with the `'langchain-retrieval-agent-fast'` index we just built. \n",
    "\n",
    "We'll also need to setup an Embedding Model component to embed our queries using `text-embedding-ada-002`, the same OpenAI model that was used to create embeddings in the pre-embedded dataset we upserted into our Pinecone index.\n",
    "\n",
    "We do that like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "id": "-og9Vt_-9SmQ"
   },
   "outputs": [],
   "source": [
    "from langchain_openai import OpenAIEmbeddings\n",
    "\n",
    "openai_api_key = os.environ.get('OPENAI_API_KEY') or 'OPENAI_API_KEY'\n",
    "\n",
    "embed = OpenAIEmbeddings(\n",
    "    model='text-embedding-ada-002',\n",
    "    openai_api_key=openai_api_key\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "id": "0MBJ477-cFNw"
   },
   "outputs": [],
   "source": [
    "from langchain_pinecone import PineconeVectorStore\n",
    "\n",
    "pinecone_vectorstore = PineconeVectorStore(\n",
    "    index_name=index_name, \n",
    "    embedding=embed, \n",
    "    text_key=\"text\"\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "3K3xRthWcXzW"
   },
   "source": [
    "As in previous examples, we can use the `similarity_search` method to do a pure semantic search (without the generation component)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "uITMZtzschJF",
    "outputId": "9b29a986-4163-48a5-a4cc-f4dfaff9d1c3"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'id': '57338724d058e614000b5c9f',\n",
      " 'metadata': {'title': 'University_of_Notre_Dame'},\n",
      " 'page_content': 'In 1919 Father James Burns became president of Notre Dame, '\n",
      "                 'and in three years he produced an academic revolution that '\n",
      "                 'brought the school up to national standards by adopting the '\n",
      "                 \"elective system and moving away from the university's \"\n",
      "                 'traditional scholastic and classical emphasis. By contrast, '\n",
      "                 'the Jesuit colleges, bastions of academic conservatism, were '\n",
      "                 'reluctant to move to a system of electives. Their graduates '\n",
      "                 'were shut out of Harvard Law School for that reason. Notre '\n",
      "                 'Dame continued to grow over the years, adding more colleges, '\n",
      "                 'programs, and sports teams. By 1921, with the addition of '\n",
      "                 'the College of Commerce, Notre Dame had grown from a small '\n",
      "                 'college to a university with five colleges and a '\n",
      "                 'professional law school. The university continued to expand '\n",
      "                 'and add new residence halls and buildings with each '\n",
      "                 'subsequent president.',\n",
      " 'type': 'Document'}\n",
      "\n",
      "{'id': '5733a6424776f41900660f51',\n",
      " 'metadata': {'title': 'University_of_Notre_Dame'},\n",
      " 'page_content': 'The College of Engineering was established in 1920, however, '\n",
      "                 'early courses in civil and mechanical engineering were a '\n",
      "                 'part of the College of Science since the 1870s. Today the '\n",
      "                 'college, housed in the Fitzpatrick, Cushing, and '\n",
      "                 'Stinson-Remick Halls of Engineering, includes five '\n",
      "                 'departments of study – aerospace and mechanical engineering, '\n",
      "                 'chemical and biomolecular engineering, civil engineering and '\n",
      "                 'geological sciences, computer science and engineering, and '\n",
      "                 'electrical engineering – with eight B.S. degrees offered. '\n",
      "                 'Additionally, the college offers five-year dual degree '\n",
      "                 'programs with the Colleges of Arts and Letters and of '\n",
      "                 'Business awarding additional B.A. and Master of Business '\n",
      "                 'Administration (MBA) degrees, respectively.',\n",
      " 'type': 'Document'}\n",
      "\n",
      "{'id': '5733974d4776f41900660e17',\n",
      " 'metadata': {'title': 'University_of_Notre_Dame'},\n",
      " 'page_content': 'Since 2005, Notre Dame has been led by John I. Jenkins, '\n",
      "                 'C.S.C., the 17th president of the university. Jenkins took '\n",
      "                 'over the position from Malloy on July 1, 2005. In his '\n",
      "                 'inaugural address, Jenkins described his goals of making the '\n",
      "                 'university a leader in research that recognizes ethics and '\n",
      "                 'building the connection between faith and studies. During '\n",
      "                 'his tenure, Notre Dame has increased its endowment, enlarged '\n",
      "                 'its student body, and undergone many construction projects '\n",
      "                 'on campus, including Compton Family Ice Arena, a new '\n",
      "                 'architecture hall, additional residence halls, and the '\n",
      "                 'Campus Crossroads, a $400m enhancement and expansion of '\n",
      "                 'Notre Dame Stadium.',\n",
      " 'type': 'Document'}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from pprint import pprint\n",
    "\n",
    "query = \"When was the college of engineering in the University of Notre Dame established?\"\n",
    "\n",
    "documents = pinecone_vectorstore.similarity_search(\n",
    "    query=query,\n",
    "    k=3  # return 3 most relevant docs\n",
    ")\n",
    "\n",
    "for doc in documents:\n",
    "    pprint(doc.__dict__)\n",
    "    print()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "-zGF6YsgczqT"
   },
   "source": [
    "Looks like we're getting good results. Let's take a look at how we can begin integrating this into a conversational agent."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "tFsIOm73dcOI"
   },
   "source": [
    "## Initializing the Conversational Agent"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "XMv6TXWkdfNR"
   },
   "source": [
    "Our conversational agent needs a Chat LLM component. We create that using:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "id": "zMRs9Klic5-Y"
   },
   "outputs": [],
   "source": [
    "from langchain_openai import ChatOpenAI\n",
    "from langchain.chains.conversation.memory import ConversationBufferWindowMemory\n",
    "from langchain.chains import RetrievalQA\n",
    "\n",
    "# Chat completion LLM\n",
    "llm = ChatOpenAI(\n",
    "    openai_api_key=openai_api_key,\n",
    "    model_name='gpt-3.5-turbo',\n",
    "    temperature=0.0\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we need to build a chain that can incorporate context from our `PineconeVectorStore` instance into prompts passed to the LLM. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.prompts import PromptTemplate\n",
    "from langchain_core.output_parsers import StrOutputParser\n",
    "from langchain_core.runnables import RunnablePassthrough\n",
    "\n",
    "# Based on the RAG template from https://smith.langchain.com/hub/rlm/rag-prompt\n",
    "template=(\n",
    "    \"You are an assistant for question-answering tasks. Use the following pieces of retrieved context to answer the question. If you don't know the answer, just say that you don't know. Use three sentences maximum and keep the answer concise.\"\n",
    "    \"Question: {question}\"\n",
    "    \"Context: {context}\"\n",
    "    \"Answer:\"\n",
    ")\n",
    "prompt = PromptTemplate(input_variables=[\"question\", \"context\"], template=template)\n",
    "\n",
    "def format_docs(docs):\n",
    "    return \"\\n\\n\".join(doc.page_content for doc in docs)\n",
    "\n",
    "# Retrieval Question-Answer chain\n",
    "qa_chain = (\n",
    "    {\n",
    "        \"context\": pinecone_vectorstore.as_retriever() | format_docs,\n",
    "        \"question\": RunnablePassthrough(),\n",
    "    }\n",
    "    | prompt\n",
    "    | llm\n",
    "    | StrOutputParser()\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "-ySfWyZLdboX"
   },
   "source": [
    "Using these we can generate an answer by invoking the chain:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 36
    },
    "id": "LaYSq0V-dxHw",
    "outputId": "d2884a86-71cc-4cbe-fa55-b92b8025a8cc"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'The College of Engineering in the University of Notre Dame was established in 1920. Today, the college includes five departments of study and offers eight B.S. degrees.'"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "qa_chain.invoke(\"When was the college of engineering in the University of Notre Dame established?\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "DtSXR5RXdyU0"
   },
   "source": [
    "# Integrating a retrieval chain into a Tool\n",
    "\n",
    "But this isn't yet ready for our conversational agent. For that we need to convert this retrieval chain into a Langchain Tool. Tools are Langchain Runnables that can be invoked by agents. We do that like this, giving the chain a name `knowledge-base` that we will see later on when the LLM invokes the tool:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "FwCYrS4duqBW"
   },
   "outputs": [],
   "source": [
    "knowledge_base_tool = qa_chain.as_tool(\n",
    "        name='knowledge-base',\n",
    "        description=(\n",
    "            'use this tool when answering general knowledge queries to get '\n",
    "            'more information about the topic'\n",
    "        )\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "wXi_0ipTvM_l"
   },
   "source": [
    "Now we are ready to incorporate these pieces into an LangGraph agent."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building a knowledgeable chatbot agent with LangGraph"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**LangGraph** is a framework from **LangChain** for building AI applications using a state machine to model complex workflows. \n",
    "\n",
    "To begin, we first define the `State` for our agent, which keeps track of a list of messages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Annotated\n",
    "from typing_extensions import TypedDict\n",
    "\n",
    "from langgraph.graph import StateGraph\n",
    "from langgraph.graph.message import add_messages\n",
    "\n",
    "class State(TypedDict):\n",
    "    messages: Annotated[list, add_messages]\n",
    "\n",
    "graph_builder = StateGraph(State)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we need to add some nodes into our graph. First, a chatbot node and a tools node along with edges that describe how it can transition from one node to the next in the state machine represented by the graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<langgraph.graph.state.StateGraph at 0xffff2e2bea20>"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langgraph.prebuilt import ToolNode, tools_condition\n",
    "\n",
    "tools = [knowledge_base_tool]\n",
    "llm_with_tools = llm.bind_tools(tools)\n",
    "\n",
    "def chatbot(state: State):\n",
    "    return {\"messages\": [llm_with_tools.invoke(state[\"messages\"])]}\n",
    "\n",
    "graph_builder.add_node(\"chatbot\", chatbot)\n",
    "\n",
    "tool_node = ToolNode(tools=tools)\n",
    "graph_builder.add_node(\"tools\", tool_node)\n",
    "\n",
    "graph_builder.add_conditional_edges(\n",
    "    \"chatbot\",\n",
    "    tools_condition,\n",
    ")\n",
    "graph_builder.add_edge(\"tools\", \"chatbot\")\n",
    "graph_builder.set_entry_point(\"chatbot\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We also want to add in a `checkpointer` and compile the graph. Checkpointing is what allows our agent to have context on earlier messages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langgraph.checkpoint.memory import MemorySaver\n",
    "\n",
    "memory = MemorySaver()\n",
    "graph = graph_builder.compile(checkpointer=memory)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "WbXl-AzVvszB"
   },
   "source": [
    "Finally, let's wrap our graph into a `agent` function to simplify interacting with it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "def agent(user_message):\n",
    "    config = {\"configurable\": {\"thread_id\": \"1\"}}\n",
    "    \n",
    "    # The config is the **second positional argument** to stream() or invoke()!\n",
    "    events = graph.stream(\n",
    "        {\"messages\": [{\"role\": \"user\", \"content\": user_message}]},\n",
    "        config,\n",
    "        stream_mode=\"values\",\n",
    "    )\n",
    "    for event in events:\n",
    "        event[\"messages\"][-1].pretty_print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using the chat agent\n",
    "\n",
    "Now we are ready to chat with our agent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "================================\u001b[1m Human Message \u001b[0m=================================\n",
      "\n",
      "Hi there! My name is Jen.\n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "\n",
      "Hello Jen! How can I assist you today?\n"
     ]
    }
   ],
   "source": [
    "agent(\"Hi there! My name is Jen.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Testing the conversational memory\n",
    "\n",
    "Let's see if it remembers what we just told it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "================================\u001b[1m Human Message \u001b[0m=================================\n",
      "\n",
      "Do you remember my name?\n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "\n",
      "Yes, your name is Jen. How can I help you today, Jen?\n"
     ]
    }
   ],
   "source": [
    "agent(\"Do you remember my name?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Leveraging context from our Knowledge Base\n",
    "\n",
    "Next let's try asking it a question that requires the LLM to invoke our knowledge base for context. Recall that our Knowledge Base has been filled with facts from Wikipedia in the [SQuAD dataset](https://huggingface.co/datasets/rajpurkar/squad), including several entries about the **University of Notre Dame**.\n",
    "\n",
    "In the output you can see the `Tool Calls` where the `knowledge-base` is invoked. `knowledge-base` is the identifier of the question answer chain `qa_chain` we defined above that uses a combination of OpenAI to embed queries and Pinecone to find context relevant to the question."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "================================\u001b[1m Human Message \u001b[0m=================================\n",
      "\n",
      "Do you know anything about the University of Notre Dame?\n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "Tool Calls:\n",
      "  knowledge-base (call_32ppeZqB9OoORtqlL85wlFSw)\n",
      " Call ID: call_32ppeZqB9OoORtqlL85wlFSw\n",
      "  Args:\n",
      "    __arg1: University of Notre Dame\n",
      "=================================\u001b[1m Tool Message \u001b[0m=================================\n",
      "Name: knowledge-base\n",
      "\n",
      "The University of Notre Dame is a Catholic research university located in South Bend, Indiana. It is known for its recognizable landmarks such as the Golden Dome and the Basilica. Notre Dame offers undergraduate programs in four colleges and has a strong alumni network.\n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "\n",
      "The University of Notre Dame is a Catholic research university located in South Bend, Indiana. It is known for its recognizable landmarks such as the Golden Dome and the Basilica. Notre Dame offers undergraduate programs in four colleges and has a strong alumni network. If you would like more information, feel free to ask!\n"
     ]
    }
   ],
   "source": [
    "agent(\"Do you know anything about the University of Notre Dame?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Follow-up question \n",
    "\n",
    "We can ask a follow-up question, and the agent remembers that \"it\" in this case is the University of Notre Dame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "================================\u001b[1m Human Message \u001b[0m=================================\n",
      "\n",
      "When was it founded?\n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "Tool Calls:\n",
      "  knowledge-base (call_UZa9Ptw4U2GTL8O7KjweZnit)\n",
      " Call ID: call_UZa9Ptw4U2GTL8O7KjweZnit\n",
      "  Args:\n",
      "    __arg1: University of Notre Dame founding date\n",
      "=================================\u001b[1m Tool Message \u001b[0m=================================\n",
      "Name: knowledge-base\n",
      "\n",
      "The University of Notre Dame was founded in 1842.\n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "\n",
      "The University of Notre Dame was founded in 1842. If you have any more questions or need further information, feel free to ask!\n"
     ]
    }
   ],
   "source": [
    "agent(\"When was it founded?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### General knowledge question\n",
    "\n",
    "Let's try asking it about something that is not in the knowledge-base."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "================================\u001b[1m Human Message \u001b[0m=================================\n",
      "\n",
      "What is 14 * 9?\n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "\n",
      "The result of 14 multiplied by 9 is 126.\n"
     ]
    }
   ],
   "source": [
    "agent(\"What is 14 * 9?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great, no `Tool Calls` are shown which means the agent correctly recognized it did not need to invoke the knowledge-base to answer this question."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Interactive chat\n",
    "\n",
    "We've left the following cell commented by default so as not to break our automated testing of this notebook.\n",
    "\n",
    "But if you want to try a continuous series of interactions with the chat bot, uncomment the code below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# print(\"Type 'quit' to exit\")\n",
    "# while True:\n",
    "#     user_input = input(\"User: \")\n",
    "#     if user_input.lower() in [\"quit\", \"exit\", \"q\"]:\n",
    "#         print(\"Goodbye!\")\n",
    "#         break\n",
    "\n",
    "#     agent(user_input)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "PWivmw9F3bCw"
   },
   "source": [
    "## Wrapup\n",
    "\n",
    "That's all for this example of building a retrieval augmented conversational agent with OpenAI and Pinecone (the OP stack) using LangChain.\n",
    "\n",
    "To recap, we:\n",
    "- Built a Pinecone index using embeddings derived from facts in the SQuAD dataset\n",
    "- Configured a `PineconeVectorStore` instance to interact with Pineco\n",
    "- Built a RAG chain using our Pinecone-backed knowledge base\n",
    "- Integrated that chain into an LLM-powered chat agent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Demo cleanup\n",
    "\n",
    "Once finished, we delete the Pinecone index to save resources:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Pa1whr8V3Wfm"
   },
   "outputs": [],
   "source": [
    "pc.delete_index(name=index_name)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "Ykg5TYA033yR"
   },
   "source": [
    "---"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.9"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "00a123d105884d8a9d9bc54e7c041a42": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0ada7be8205d4f5fb60e8f168461a98f": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0b730c02e45d4c458ce5f13d17d077d6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_ca5e4518d60544198e6c61b4e6f474c6",
       "IPY_MODEL_6f4b8a7d4d3845b9babd09a8c4e532aa",
       "IPY_MODEL_3c958cf09ae44e56bf0b3b2fc874d5e6"
      ],
      "layout": "IPY_MODEL_d8afd524312a491287cae59a4362cf9a"
     }
    },
    "0ed671c27e454f29a2932579287d9d65": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1e766970eec4407dadc3d893bd183d10": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "34defdaf2a0546c3adadc75f85826807": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "3c958cf09ae44e56bf0b3b2fc874d5e6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_4003e6e3af06435a86632dc5ee372771",
      "placeholder": "​",
      "style": "IPY_MODEL_00a123d105884d8a9d9bc54e7c041a42",
      "value": " 189/189 [00:02&lt;00:00, 926.99it/s]"
     }
    },
    "4003e6e3af06435a86632dc5ee372771": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "48ff01c7ae0f4ddc977b3312b37f110c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "6b62eba67ca242c0aab2d89db8ecfe8e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0ada7be8205d4f5fb60e8f168461a98f",
      "max": 18891,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_1e766970eec4407dadc3d893bd183d10",
      "value": 18891
     }
    },
    "6c147bb1b4ff4ce5b23581e6247d1dae": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "6f4b8a7d4d3845b9babd09a8c4e532aa": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0ed671c27e454f29a2932579287d9d65",
      "max": 189,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_ae72fe004bf3446e93684f2e740710e1",
      "value": 189
     }
    },
    "85411136569942de8517f0fb87c6c219": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9a2f24d0fb9c46f8acc06b5cbab26b0f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c8eb7fd7e1054081b787957f7629db93",
      "placeholder": "​",
      "style": "IPY_MODEL_34defdaf2a0546c3adadc75f85826807",
      "value": "sending upsert requests: 100%"
     }
    },
    "ae72fe004bf3446e93684f2e740710e1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "c8eb7fd7e1054081b787957f7629db93": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ca5e4518d60544198e6c61b4e6f474c6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f4b5f98a271d4910bc371968efd89bba",
      "placeholder": "​",
      "style": "IPY_MODEL_6c147bb1b4ff4ce5b23581e6247d1dae",
      "value": "collecting async responses: 100%"
     }
    },
    "d3c5b4e6dd3f42b28ab2aa31467c225a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d7a718b5074941d7869e6e757341baa5",
      "placeholder": "​",
      "style": "IPY_MODEL_48ff01c7ae0f4ddc977b3312b37f110c",
      "value": " 18891/18891 [00:18&lt;00:00, 3552.98it/s]"
     }
    },
    "d7a718b5074941d7869e6e757341baa5": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d8afd524312a491287cae59a4362cf9a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f209238acc1e4f1ab2d96ffb11be8bbf": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_9a2f24d0fb9c46f8acc06b5cbab26b0f",
       "IPY_MODEL_6b62eba67ca242c0aab2d89db8ecfe8e",
       "IPY_MODEL_d3c5b4e6dd3f42b28ab2aa31467c225a"
      ],
      "layout": "IPY_MODEL_85411136569942de8517f0fb87c6c219"
     }
    },
    "f4b5f98a271d4910bc371968efd89bba": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}