diff --git a/lab-agent-vector-store.ipynb b/lab-agent-vector-store.ipynb deleted file mode 100644 index c0f762e..0000000 --- a/lab-agent-vector-store.ipynb +++ /dev/null @@ -1,624 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "id": "efad6479-7fb1-40c4-8610-8963f8de307f", - "metadata": {}, - "source": [ - "# Lab | Agent & Vector store\n", - "\n", - "**Change the state union dataset and replicate this lab by updating the prompts accordingly.**\n", - "\n", - "One such dataset is the [sonnets.txt](https://github.com/martin-gorner/tensorflow-rnn-shakespeare/blob/master/shakespeare/sonnets.txt) dataset or any other data of your choice from the same git." - ] - }, - { - "cell_type": "markdown", - "id": "68b24990", - "metadata": {}, - "source": [ - "# Combine agents and vector stores\n", - "\n", - "This notebook covers how to combine agents and vector stores. The use case for this is that you've ingested your data into a vector store and want to interact with it in an agentic manner.\n", - "\n", - "The recommended method for doing so is to create a `RetrievalQA` and then use that as a tool in the overall agent. Let's take a look at doing this below. You can do this with multiple different vector DBs, and use the agent as a way to route between them. There are two different ways of doing this - you can either let the agent use the vector stores as normal tools, or you can set `return_direct=True` to really just use the agent as a router." - ] - }, - { - "cell_type": "markdown", - "id": "9b22020a", - "metadata": {}, - "source": [ - "## Create the vector store" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "12df91e0-7322-43c2-96cd-0159d017a1e1", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "#!pip install chromadb langchain langchain_community langchain_openai" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "2868cff1-a3ef-426f-b8d7-7fe89047a5b7", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "from langchain.chains import RetrievalQA\n", - "from langchain_community.vectorstores import Chroma\n", - "from langchain_openai import OpenAI, OpenAIEmbeddings\n", - "from langchain_text_splitters import CharacterTextSplitter\n", - "from langchain_community.document_loaders import TextLoader" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "aeff3a2d-b0cc-429f-a34c-4381d71f1a5f", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "import os\n", - "from dotenv import load_dotenv, find_dotenv\n", - "_ = load_dotenv(find_dotenv())\n", - "\n", - "OPENAI_API_KEY = os.getenv('OPENAI_API_KEY')" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "f16d010b", - "metadata": {}, - "outputs": [], - "source": [ - "# If you're using colab, run this\n", - "#os.environ['OPENAI_API_KEY'] = \"YOUR_OPENAI_API_KEY\"" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "id": "14331eec-fd46-42e0-b6e7-adaf21824ef7", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "llm = OpenAI(temperature=0)" - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "id": "0b7b772b", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "from pathlib import Path\n", - "\n", - "relevant_parts = []\n", - "for p in Path(\".\").absolute().parts:\n", - " relevant_parts.append(p)\n", - " if relevant_parts[-3:] == [\"langchain\", \"docs\", \"modules\"]:\n", - " break\n", - "doc_path = str(Path(*relevant_parts) / \"state_of_the_union.txt\")" - ] - }, - { - "cell_type": "code", - "execution_count": 21, - "id": "f2675861", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "loader = TextLoader(doc_path)\n", - "documents = loader.load()\n", - "text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)\n", - "texts = text_splitter.split_documents(documents)\n", - "\n", - "embeddings = OpenAIEmbeddings()\n", - "docsearch = Chroma.from_documents(texts, embeddings, collection_name=\"state-of-union\")" - ] - }, - { - "cell_type": "code", - "execution_count": 23, - "id": "bc5403d4", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "state_of_union = RetrievalQA.from_chain_type(\n", - " llm=llm, chain_type=\"stuff\", retriever=docsearch.as_retriever()\n", - ")" - ] - }, - { - "cell_type": "code", - "execution_count": 25, - "id": "1431cded", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "from langchain_community.document_loaders import WebBaseLoader" - ] - }, - { - "cell_type": "code", - "execution_count": 27, - "id": "915d3ff3", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "loader = WebBaseLoader(\"https://beta.ruff.rs/docs/faq/\")" - ] - }, - { - "cell_type": "code", - "execution_count": 29, - "id": "96a2edf8", - "metadata": { - "tags": [] - }, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "Created a chunk of size 2134, which is longer than the specified 1000\n", - "Created a chunk of size 2470, which is longer than the specified 1000\n", - "Created a chunk of size 1259, which is longer than the specified 1000\n", - "Created a chunk of size 1636, which is longer than the specified 1000\n" - ] - } - ], - "source": [ - "docs = loader.load()\n", - "ruff_texts = text_splitter.split_documents(docs)\n", - "ruff_db = Chroma.from_documents(ruff_texts, embeddings, collection_name=\"ruff\")\n", - "ruff = RetrievalQA.from_chain_type(\n", - " llm=llm, chain_type=\"stuff\", retriever=ruff_db.as_retriever()\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "c0a6c031", - "metadata": {}, - "source": [ - "## Create the Agent" - ] - }, - { - "cell_type": "code", - "execution_count": 32, - "id": "eb142786", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "# Import things that are needed generically\n", - "from langchain.agents import AgentType, Tool, initialize_agent\n", - "from langchain_openai import OpenAI" - ] - }, - { - "cell_type": "code", - "execution_count": 34, - "id": "850bc4e9", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "tools = [\n", - " Tool(\n", - " name=\"State of Union QA System\",\n", - " func=state_of_union.run,\n", - " description=\"useful for when you need to answer questions about the most recent state of the union address. Input should be a fully formed question.\",\n", - " ),\n", - " Tool(\n", - " name=\"Ruff QA System\",\n", - " func=ruff.run,\n", - " description=\"useful for when you need to answer questions about ruff (a python linter). Input should be a fully formed question.\",\n", - " ),\n", - "]" - ] - }, - { - "cell_type": "code", - "execution_count": 36, - "id": "fc47f230", - "metadata": { - "tags": [] - }, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/opt/anaconda3/lib/python3.11/site-packages/langchain_core/_api/deprecation.py:119: LangChainDeprecationWarning: The function `initialize_agent` was deprecated in LangChain 0.1.0 and will be removed in 0.2.0. Use Use new agent constructor methods like create_react_agent, create_json_agent, create_structured_chat_agent, etc. instead.\n", - " warn_deprecated(\n" - ] - } - ], - "source": [ - "# Construct the agent. We will use the default agent type here.\n", - "# See documentation for a full list of options.\n", - "agent = initialize_agent(\n", - " tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True\n", - ")" - ] - }, - { - "cell_type": "code", - "execution_count": 38, - "id": "10ca2db8", - "metadata": { - "tags": [] - }, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/opt/anaconda3/lib/python3.11/site-packages/langchain_core/_api/deprecation.py:119: LangChainDeprecationWarning: The method `Chain.run` was deprecated in langchain 0.1.0 and will be removed in 0.2.0. Use invoke instead.\n", - " warn_deprecated(\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\n", - "\n", - "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", - "\u001b[32;1m\u001b[1;3m I should use the State of Union QA System to answer this question\n", - "Action: State of Union QA System\n", - "Action Input: What did biden say about ketanji brown jackson in the state of the union address?\u001b[0m\n", - "Observation: \u001b[36;1m\u001b[1;3m Biden mentioned that he nominated Circuit Court of Appeals Judge Ketanji Brown Jackson to serve on the United States Supreme Court, praising her as one of the nation's top legal minds who will continue Justice Breyer's legacy of excellence.\u001b[0m\n", - "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n", - "Final Answer: Biden mentioned that he nominated Circuit Court of Appeals Judge Ketanji Brown Jackson to serve on the United States Supreme Court, praising her as one of the nation's top legal minds who will continue Justice Breyer's legacy of excellence.\u001b[0m\n", - "\n", - "\u001b[1m> Finished chain.\u001b[0m\n" - ] - }, - { - "data": { - "text/plain": [ - "\"Biden mentioned that he nominated Circuit Court of Appeals Judge Ketanji Brown Jackson to serve on the United States Supreme Court, praising her as one of the nation's top legal minds who will continue Justice Breyer's legacy of excellence.\"" - ] - }, - "execution_count": 38, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "agent.invoke(\n", - " \"What did biden say about ketanji brown jackson in the state of the union address?\"\n", - ")" - ] - }, - { - "cell_type": "code", - "execution_count": 40, - "id": "4e91b811", - "metadata": { - "tags": [] - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\n", - "\n", - "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", - "\u001b[32;1m\u001b[1;3m You should always think about the differences between ruff and flake8\n", - "Action: Ruff QA System\n", - "Action Input: \"Why use ruff over flake8?\"\u001b[0m\n", - "Observation: \u001b[33;1m\u001b[1;3m Ruff can replace multiple Flake8 plugins, has a larger rule set, and can automatically fix its own lint violations. It also supports Python versions from 3.7 onwards and does not require the installation of Rust. Additionally, Ruff can be used independently as a linter or formatter, and can be used alongside a type checker for more comprehensive error detection.\u001b[0m\n", - "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n", - "Final Answer: Ruff offers more features and flexibility compared to flake8, making it a better choice for linting and formatting Python code.\u001b[0m\n", - "\n", - "\u001b[1m> Finished chain.\u001b[0m\n" - ] - }, - { - "data": { - "text/plain": [ - "{'input': 'Why use ruff over flake8?',\n", - " 'output': 'Ruff offers more features and flexibility compared to flake8, making it a better choice for linting and formatting Python code.'}" - ] - }, - "execution_count": 40, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "agent.invoke(\"Why use ruff over flake8?\")" - ] - }, - { - "cell_type": "markdown", - "id": "787a9b5e", - "metadata": {}, - "source": [ - "## Use the Agent solely as a router" - ] - }, - { - "cell_type": "markdown", - "id": "9161ba91", - "metadata": {}, - "source": [ - "You can also set `return_direct=True` if you intend to use the agent as a router and just want to directly return the result of the RetrievalQAChain.\n", - "\n", - "Notice that in the above examples the agent did some extra work after querying the RetrievalQAChain. You can avoid that and just return the result directly." - ] - }, - { - "cell_type": "code", - "execution_count": 44, - "id": "f59b377e", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "tools = [\n", - " Tool(\n", - " name=\"State of Union QA System\",\n", - " func=state_of_union.run,\n", - " description=\"useful for when you need to answer questions about the most recent state of the union address. Input should be a fully formed question.\",\n", - " return_direct=True,\n", - " ),\n", - " Tool(\n", - " name=\"Ruff QA System\",\n", - " func=ruff.run,\n", - " description=\"useful for when you need to answer questions about ruff (a python linter). Input should be a fully formed question.\",\n", - " return_direct=True,\n", - " ),\n", - "]" - ] - }, - { - "cell_type": "code", - "execution_count": 46, - "id": "8615707a", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "agent = initialize_agent(\n", - " tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True\n", - ")" - ] - }, - { - "cell_type": "code", - "execution_count": 48, - "id": "36e718a9", - "metadata": { - "tags": [] - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\n", - "\n", - "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", - "\u001b[32;1m\u001b[1;3m I should use the State of Union QA System to answer this question\n", - "Action: State of Union QA System\n", - "Action Input: What did biden say about ketanji brown jackson in the state of the union address?\u001b[0m\n", - "Observation: \u001b[36;1m\u001b[1;3m Biden mentioned that he nominated Circuit Court of Appeals Judge Ketanji Brown Jackson to serve on the United States Supreme Court, praising her as one of the nation's top legal minds who will continue Justice Breyer's legacy of excellence.\u001b[0m\n", - "\u001b[32;1m\u001b[1;3m\u001b[0m\n", - "\n", - "\u001b[1m> Finished chain.\u001b[0m\n" - ] - }, - { - "data": { - "text/plain": [ - "{'input': 'What did biden say about ketanji brown jackson in the state of the union address?',\n", - " 'output': \" Biden mentioned that he nominated Circuit Court of Appeals Judge Ketanji Brown Jackson to serve on the United States Supreme Court, praising her as one of the nation's top legal minds who will continue Justice Breyer's legacy of excellence.\"}" - ] - }, - "execution_count": 48, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "agent.invoke(\n", - " \"What did biden say about ketanji brown jackson in the state of the union address?\"\n", - ")" - ] - }, - { - "cell_type": "code", - "execution_count": 50, - "id": "edfd0a1a", - "metadata": { - "tags": [] - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\n", - "\n", - "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", - "\u001b[32;1m\u001b[1;3m You should always think about the differences between ruff and flake8\n", - "Action: Ruff QA System\n", - "Action Input: \"Why use ruff over flake8?\"\u001b[0m\n", - "Observation: \u001b[33;1m\u001b[1;3m Ruff can replace multiple Flake8 plugins, has a larger rule set, and can automatically fix its own lint violations. It also supports Python versions from 3.7 onwards and does not require the installation of Rust. Additionally, Ruff can be used independently as a linter or formatter, and can be used alongside a type checker for more comprehensive error detection.\u001b[0m\n", - "\u001b[32;1m\u001b[1;3m\u001b[0m\n", - "\n", - "\u001b[1m> Finished chain.\u001b[0m\n" - ] - }, - { - "data": { - "text/plain": [ - "{'input': 'Why use ruff over flake8?',\n", - " 'output': ' Ruff can replace multiple Flake8 plugins, has a larger rule set, and can automatically fix its own lint violations. It also supports Python versions from 3.7 onwards and does not require the installation of Rust. Additionally, Ruff can be used independently as a linter or formatter, and can be used alongside a type checker for more comprehensive error detection.'}" - ] - }, - "execution_count": 50, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "agent.invoke(\"Why use ruff over flake8?\")" - ] - }, - { - "cell_type": "markdown", - "id": "49a0cbbe", - "metadata": {}, - "source": [ - "## Multi-Hop vector store reasoning\n", - "\n", - "Because vector stores are easily usable as tools in agents, it is easy to use answer multi-hop questions that depend on vector stores using the existing agent framework." - ] - }, - { - "cell_type": "code", - "execution_count": 53, - "id": "d397a233", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "tools = [\n", - " Tool(\n", - " name=\"State of Union QA System\",\n", - " func=state_of_union.run,\n", - " description=\"useful for when you need to answer questions about the most recent state of the union address. Input should be a fully formed question, not referencing any obscure pronouns from the conversation before.\",\n", - " ),\n", - " Tool(\n", - " name=\"Ruff QA System\",\n", - " func=ruff.run,\n", - " description=\"useful for when you need to answer questions about ruff (a python linter). Input should be a fully formed question, not referencing any obscure pronouns from the conversation before.\",\n", - " ),\n", - "]" - ] - }, - { - "cell_type": "code", - "execution_count": 55, - "id": "06157240", - "metadata": { - "tags": [] - }, - "outputs": [], - "source": [ - "# Construct the agent. We will use the default agent type here.\n", - "# See documentation for a full list of options.\n", - "agent = initialize_agent(\n", - " tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True\n", - ")" - ] - }, - { - "cell_type": "code", - "execution_count": 57, - "id": "b492b520", - "metadata": { - "tags": [] - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\n", - "\n", - "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", - "\u001b[32;1m\u001b[1;3m I should use the Ruff QA System to answer the first question and the State of Union QA System to answer the second question.\n", - "Action: Ruff QA System\n", - "Action Input: What tool does ruff use to run over Jupyter Notebooks?\u001b[0m\n", - "Observation: \u001b[33;1m\u001b[1;3m Ruff integrates with nbQA, a tool for running linters and code formatters over Jupyter Notebooks.\u001b[0m\n", - "Thought:\u001b[32;1m\u001b[1;3m I should use the State of Union QA System to answer the second question.\n", - "Action: State of Union QA System\n", - "Action Input: Did the president mention that tool in the state of the union?\u001b[0m\n", - "Observation: \u001b[36;1m\u001b[1;3m\n", - "No, the president did not mention a specific tool in the state of the union.\u001b[0m\n", - "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer.\n", - "Final Answer: No, the president did not mention a specific tool in the state of the union.\u001b[0m\n", - "\n", - "\u001b[1m> Finished chain.\u001b[0m\n" - ] - }, - { - "data": { - "text/plain": [ - "{'input': 'What tool does ruff use to run over Jupyter Notebooks? Did the president mention that tool in the state of the union?',\n", - " 'output': 'No, the president did not mention a specific tool in the state of the union.'}" - ] - }, - "execution_count": 57, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "agent.invoke(\n", - " \"What tool does ruff use to run over Jupyter Notebooks? Did the president mention that tool in the state of the union?\"\n", - ")" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.8" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -}