diff --git a/solution.ipynb b/solution.ipynb new file mode 100644 index 0000000..e4fef19 --- /dev/null +++ b/solution.ipynb @@ -0,0 +1,1051 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "efad6479-7fb1-40c4-8610-8963f8de307f", + "metadata": {}, + "source": [ + "# Lab | Agent & Vector store\n" + ] + }, + { + "cell_type": "markdown", + "id": "25fb0c6d", + "metadata": {}, + "source": [ + "
\n", + "\n", + "## Intro\n", + "\n", + "In this lab you'll build an AI agent that knows when to consult *different* knowledge bases to answer a question — instead of relying on a single source of truth.\n", + "\n", + "Here's what to expect:\n", + "\n", + "1. **Follow a full worked demo** — We'll walk through every step together: ingesting the *state of the union* speech and the *Ruff* docs into two vector stores, wrapping each in a `RetrievalQA` tool, and building an agent that picks the right tool (or both!) depending on the question.\n", + "\n", + "2. **Replicate it yourself with a new dataset** — Then, you'll swap in a dataset of your choice and rebuild the same pipeline, adapting the prompts and tools along the way.\n", + "\n", + "By the end of this lab, you'll understand how to build multi-source AI agents and be able to apply the pattern to your own datasets." + ] + }, + { + "cell_type": "markdown", + "id": "68b24990", + "metadata": {}, + "source": [ + "
\n", + "\n", + "## Combine agents and vector stores\n", + "\n", + "Let's get into the demo. We'll wrap each vector store in a `RetrievalQA` chain and hand it to an agent as a `Tool`. The agent then decides, at each step, which tool to call based purely on its description — this is what lets it route between multiple knowledge sources.\n", + "\n", + "There are two flavors of this pattern, both of which we'll try below:\n", + "\n", + "- **Agent as reasoner** — the agent calls a tool and can keep reasoning afterward (e.g. to combine results from multiple sources).\n", + "- **Agent as router** (`return_direct=True`) — the agent just picks the right tool and returns its answer immediately, no extra reasoning." + ] + }, + { + "cell_type": "markdown", + "id": "9b22020a", + "metadata": {}, + "source": [ + "
\n", + "\n", + "## Install dependencies\n", + "\n", + "Uncomment and run the cells below to install the required dependencies for this notebook." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "12df91e0-7322-43c2-96cd-0159d017a1e1", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "# !pip install \"langchain<0.3\" \"langchain-core<0.3\" \"langchain-community<0.3\" \"langchain-openai<0.2\"" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "936c402a", + "metadata": {}, + "outputs": [], + "source": [ + "# !pip install python-dotenv==1.2.2 chromadb==1.5.9 beautifulsoup4==4.15.0" + ] + }, + { + "cell_type": "markdown", + "id": "565a6825", + "metadata": {}, + "source": [ + "
\n", + "\n", + "## Initial Setup\n", + "\n", + "Before building anything, we need to load our API credentials, instantiate the LLM we'll use throughout the notebook, and locate the sample document we'll be querying." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "2868cff1-a3ef-426f-b8d7-7fe89047a5b7", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "from langchain.chains import RetrievalQA\n", + "from langchain_community.vectorstores import Chroma\n", + "from langchain_openai import OpenAI, OpenAIEmbeddings\n", + "from langchain_text_splitters import CharacterTextSplitter\n", + "from langchain_community.document_loaders import TextLoader" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "aeff3a2d-b0cc-429f-a34c-4381d71f1a5f", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "import os\n", + "from dotenv import load_dotenv, find_dotenv\n", + "_ = load_dotenv(find_dotenv())\n", + "\n", + "OPENAI_API_KEY = os.getenv('OPENAI_API_KEY')\n" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "14331eec-fd46-42e0-b6e7-adaf21824ef7", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "llm = OpenAI(temperature=0, api_key=OPENAI_API_KEY)" + ] + }, + { + "cell_type": "markdown", + "id": "b3f2857f", + "metadata": {}, + "source": [ + "
\n", + "\n", + "Now let's ingest the state of the union speech: load the raw text, split it into manageable chunks, embed those chunks, and store them in a Chroma vector store." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "bbb327d2", + "metadata": {}, + "outputs": [], + "source": [ + "doc_path = \"datasets/merchantofvenice.txt\"" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "f2675861", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Created a chunk of size 1100, which is longer than the specified 1000\n", + "Created a chunk of size 1549, which is longer than the specified 1000\n", + "Created a chunk of size 2106, which is longer than the specified 1000\n", + "Created a chunk of size 1554, which is longer than the specified 1000\n", + "Created a chunk of size 1042, which is longer than the specified 1000\n", + "Created a chunk of size 1058, which is longer than the specified 1000\n", + "Created a chunk of size 1004, which is longer than the specified 1000\n", + "Created a chunk of size 1543, which is longer than the specified 1000\n", + "Created a chunk of size 1135, which is longer than the specified 1000\n", + "Created a chunk of size 1017, which is longer than the specified 1000\n", + "Created a chunk of size 1202, which is longer than the specified 1000\n" + ] + } + ], + "source": [ + "loader = TextLoader(doc_path, encoding=\"utf-8\")\n", + "documents = loader.load()\n", + "text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)\n", + "texts = text_splitter.split_documents(documents)\n", + "\n", + "embeddings = OpenAIEmbeddings(api_key=OPENAI_API_KEY)\n", + "\n", + "docsearch = Chroma.from_documents(texts, embeddings, collection_name=\"state-of-union\")" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "bc5403d4", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "merchantofvenice = RetrievalQA.from_chain_type(\n", + " llm=llm, chain_type=\"stuff\", retriever=docsearch.as_retriever()\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "fe780cdc", + "metadata": {}, + "source": [ + "
\n", + "\n", + "## Adding a second knowledge source\n", + "\n", + "To show how an agent can route between multiple tools, let's add a second vector store — this time built from the Ruff FAQ web page instead of a local file." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "1431cded", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "USER_AGENT environment variable not set, consider setting it to identify your requests.\n" + ] + } + ], + "source": [ + "from langchain_community.document_loaders import WebBaseLoader" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "915d3ff3", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "loader = WebBaseLoader(\"https://beta.ruff.rs/docs/faq/\")" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "96a2edf8", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Created a chunk of size 2122, which is longer than the specified 1000\n", + "Created a chunk of size 3187, which is longer than the specified 1000\n", + "Created a chunk of size 1017, which is longer than the specified 1000\n", + "Created a chunk of size 2321, which is longer than the specified 1000\n" + ] + } + ], + "source": [ + "docs = loader.load()\n", + "ruff_texts = text_splitter.split_documents(docs)\n", + "ruff_db = Chroma.from_documents(ruff_texts, embeddings, collection_name=\"ruff\")\n", + "ruff = RetrievalQA.from_chain_type(\n", + " llm=llm, chain_type=\"stuff\", retriever=ruff_db.as_retriever()\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "c0a6c031", + "metadata": {}, + "source": [ + "
\n", + "\n", + "## Create the Agent\n", + "\n", + "With both `RetrievalQA` chains ready, we wrap each one in a `Tool` (giving it a name and a description the agent will use to decide when to call it), then hand both tools to an agent." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "eb142786", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "# Import things that are needed generically\n", + "from langchain.agents import AgentType, Tool, initialize_agent\n", + "from langchain_openai import OpenAI" + ] + }, + { + "cell_type": "markdown", + "id": "6fa28c55", + "metadata": {}, + "source": [ + "
\n", + "\n", + "Let's try it out — first with a question only the state of the union tool can answer, then one only Ruff can answer. Watch the verbose output to see which tool the agent picks each time." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "850bc4e9", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "tools = [\n", + " Tool(\n", + " name=\"Merchant of Venice QA System\",\n", + " func=merchantofvenice.run,\n", + " description=\"useful for when you need to answer questions about the merchant of venice book. Input should be a fully formed question.\",\n", + " ),\n", + " Tool(\n", + " name=\"Ruff QA System\",\n", + " func=ruff.run,\n", + " description=\"useful for when you need to answer questions about ruff (a python linter). Input should be a fully formed question.\",\n", + " ),\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "fc47f230", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "C:\\Users\\kriti\\AppData\\Local\\Temp\\ipykernel_33308\\1834837320.py:3: LangChainDeprecationWarning: The function `initialize_agent` was deprecated in LangChain 0.1.0 and will be removed in 1.0. Use Use new agent constructor methods like create_react_agent, create_json_agent, create_structured_chat_agent, etc. instead.\n", + " agent = initialize_agent(\n" + ] + } + ], + "source": [ + "# Construct the agent. We will use the default agent type here.\n", + "# See documentation for a full list of options.\n", + "agent = initialize_agent(\n", + " tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "10ca2db8", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Error in StdOutCallbackHandler.on_chain_start callback: AttributeError(\"'NoneType' object has no attribute 'get'\")\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3m I should use the Merchant of Venice QA System to answer this question.\n", + "Action: Merchant of Venice QA System\n", + "Action Input: \"Who were the suitors to Portia?\"\u001b[0m\n", + "Observation: \u001b[36;1m\u001b[1;3m The suitors to Portia were the Neapolitan prince, the County Palatine, and Bassanio.\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer.\n", + "Final Answer: The suitors to Portia were the Neapolitan prince, the County Palatine, and Bassanio.\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "{'input': 'Who were the suitors to Portia?',\n", + " 'output': 'The suitors to Portia were the Neapolitan prince, the County Palatine, and Bassanio.'}" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "agent.invoke(\n", + " \"Who were the suitors to Portia?\"\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "id": "4e91b811", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Error in StdOutCallbackHandler.on_chain_start callback: AttributeError(\"'NoneType' object has no attribute 'get'\")\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3m You should consider the differences between ruff and flake8 before deciding which one to use.\n", + "Action: Ruff QA System\n", + "Action Input: \"What are the differences between ruff and flake8?\"\u001b[0m\n", + "Observation: \u001b[33;1m\u001b[1;3m Ruff has a larger rule set and does not support custom lint rules, while Flake8 supports plugins and allows for custom and third-party rules. Ruff also has a formatter and can automatically fix its own lint violations, while Flake8 does not have these capabilities. Additionally, Ruff is written in Rust while Flake8 is written in Python.\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m Now that I know the differences, I can make a more informed decision.\n", + "Final Answer: It ultimately depends on your specific needs and preferences, but some potential reasons to choose Ruff over Flake8 could include its larger rule set and automatic fixing capabilities.\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "{'input': 'Why use ruff over flake8?',\n", + " 'output': 'It ultimately depends on your specific needs and preferences, but some potential reasons to choose Ruff over Flake8 could include its larger rule set and automatic fixing capabilities.'}" + ] + }, + "execution_count": 16, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "agent.invoke(\"Why use ruff over flake8?\")" + ] + }, + { + "cell_type": "markdown", + "id": "787a9b5e", + "metadata": {}, + "source": [ + "## Use the Agent solely as a router" + ] + }, + { + "cell_type": "markdown", + "id": "9161ba91", + "metadata": {}, + "source": [ + "
\n", + "\n", + "You can also set `return_direct=True` if you intend to use the agent as a router and just want to directly return the result of the RetrievalQAChain.\n", + "\n", + "Notice that in the above examples the agent did some extra work after querying the RetrievalQAChain. You can avoid that and just return the result directly." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "id": "f59b377e", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "tools = [\n", + " Tool(\n", + " name=\"Merchant of Venice QA System\",\n", + " func=merchantofvenice.run,\n", + " description=\"useful for when you need to answer questions about the merchant of venice book. Input should be a fully formed question.\",\n", + " return_direct=True,\n", + " ),\n", + " Tool(\n", + " name=\"Ruff QA System\",\n", + " func=ruff.run,\n", + " description=\"useful for when you need to answer questions about ruff (a python linter). Input should be a fully formed question.\",\n", + " return_direct=True,\n", + " ),\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "id": "8615707a", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "agent = initialize_agent(\n", + " tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "id": "c083c700", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Error in StdOutCallbackHandler.on_chain_start callback: AttributeError(\"'NoneType' object has no attribute 'get'\")\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3m I should use the Merchant of Venice QA System to answer this question.\n", + "Action: Merchant of Venice QA System\n", + "Action Input: \"Who were the suitors to Portia?\"\u001b[0m\n", + "Observation: \u001b[36;1m\u001b[1;3m The suitors to Portia were the Neapolitan prince, the County Palatine, and Bassanio.\u001b[0m\n", + "\u001b[32;1m\u001b[1;3m\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "{'input': 'Who were the suitors to Portia?',\n", + " 'output': ' The suitors to Portia were the Neapolitan prince, the County Palatine, and Bassanio.'}" + ] + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "agent.invoke(\n", + " \"Who were the suitors to Portia?\"\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "id": "edfd0a1a", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Error in StdOutCallbackHandler.on_chain_start callback: AttributeError(\"'NoneType' object has no attribute 'get'\")\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3m You should consider the differences between ruff and flake8 before deciding which one to use.\n", + "Action: Ruff QA System\n", + "Action Input: \"What are the differences between ruff and flake8?\"\u001b[0m\n", + "Observation: \u001b[33;1m\u001b[1;3m Ruff has a larger rule set and does not support custom lint rules, while Flake8 supports plugins and allows for custom and third-party rules. Ruff also has a formatter and can automatically fix its own lint violations, while Flake8 does not have these capabilities. Additionally, Ruff is written in Rust while Flake8 is written in Python.\u001b[0m\n", + "\u001b[32;1m\u001b[1;3m\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "{'input': 'Why use ruff over flake8?',\n", + " 'output': ' Ruff has a larger rule set and does not support custom lint rules, while Flake8 supports plugins and allows for custom and third-party rules. Ruff also has a formatter and can automatically fix its own lint violations, while Flake8 does not have these capabilities. Additionally, Ruff is written in Rust while Flake8 is written in Python.'}" + ] + }, + "execution_count": 20, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "agent.invoke(\"Why use ruff over flake8?\")" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "id": "36e718a9", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Error in StdOutCallbackHandler.on_chain_start callback: AttributeError(\"'NoneType' object has no attribute 'get'\")\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3m I should use the Merchant of Venice QA System since this question is about a political event.\n", + "Action: Merchant of Venice QA System\n", + "Action Input: \"What did Biden say about Ketanji Brown Jackson in the State of the Union address?\"\u001b[0m\n", + "Observation: \u001b[36;1m\u001b[1;3m I don't know, as this context is from a play by William Shakespeare and does not mention a State of the Union address or Biden.\u001b[0m\n", + "\u001b[32;1m\u001b[1;3m\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "{'input': 'What did biden say about ketanji brown jackson in the state of the union address?',\n", + " 'output': \" I don't know, as this context is from a play by William Shakespeare and does not mention a State of the Union address or Biden.\"}" + ] + }, + "execution_count": 21, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "agent.invoke(\n", + " \"What did biden say about ketanji brown jackson in the state of the union address?\"\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "49a0cbbe", + "metadata": {}, + "source": [ + "
\n", + "\n", + "## Multi-Hop vector store reasoning\n", + "\n", + "Because vector stores are easily usable as tools in agents, it is easy to use answer multi-hop questions that depend on vector stores using the existing agent framework." + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "id": "d397a233", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "tools = [\n", + " Tool(\n", + " name=\"Merchant of Venice QA System\",\n", + " func=merchantofvenice.run,\n", + " description=\"useful for when you need to answer questions about the merchant of venice book. Input should be a fully formed question, not referencing any obscure pronouns from the conversation before.\",\n", + " ),\n", + " Tool(\n", + " name=\"Ruff QA System\",\n", + " func=ruff.run,\n", + " description=\"useful for when you need to answer questions about ruff (a python linter). Input should be a fully formed question, not referencing any obscure pronouns from the conversation before.\",\n", + " ),\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "id": "06157240", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "# Construct the agent. We will use the default agent type here.\n", + "# See documentation for a full list of options.\n", + "agent = initialize_agent(\n", + " tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "id": "e5aa51a1", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Error in StdOutCallbackHandler.on_chain_start callback: AttributeError(\"'NoneType' object has no attribute 'get'\")\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3m Yes, ruff can be used in VScode\n", + "Action: Ruff QA System\n", + "Action Input: Can ruff be used in VScode\u001b[0m\n", + "Observation: \u001b[33;1m\u001b[1;3m Yes, Ruff can be used in VScode. It is available as a VScode extension and can be installed through the VScode marketplace.\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n", + "Final Answer: Yes, ruff can be used in VScode.\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "{'input': 'Can ruff be used in VScode',\n", + " 'output': 'Yes, ruff can be used in VScode.'}" + ] + }, + "execution_count": 24, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "agent.invoke(\n", + " \"Can ruff be used in VScode\"\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "id": "b492b520", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Error in StdOutCallbackHandler.on_chain_start callback: AttributeError(\"'NoneType' object has no attribute 'get'\")\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3m I should use the Ruff QA System to answer this question.\n", + "Action: Ruff QA System\n", + "Action Input: What tool does ruff use to run over Jupyter Notebooks? Was Bassanio a python coder and did he use Jupyter Notebooks?\u001b[0m\n", + "Observation: \u001b[33;1m\u001b[1;3m Ruff uses nbQA to run over Jupyter Notebooks. There is no information available about Bassanio's coding skills or use of Jupyter Notebooks.\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m I should use the Merchant of Venice QA System to answer the second part of the question.\n", + "Action: Merchant of Venice QA System\n", + "Action Input: Was Bassanio a python coder and did he use Jupyter Notebooks?\u001b[0m\n", + "Observation: \u001b[36;1m\u001b[1;3m I don't know.\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m I should try rephrasing the question to get more information.\n", + "Action: Merchant of Venice QA System\n", + "Action Input: Was Bassanio a programmer and did he use any coding tools?\u001b[0m\n", + "Observation: \u001b[36;1m\u001b[1;3m No, there is no indication in the context that Bassanio was a programmer or used any coding tools. He is described as a gentleman and is engaged in business ventures involving ships and trade, but there is no mention of technology or coding.\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer.\n", + "Final Answer: Ruff uses nbQA to run over Jupyter Notebooks, but there is no information available about Bassanio's coding skills or use of Jupyter Notebooks.\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "{'input': 'What tool does ruff use to run over Jupyter Notebooks? Was Bassanio a python coder and did he use Jupyter Notebooks?',\n", + " 'output': \"Ruff uses nbQA to run over Jupyter Notebooks, but there is no information available about Bassanio's coding skills or use of Jupyter Notebooks.\"}" + ] + }, + "execution_count": 25, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "agent.invoke(\n", + " \"What tool does ruff use to run over Jupyter Notebooks? Was Bassanio a python coder and did he use Jupyter Notebooks?\"\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "id": "5d4a3c23", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Error in StdOutCallbackHandler.on_chain_start callback: AttributeError(\"'NoneType' object has no attribute 'get'\")\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3m I should check if Ruff supports Python first before answering the second question.\n", + "Action: Ruff QA System\n", + "Action Input: Does Ruff support Python?\u001b[0m\n", + "Observation: \u001b[33;1m\u001b[1;3m Yes, Ruff supports Python versions 3.7 and above.\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m Now that I know Ruff supports Python, I can answer the second question.\n", + "Action: Merchant of Venice QA System\n", + "Action Input: Was Bassanio a python coder?\u001b[0m\n", + "Observation: \u001b[36;1m\u001b[1;3m I don't know.\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m I should try rephrasing the question.\n", + "Action: Merchant of Venice QA System\n", + "Action Input: Was Bassanio a coder in Python?\u001b[0m\n", + "Observation: \u001b[36;1m\u001b[1;3m I don't know.\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m I should try asking about Bassanio's profession instead.\n", + "Action: Merchant of Venice QA System\n", + "Action Input: What was Bassanio's profession?\u001b[0m\n", + "Observation: \u001b[36;1m\u001b[1;3m Bassanio's profession was not explicitly stated in the given context. However, it can be inferred that he is a wealthy merchant, as he is able to borrow a large sum of money from Antonio and is also able to afford a trip to Belmont.\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer.\n", + "Final Answer: Bassanio's profession was not explicitly stated, but he is most likely a wealthy merchant.\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "{'input': 'Does Ruff support Python? Was Bassanio a python coder?',\n", + " 'output': \"Bassanio's profession was not explicitly stated, but he is most likely a wealthy merchant.\"}" + ] + }, + "execution_count": 26, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "agent.invoke(\n", + " \"Does Ruff support Python? Was Bassanio a python coder?\"\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "id": "337ee0a5", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Error in StdOutCallbackHandler.on_chain_start callback: AttributeError(\"'NoneType' object has no attribute 'get'\")\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3m I should use the Ruff QA System to answer the first question and the Merchant of Venice QA System to answer the second question.\n", + "Action: Ruff QA System\n", + "Action Input: Does Ruff support Python?\u001b[0m\n", + "Observation: \u001b[33;1m\u001b[1;3m Yes, Ruff supports Python versions 3.7 and above.\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m Now I should use the Merchant of Venice QA System to answer the second question.\n", + "Action: Merchant of Venice QA System\n", + "Action Input: Who did Portia marry?\u001b[0m\n", + "Observation: \u001b[36;1m\u001b[1;3m Bassanio\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer.\n", + "Final Answer: Bassanio\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "{'input': 'Does Ruff support Python? Who did Portia marry?',\n", + " 'output': 'Bassanio'}" + ] + }, + "execution_count": 27, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "agent.invoke(\n", + " \"Does Ruff support Python? Who did Portia marry?\"\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "id": "f3c94573", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Error in StdOutCallbackHandler.on_chain_start callback: AttributeError(\"'NoneType' object has no attribute 'get'\")\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3m I should use the Ruff QA System to answer this question since it is about Pylint.\n", + "Action: Ruff QA System\n", + "Action Input: What is the difference between Ruff and Pylint?\u001b[0m\n", + "Observation: \u001b[33;1m\u001b[1;3m Ruff and Pylint are both code quality tools for Python, but they have different approaches and capabilities. Ruff is a linter, which means it checks for code style and potential errors, while Pylint is a type checker, which means it checks for type errors and potential bugs. Ruff has a larger rule set and can automatically fix some lint violations, but it does not support custom or third-party rules like Pylint does. Additionally, Ruff is designed to be used alongside a type checker, while Pylint can be used on its own.\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m Now I need to find out if Shylock knew about Pylint.\n", + "Action: Merchant of Venice QA System\n", + "Action Input: Did Shylock know about Pylint?\u001b[0m\n", + "Observation: \u001b[36;1m\u001b[1;3m No, Shylock did not know about Pylint.\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer.\n", + "Final Answer: No, Shylock did not know about Pylint. The difference between Ruff and Pylint is that Ruff is a linter and Pylint is a type checker.\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "{'input': 'Did Shylock know about Pylint? What is the difference between Ruff and Pylint?',\n", + " 'output': 'No, Shylock did not know about Pylint. The difference between Ruff and Pylint is that Ruff is a linter and Pylint is a type checker.'}" + ] + }, + "execution_count": 28, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "agent.invoke(\n", + " \"Did Shylock know about Pylint? What is the difference between Ruff and Pylint?\"\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "id": "9cee70d2", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Error in StdOutCallbackHandler.on_chain_start callback: AttributeError(\"'NoneType' object has no attribute 'get'\")\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3m This is a simple math question that can be easily answered.\n", + "Action: None\n", + "Action Input: None\u001b[0m\n", + "Observation: None is not a valid tool, try one of [Merchant of Venice QA System, Ruff QA System].\n", + "Thought:\u001b[32;1m\u001b[1;3m I should use a math calculator to solve this question.\n", + "Action: None\n", + "Action Input: None\u001b[0m\n", + "Observation: None is not a valid tool, try one of [Merchant of Venice QA System, Ruff QA System].\n", + "Thought:\u001b[32;1m\u001b[1;3m I should use a math library to solve this question.\n", + "Action: None\n", + "Action Input: None\u001b[0m\n", + "Observation: None is not a valid tool, try one of [Merchant of Venice QA System, Ruff QA System].\n", + "Thought:\u001b[32;1m\u001b[1;3m I should use a math function to solve this question.\n", + "Action: None\n", + "Action Input: None\u001b[0m\n", + "Observation: None is not a valid tool, try one of [Merchant of Venice QA System, Ruff QA System].\n", + "Thought:\u001b[32;1m\u001b[1;3m I should use a calculator tool to solve this question.\n", + "Action: Merchant of Venice QA System\n", + "Action Input: \"What is 2+2?\"\u001b[0m\n", + "Observation: \u001b[36;1m\u001b[1;3m I don't know.\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m I should try a different tool.\n", + "Action: Ruff QA System\n", + "Action Input: \"What is 2+2?\"\u001b[0m\n", + "Observation: \u001b[33;1m\u001b[1;3m I don't know, as this question is not related to the context provided.\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m I should try a different question.\n", + "Action: Merchant of Venice QA System\n", + "Action Input: \"What is the capital of Italy?\"\u001b[0m\n", + "Observation: \u001b[36;1m\u001b[1;3m I don't know.\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m I should try a different tool.\n", + "Action: Ruff QA System\n", + "Action Input: \"What is the capital of Italy?\"\u001b[0m\n", + "Observation: \u001b[33;1m\u001b[1;3m I don't know.\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer.\n", + "Final Answer: None of the tools provided can answer the question \"What is 2+2?\" as it is not related to the context provided.\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "{'input': 'What is 2+2?',\n", + " 'output': 'None of the tools provided can answer the question \"What is 2+2?\" as it is not related to the context provided.'}" + ] + }, + "execution_count": 29, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "agent.invoke(\n", + " \"What is 2+2?\"\n", + ")" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "langchain-v0.2", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.13.14" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}