From d768bf9246336405190a892694b068bef9171462 Mon Sep 17 00:00:00 2001 From: karimamzghi Date: Wed, 19 Aug 2026 12:16:06 +0200 Subject: [PATCH] completing lab --- lab-extractive-question-answering.ipynb | 1496 ++++++++++++----------- 1 file changed, 760 insertions(+), 736 deletions(-) diff --git a/lab-extractive-question-answering.ipynb b/lab-extractive-question-answering.ipynb index 7a04dde..3f4079c 100644 --- a/lab-extractive-question-answering.ipynb +++ b/lab-extractive-question-answering.ipynb @@ -1,738 +1,762 @@ { - "cells": [ - { - "cell_type": "markdown", - "id": "split-aluminum", - "metadata": { - "id": "split-aluminum", - "papermill": { - "duration": 0.048394, - "end_time": "2021-04-15T21:06:39.560571", - "exception": false, - "start_time": "2021-04-15T21:06:39.512177", - "status": "completed" - }, - "tags": [] - }, - "source": [ - "# LAB | Extractive Question Answering" - ] - }, - { - "cell_type": "markdown", - "id": "e3f44179", - "metadata": {}, - "source": [ - "
\n", - "\n", - "
\n", - "\n", - "**Run this notebook in [Google Colab](https://colab.research.google.com/) for GPU acceleration.**\n", - "\n", - "
\n", - "\n", - "
" - ] - }, - { - "cell_type": "markdown", - "id": "prospective-turner", - "metadata": { - "id": "prospective-turner", - "papermill": { - "duration": 0.045573, - "end_time": "2021-04-15T21:06:39.651272", - "exception": false, - "start_time": "2021-04-15T21:06:39.605699", - "status": "completed" - }, - "tags": [] - }, - "source": [ - "This notebook demonstrates how Pinecone helps you build an extractive question-answering application. To build an extractive question-answering system, we need three main components:\n", - "\n", - "- A vector index to store and run semantic search\n", - "- A retriever model for embedding context passages\n", - "- A reader model to extract answers\n", - "\n", - "We will use the SQuAD dataset, which consists of **questions** and **context** paragraphs containing question **answers**. We generate embeddings for the context passages using the retriever, index them in the vector database, and query with semantic search to retrieve the top k most relevant contexts containing potential answers to our question. We then use the reader model to extract the answers from the returned contexts." - ] - }, - { - "cell_type": "markdown", - "id": "oC3GG-dWkZJ6", - "metadata": { - "id": "oC3GG-dWkZJ6" - }, - "source": [ - "Let's get started by installing the packages needed for notebook to run:" - ] - }, - { - "cell_type": "markdown", - "id": "terminal-export", - "metadata": { - "id": "terminal-export", - "papermill": { - "duration": 0.044413, - "end_time": "2021-04-15T21:06:39.741951", - "exception": false, - "start_time": "2021-04-15T21:06:39.697538", - "status": "completed" - }, - "tags": [] - }, - "source": [ - "# Install Dependencies" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "expressed-executive", - "metadata": { - "execution": { - "iopub.execute_input": "2021-04-15T21:06:39.845309Z", - "iopub.status.busy": "2021-04-15T21:06:39.842494Z", - "iopub.status.idle": "2021-04-15T21:08:22.163939Z", - "shell.execute_reply": "2021-04-15T21:08:22.164616Z" - }, - "id": "expressed-executive", - "papermill": { - "duration": 102.376674, - "end_time": "2021-04-15T21:08:22.165052", - "exception": false, - "start_time": "2021-04-15T21:06:39.788378", - "status": "completed" - }, - "tags": [] - }, - "outputs": [], - "source": [ - "!pip install -q \\\n", - " transformers==4.46.3 \\\n", - " datasets==4.5.0 \\\n", - " pinecone==5.4.2 \\\n", - " sentence-transformers==3.3.1 \\\n", - " huggingface_hub==0.26.5\n", - " \n", - "!rm -rf ~/.cache/huggingface" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "2DCPtl6IhgSz", - "metadata": { - "id": "2DCPtl6IhgSz" - }, - "outputs": [], - "source": [ - "# Force-kill the kernel process to restart it, ensuring the newly\n", - "# installed/uninstalled package versions are loaded in a fresh process\n", - "\n", - "import os\n", - "import sys\n", - "\n", - "print(\n", - " \"⚠️ WARNING:\\n\"\n", - " \"This code cell will restart the kernel to load the newly installed packages...\\n\"\n", - " \"If you used 'Run all', it will stop here — that's expected.\\n\"\n", - " \"See the note below on how to continue running the rest of the notebook.\",\n", - " flush=True,\n", - ")\n", - "sys.stdout.flush()\n", - "\n", - "os.kill(os.getpid(), 9)" - ] - }, - { - "cell_type": "markdown", - "id": "d339487b", - "metadata": {}, - "source": [ - "
\n", - "\n", - "⚠️ **Do not use \"Run all\" for this whole notebook in one go.** The cell above force-restarts the kernel (`os.kill`) so the newly installed packages load cleanly. Colab's \"Run all\" cannot continue past a kernel restart — it will always stop right after that cell, no matter how many times you retry it.\n", - "\n", - "Instead, run it in two steps:\n", - "1. Run only the two cells above (install + restart). Wait for the kernel to finish restarting (a few seconds).\n", - "2. Click on the next cell below, then use **Runtime → Run after** (or manually run the remaining cells one by one / select them and press Shift+Enter). Do not click \"Run all\" again, since it would just re-trigger the restart at the same spot.\n", - "\n", - "
" - ] - }, - { - "cell_type": "markdown", - "id": "29ad3840", - "metadata": { - "id": "29ad3840" - }, - "source": [ - "# Load Dataset" - ] - }, - { - "cell_type": "markdown", - "id": "hgIieQukgagu", - "metadata": { - "id": "hgIieQukgagu" - }, - "source": [ - "Now let's load the SQUAD dataset from the HuggingFace Model Hub. We load the dataset into a pandas dataframe and filter the title, question, and context columns, and we drop any duplicate context passages." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "J250IJeh7NIb", - "metadata": { - "id": "J250IJeh7NIb" - }, - "outputs": [], - "source": [ - "from datasets import load_dataset\n", - "\n", - "# load the squad dataset into a pandas dataframe\n", - "df = load_dataset(\"rajpurkar/squad\", split=\"train\").to_pandas()" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "FcmeNO97dHDO", - "metadata": { - "id": "FcmeNO97dHDO" - }, - "outputs": [], - "source": [ - "# select only title and context column\n", - "df = None\n", - "# drop rows containing duplicate context passages\n", - "df = None\n", - "df" - ] - }, - { - "cell_type": "markdown", - "id": "57bbcb57", - "metadata": { - "id": "57bbcb57" - }, - "source": [ - "# Pinecone Setup" - ] - }, - { - "cell_type": "markdown", - "id": "e24d904c", - "metadata": { - "id": "e24d904c" - }, - "source": [ - "Pinecone is a managed vector database that lets us store context embeddings and run fast semantic search over them. In this section, we set up the connection to Pinecone using an API key and create the index that will hold our context embeddings." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "472402a1", - "metadata": {}, - "outputs": [], - "source": [ - "import os\n", - "import getpass\n", - "from pinecone import Pinecone\n", - "\n", - "#\n", - "# For this notebook, you'll need a Pinecone API key, \n", - "# to get one, sign up for free at https://app.pinecone.io/,\n", - "# then go to \"API Keys\" and copy the default key (or create a new one).\n", - "#\n", - "# Once you have your Pinecone API key,\n", - "# load it from Colab's \"Secrets\" manager \n", - "# (the key icon in the left sidebar). \n", - "# Add a secret named PINECONE_API_KEY there and enable notebook access.\n", - "#\n", - "# Note: Colab Secrets rely on a UI popup to grant access, which only works when\n", - "# running in the Colab UI itself. If you're running this notebook through\n", - "# another editor (e.g. VS Code's Colab extension), that popup isn't available,\n", - "# so we fall back to an environment variable or a secure manual prompt instead.\n", - "api_key = os.environ.get('PINECONE_API_KEY')\n", - "\n", - "\n", - "if not api_key:\n", - " try:\n", - " from google.colab import userdata\n", - " api_key = userdata.get('PINECONE_API_KEY')\n", - " except Exception:\n", - " api_key = None\n", - "\n", - "if not api_key:\n", - " api_key = getpass.getpass('Enter your Pinecone API key: ')\n", - "\n" - ] - }, - { - "cell_type": "markdown", - "id": "7ca5ef09", - "metadata": {}, - "source": [ - "
\n", - "\n", - "Once we have the API key, we can initialize the connection:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "092d1e71", - "metadata": { - "id": "092d1e71" - }, - "outputs": [], - "source": [ - "from pinecone import Pinecone, ServerlessSpec\n", - "\n", - "spec = ServerlessSpec(\n", - " cloud=\"aws\", region=\"us-east-1\"\n", - ")\n", - "\n", - "# connect to pinecone environment\n", - "pc = Pinecone(\n", - " api_key = api_key\n", - ")" - ] - }, - { - "cell_type": "markdown", - "id": "58028e12", - "metadata": { - "id": "58028e12" - }, - "source": [ - "Now we create a new index called \"question-answering\" — we can name the index anything we want. We specify the metric type as \"cosine\" and dimension as 384 because the retriever we use to generate context embeddings is optimized for cosine similarity and outputs 384-dimension vectors." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "b3206184", - "metadata": { - "id": "b3206184" - }, - "outputs": [], - "source": [ - "index_name = None\n", - "\n", - "# check if the extractive-question-answering index exists\n", - "if index_name not in pinecone.list_indexes().names():\n", - " # create the index if it does not exist\n", - " None\n", - "# connect to extractive-question-answering index we created\n", - "index = pinecone.Index(index_name)" - ] - }, - { - "cell_type": "markdown", - "id": "6e84a3e5", - "metadata": { - "id": "6e84a3e5" - }, - "source": [ - "# Initialize Retriever" - ] - }, - { - "cell_type": "markdown", - "id": "oZzhGS1Lpj0g", - "metadata": { - "id": "oZzhGS1Lpj0g" - }, - "source": [ - "Next, we need to initialize our retriever. The retriever will mainly do two things:\n", - "\n", - "- Generate embeddings for all context passages (context vectors/embeddings)\n", - "- Generate embeddings for our questions (query vector/embedding)\n", - "\n", - "The retriever will generate embeddings in a way that the questions and context passages containing answers to our questions are nearby in the vector space. We can use cosine similarity to calculate the similarity between the query and context embeddings to find the context passages that contain potential answers to our question.\n", - "\n", - "We will use a SentenceTransformer model named ``multi-qa-MiniLM-L6-cos-v1`` designed for semantic search and trained on 215M (question, answer) pairs from diverse sources as our retriever." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "31a85bb3", - "metadata": { - "id": "31a85bb3" - }, - "outputs": [], - "source": [ - "import torch\n", - "from sentence_transformers import SentenceTransformer\n", - "\n", - "# set device to GPU if available\n", - "device = 'cuda' if torch.cuda.is_available() else 'cpu'\n", - "# load the retriever model from huggingface model hub\n", - "retriever = None #use the 'multi-qa-MiniLM-L6-cos-v1' model from HuggingFace to build the retriever\n", - "\n", - "retriever" - ] - }, - { - "cell_type": "markdown", - "id": "8aaad0a2", - "metadata": { - "id": "8aaad0a2" - }, - "source": [ - "# Generate Embeddings and Upsert" - ] - }, - { - "cell_type": "markdown", - "id": "Hgy7AagJtO_p", - "metadata": { - "id": "Hgy7AagJtO_p" - }, - "source": [ - "Next, we need to generate embeddings for the context passages. We will do this in batches to help us more quickly generate embeddings and upload them to the Pinecone index. When passing the documents to Pinecone, we need an id (a unique value), context embedding, and metadata for each document representing context passages in the dataset. The metadata is a dictionary containing data relevant to our embeddings, such as the article title, context passage, etc." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "a17824ef", - "metadata": { - "id": "a17824ef", - "tags": [] - }, - "outputs": [], - "source": [ - "from tqdm.auto import tqdm\n", - "\n", - "# we will use batches of 64\n", - "batch_size = 64\n", - "\n", - "for i in tqdm(range(0, len(df), batch_size)):\n", - " # find end of batch\n", - " None\n", - " # extract batch\n", - " None\n", - " # generate embeddings for batch\n", - " emb = None\n", - " # get metadata\n", - " meta = None\n", - " # create unique IDs\n", - " ids = None\n", - " # add all to upsert list\n", - " to_upsert = None\n", - " # upsert/insert these records to pinecone\n", - " _ = index.upsert(vectors=to_upsert)\n", - "\n", - "# check that we have all vectors in index\n", - "index.describe_index_stats()" - ] - }, - { - "cell_type": "markdown", - "id": "YFyBYafuJ0y0", - "metadata": { - "id": "YFyBYafuJ0y0" - }, - "source": [ - "# Initialize Reader" - ] - }, - { - "cell_type": "markdown", - "id": "HgdiLCz5ynOk", - "metadata": { - "id": "HgdiLCz5ynOk" - }, - "source": [ - "We use the `deepset/electra-base-squad2` model from the HuggingFace model hub as our reader model. We load this model into a \"question-answering\" pipeline from HuggingFace transformers and feed it our questions and context passages individually. The model gives a prediction for each context we pass through the pipeline." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "hg9XTDkIJzH_", - "metadata": { - "id": "hg9XTDkIJzH_" - }, - "outputs": [], - "source": [ - "from transformers import pipeline\n", - "\n", - "model_name = 'deepset/electra-base-squad2'\n", - "# load the reader model into a question-answering pipeline\n", - "\n", - "device_id = 0 if torch.cuda.is_available() else -1\n", - "reader = pipeline(tokenizer=model_name, model=model_name, task='question-answering', device=device_id)\n", - "\n", - "reader" - ] - }, - { - "cell_type": "markdown", - "id": "e14d89d6", - "metadata": { - "id": "e14d89d6" - }, - "source": [ - "Now all the components we need are ready. Let's write some helper functions to execute our queries. The `get_context` function retrieves the context embeddings containing answers to our question from the Pinecone index, and the `extract_answer` function extracts the answers from these context passages." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "lyYaY3QEQiHZ", - "metadata": { - "id": "lyYaY3QEQiHZ" - }, - "outputs": [], - "source": [ - "# gets context passages from the pinecone index\n", - "def get_context(question, top_k):\n", - " # generate embeddings for the question\n", - " xq = None\n", - " # search pinecone index for context passage with the answer\n", - " xc = None\n", - " # extract the context passage from pinecone search result\n", - " c = None\n", - " return c\n", - "\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "Dc9VYOiUQA7B", - "metadata": { - "id": "Dc9VYOiUQA7B" - }, - "outputs": [], - "source": [ - "from pprint import pprint\n", - "\n", - "# extracts answer from the context passage\n", - "def extract_answer(question, context):\n", - " results = []\n", - " for c in context:\n", - " # feed the reader the question and contexts to extract answers\n", - " answer = reader(question=question, context=c)\n", - " # add the context to answer dict for printing both together\n", - " answer[\"context\"] = c\n", - " results.append(answer)\n", - " # sort the result based on the score from reader model\n", - " sorted_result = pprint(sorted(results, key=lambda x: x['score'], reverse=True))\n", - " return sorted_result" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "5E3a3dkJ5ZQD", - "metadata": { - "id": "5E3a3dkJ5ZQD" - }, - "outputs": [], - "source": [ - "question = \"How much oil is Egypt producing in a day?\"\n", - "context = get_context(question, top_k = 1)\n", - "context" - ] - }, - { - "cell_type": "markdown", - "id": "heKNVbWQ_LtC", - "metadata": { - "id": "heKNVbWQ_LtC" - }, - "source": [ - "As we can see, the retiever is working fine and gets us the context passage that contains the answer to our question. Now let's use the reader to extract the exact answer from the context passage." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "DQ4GWdbMSjPl", - "metadata": { - "id": "DQ4GWdbMSjPl" - }, - "outputs": [], - "source": [ - "extract_answer(question, context)" - ] - }, - { - "cell_type": "markdown", - "id": "fMD_ABuDAyhN", - "metadata": { - "id": "fMD_ABuDAyhN" - }, - "source": [ - "The reader model predicted with 99% accuracy the correct answer *691,000 bbl/d* as seen from the context passage. Let's run few more queries." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "_4NRgV4mGWoj", - "metadata": { - "id": "_4NRgV4mGWoj" - }, - "outputs": [], - "source": [ - "question = \"What are the first names of the men that invented youtube?\"\n", - "context = get_context(question, top_k=1)\n", - "extract_answer(question, context)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "juXlctWgJgMF", - "metadata": { - "id": "juXlctWgJgMF" - }, - "outputs": [], - "source": [ - "question = \"What is Albert Eistein famous for?\"\n", - "context = get_context(question, top_k=1)\n", - "extract_answer(question, context)" - ] - }, - { - "cell_type": "markdown", - "id": "OhCgeny_BVno", - "metadata": { - "id": "OhCgeny_BVno" - }, - "source": [ - "Let's run another question. This time for top 3 context passages from the retriever." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "iXACn71xmett", - "metadata": { - "id": "iXACn71xmett" - }, - "outputs": [], - "source": [ - "question = \"Who was the first person to step foot on the moon?\"\n", - "context = get_context(question, top_k=3)\n", - "extract_answer(question, context)" - ] - }, - { - "cell_type": "markdown", - "id": "27fecadc-6976-43b2-90e1-788a8633ecc7", - "metadata": { - "id": "27fecadc-6976-43b2-90e1-788a8633ecc7" - }, - "source": [ - "
\n", - "\n", - "### 🚀 Challenge\n", - "\n", - "Try adding a few more questions of your own in new cells below (reusing `get_context` and `extract_answer`). Experiment with different `top_k` values and observe:\n", - "\n", - "- Does the retriever always return a context that actually contains the answer?\n", - "- How does the reader's confidence score change with harder or more ambiguous questions?\n", - "- What happens with questions that have no good answer in the dataset?" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "898daa2f", - "metadata": {}, - "outputs": [], - "source": [ - "# ...\n", - "# ...\n", - "# ..." - ] - }, - { - "cell_type": "markdown", - "id": "d4ed6384", - "metadata": {}, - "source": [ - "
\n", - "\n", - "---\n", - "\n", - "### Clean Up\n", - "\n", - "Now that we're done experimenting, let's delete the Pinecone index we created. This frees up the resources it was using (and, when using a paid plan, avoids being billed for an index we no longer need)." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "id": "c53493d1", - "metadata": {}, - "outputs": [], - "source": [ - "# To delete the Pinecone index, \n", - "# uncomment the line below + run this code cell\n", - "\n", - "# pc.delete_index(index_name)" - ] - } - ], - "metadata": { - "accelerator": "GPU", - "colab": { - "gpuType": "T4", - "machine_shape": "hm", - "provenance": [] - }, - "environment": { - "name": "tf2-gpu.2-3.m65", - "type": "gcloud", - "uri": "gcr.io/deeplearning-platform-release/tf2-gpu.2-3:m65" - }, - "kernelspec": { - "display_name": "Python 3", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.11.8" - }, + "cells": [ + { + "cell_type": "markdown", + "id": "split-aluminum", + "metadata": { + "id": "split-aluminum", "papermill": { - "default_parameters": {}, - "duration": 333.240754, - "end_time": "2021-04-15T21:12:11.363566", - "environment_variables": {}, - "exception": null, - "input_path": "/notebooks/question_answering/question_answering.ipynb", - "output_path": "/notebooks/tmp/question_answering/question_answering.ipynb", - "parameters": {}, - "start_time": "2021-04-15T21:06:38.122812", - "version": "2.3.3" - }, - "vscode": { - "interpreter": { - "hash": "5fe10bf018ef3e697f9035d60bf60847932a12bface18908407fd371fe880db9" - } - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} + "duration": 0.048394, + "end_time": "2021-04-15T21:06:39.560571", + "exception": false, + "start_time": "2021-04-15T21:06:39.512177", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "# LAB | Extractive Question Answering" + ] + }, + { + "cell_type": "markdown", + "id": "e3f44179", + "metadata": {}, + "source": [ + "
\n", + "\n", + "
\n", + "\n", + "**Run this notebook in [Google Colab](https://colab.research.google.com/) for GPU acceleration.**\n", + "\n", + "
\n", + "\n", + "
" + ] + }, + { + "cell_type": "markdown", + "id": "prospective-turner", + "metadata": { + "id": "prospective-turner", + "papermill": { + "duration": 0.045573, + "end_time": "2021-04-15T21:06:39.651272", + "exception": false, + "start_time": "2021-04-15T21:06:39.605699", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "This notebook demonstrates how Pinecone helps you build an extractive question-answering application. To build an extractive question-answering system, we need three main components:\n", + "\n", + "- A vector index to store and run semantic search\n", + "- A retriever model for embedding context passages\n", + "- A reader model to extract answers\n", + "\n", + "We will use the SQuAD dataset, which consists of **questions** and **context** paragraphs containing question **answers**. We generate embeddings for the context passages using the retriever, index them in the vector database, and query with semantic search to retrieve the top k most relevant contexts containing potential answers to our question. We then use the reader model to extract the answers from the returned contexts." + ] + }, + { + "cell_type": "markdown", + "id": "oC3GG-dWkZJ6", + "metadata": { + "id": "oC3GG-dWkZJ6" + }, + "source": [ + "Let's get started by installing the packages needed for notebook to run:" + ] + }, + { + "cell_type": "markdown", + "id": "terminal-export", + "metadata": { + "id": "terminal-export", + "papermill": { + "duration": 0.044413, + "end_time": "2021-04-15T21:06:39.741951", + "exception": false, + "start_time": "2021-04-15T21:06:39.697538", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "# Install Dependencies" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "expressed-executive", + "metadata": { + "execution": { + "iopub.execute_input": "2021-04-15T21:06:39.845309Z", + "iopub.status.busy": "2021-04-15T21:06:39.842494Z", + "iopub.status.idle": "2021-04-15T21:08:22.163939Z", + "shell.execute_reply": "2021-04-15T21:08:22.164616Z" + }, + "id": "expressed-executive", + "papermill": { + "duration": 102.376674, + "end_time": "2021-04-15T21:08:22.165052", + "exception": false, + "start_time": "2021-04-15T21:06:39.788378", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "!pip install -q \\\n", + " transformers==4.46.3 \\\n", + " datasets==4.5.0 \\\n", + " pinecone==5.4.2 \\\n", + " sentence-transformers==3.3.1 \\\n", + " huggingface_hub==0.26.5\n", + " \n", + "!rm -rf ~/.cache/huggingface" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2DCPtl6IhgSz", + "metadata": { + "id": "2DCPtl6IhgSz" + }, + "outputs": [], + "source": [ + "# Force-kill the kernel process to restart it, ensuring the newly\n", + "# installed/uninstalled package versions are loaded in a fresh process\n", + "\n", + "import os\n", + "import sys\n", + "\n", + "print(\n", + " \"\u26a0\ufe0f WARNING:\\n\"\n", + " \"This code cell will restart the kernel to load the newly installed packages...\\n\"\n", + " \"If you used 'Run all', it will stop here \u2014 that's expected.\\n\"\n", + " \"See the note below on how to continue running the rest of the notebook.\",\n", + " flush=True,\n", + ")\n", + "sys.stdout.flush()\n", + "\n", + "os.kill(os.getpid(), 9)" + ] + }, + { + "cell_type": "markdown", + "id": "d339487b", + "metadata": {}, + "source": [ + "
\n", + "\n", + "\u26a0\ufe0f **Do not use \"Run all\" for this whole notebook in one go.** The cell above force-restarts the kernel (`os.kill`) so the newly installed packages load cleanly. Colab's \"Run all\" cannot continue past a kernel restart \u2014 it will always stop right after that cell, no matter how many times you retry it.\n", + "\n", + "Instead, run it in two steps:\n", + "1. Run only the two cells above (install + restart). Wait for the kernel to finish restarting (a few seconds).\n", + "2. Click on the next cell below, then use **Runtime \u2192 Run after** (or manually run the remaining cells one by one / select them and press Shift+Enter). Do not click \"Run all\" again, since it would just re-trigger the restart at the same spot.\n", + "\n", + "
" + ] + }, + { + "cell_type": "markdown", + "id": "29ad3840", + "metadata": { + "id": "29ad3840" + }, + "source": [ + "# Load Dataset" + ] + }, + { + "cell_type": "markdown", + "id": "hgIieQukgagu", + "metadata": { + "id": "hgIieQukgagu" + }, + "source": [ + "Now let's load the SQUAD dataset from the HuggingFace Model Hub. We load the dataset into a pandas dataframe and filter the title, question, and context columns, and we drop any duplicate context passages." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "J250IJeh7NIb", + "metadata": { + "id": "J250IJeh7NIb" + }, + "outputs": [], + "source": [ + "from datasets import load_dataset\n", + "\n", + "# load the squad dataset into a pandas dataframe\n", + "df = load_dataset(\"rajpurkar/squad\", split=\"train\").to_pandas()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "FcmeNO97dHDO", + "metadata": { + "id": "FcmeNO97dHDO" + }, + "outputs": [], + "source": [ + "# select only title and context columns\n", + "df = df[[\"title\", \"context\"]]\n", + "\n", + "# drop rows containing duplicate context passages\n", + "df = df.drop_duplicates(subset=\"context\").reset_index(drop=True)\n", + "\n", + "df\n" + ] + }, + { + "cell_type": "markdown", + "id": "57bbcb57", + "metadata": { + "id": "57bbcb57" + }, + "source": [ + "# Pinecone Setup" + ] + }, + { + "cell_type": "markdown", + "id": "e24d904c", + "metadata": { + "id": "e24d904c" + }, + "source": [ + "Pinecone is a managed vector database that lets us store context embeddings and run fast semantic search over them. In this section, we set up the connection to Pinecone using an API key and create the index that will hold our context embeddings." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "472402a1", + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "import getpass\n", + "from pinecone import Pinecone\n", + "\n", + "#\n", + "# For this notebook, you'll need a Pinecone API key, \n", + "# to get one, sign up for free at https://app.pinecone.io/,\n", + "# then go to \"API Keys\" and copy the default key (or create a new one).\n", + "#\n", + "# Once you have your Pinecone API key,\n", + "# load it from Colab's \"Secrets\" manager \n", + "# (the key icon in the left sidebar). \n", + "# Add a secret named PINECONE_API_KEY there and enable notebook access.\n", + "#\n", + "# Note: Colab Secrets rely on a UI popup to grant access, which only works when\n", + "# running in the Colab UI itself. If you're running this notebook through\n", + "# another editor (e.g. VS Code's Colab extension), that popup isn't available,\n", + "# so we fall back to an environment variable or a secure manual prompt instead.\n", + "api_key = os.environ.get('PINECONE_API_KEY')\n", + "\n", + "\n", + "if not api_key:\n", + " try:\n", + " from google.colab import userdata\n", + " api_key = userdata.get('PINECONE_API_KEY')\n", + " except Exception:\n", + " api_key = None\n", + "\n", + "if not api_key:\n", + " api_key = getpass.getpass('Enter your Pinecone API key: ')\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "7ca5ef09", + "metadata": {}, + "source": [ + "
\n", + "\n", + "Once we have the API key, we can initialize the connection:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "092d1e71", + "metadata": { + "id": "092d1e71" + }, + "outputs": [], + "source": [ + "from pinecone import Pinecone, ServerlessSpec\n", + "\n", + "spec = ServerlessSpec(\n", + " cloud=\"aws\", region=\"us-east-1\"\n", + ")\n", + "\n", + "# connect to pinecone environment\n", + "pc = Pinecone(\n", + " api_key = api_key\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "58028e12", + "metadata": { + "id": "58028e12" + }, + "source": [ + "Now we create a new index called \"question-answering\" \u2014 we can name the index anything we want. We specify the metric type as \"cosine\" and dimension as 384 because the retriever we use to generate context embeddings is optimized for cosine similarity and outputs 384-dimension vectors." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b3206184", + "metadata": { + "id": "b3206184" + }, + "outputs": [], + "source": [ + "index_name = \"question-answering\"\n", + "\n", + "# check if the question-answering index exists\n", + "if index_name not in pc.list_indexes().names():\n", + " # create the index if it does not exist\n", + " pc.create_index(\n", + " name=index_name,\n", + " dimension=384,\n", + " metric=\"cosine\",\n", + " spec=spec,\n", + " )\n", + "\n", + "# connect to the question-answering index we created\n", + "index = pc.Index(index_name)\n" + ] + }, + { + "cell_type": "markdown", + "id": "6e84a3e5", + "metadata": { + "id": "6e84a3e5" + }, + "source": [ + "# Initialize Retriever" + ] + }, + { + "cell_type": "markdown", + "id": "oZzhGS1Lpj0g", + "metadata": { + "id": "oZzhGS1Lpj0g" + }, + "source": [ + "Next, we need to initialize our retriever. The retriever will mainly do two things:\n", + "\n", + "- Generate embeddings for all context passages (context vectors/embeddings)\n", + "- Generate embeddings for our questions (query vector/embedding)\n", + "\n", + "The retriever will generate embeddings in a way that the questions and context passages containing answers to our questions are nearby in the vector space. We can use cosine similarity to calculate the similarity between the query and context embeddings to find the context passages that contain potential answers to our question.\n", + "\n", + "We will use a SentenceTransformer model named ``multi-qa-MiniLM-L6-cos-v1`` designed for semantic search and trained on 215M (question, answer) pairs from diverse sources as our retriever." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "31a85bb3", + "metadata": { + "id": "31a85bb3" + }, + "outputs": [], + "source": [ + "import torch\n", + "from sentence_transformers import SentenceTransformer\n", + "\n", + "# set device to GPU if available\n", + "device = 'cuda' if torch.cuda.is_available() else 'cpu'\n", + "\n", + "# load the retriever model from HuggingFace model hub\n", + "retriever = SentenceTransformer('multi-qa-MiniLM-L6-cos-v1', device=device)\n", + "\n", + "retriever\n" + ] + }, + { + "cell_type": "markdown", + "id": "8aaad0a2", + "metadata": { + "id": "8aaad0a2" + }, + "source": [ + "# Generate Embeddings and Upsert" + ] + }, + { + "cell_type": "markdown", + "id": "Hgy7AagJtO_p", + "metadata": { + "id": "Hgy7AagJtO_p" + }, + "source": [ + "Next, we need to generate embeddings for the context passages. We will do this in batches to help us more quickly generate embeddings and upload them to the Pinecone index. When passing the documents to Pinecone, we need an id (a unique value), context embedding, and metadata for each document representing context passages in the dataset. The metadata is a dictionary containing data relevant to our embeddings, such as the article title, context passage, etc." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a17824ef", + "metadata": { + "id": "a17824ef", + "tags": [] + }, + "outputs": [], + "source": [ + "from tqdm.auto import tqdm\n", + "\n", + "# we will use batches of 64\n", + "batch_size = 64\n", + "\n", + "for i in tqdm(range(0, len(df), batch_size)):\n", + " # find end of batch\n", + " i_end = min(i + batch_size, len(df))\n", + "\n", + " # extract batch\n", + " batch = df.iloc[i:i_end]\n", + "\n", + " # generate embeddings for batch\n", + " emb = retriever.encode(batch[\"context\"].tolist()).tolist()\n", + "\n", + " # get metadata\n", + " meta = batch[[\"title\", \"context\"]].to_dict(orient=\"records\")\n", + "\n", + " # create unique IDs\n", + " ids = [str(x) for x in range(i, i_end)]\n", + "\n", + " # add all to upsert list\n", + " to_upsert = list(zip(ids, emb, meta))\n", + "\n", + " # upsert/insert these records to pinecone\n", + " _ = index.upsert(vectors=to_upsert)\n", + "\n", + "# check that we have all vectors in index\n", + "index.describe_index_stats()\n" + ] + }, + { + "cell_type": "markdown", + "id": "YFyBYafuJ0y0", + "metadata": { + "id": "YFyBYafuJ0y0" + }, + "source": [ + "# Initialize Reader" + ] + }, + { + "cell_type": "markdown", + "id": "HgdiLCz5ynOk", + "metadata": { + "id": "HgdiLCz5ynOk" + }, + "source": [ + "We use the `deepset/electra-base-squad2` model from the HuggingFace model hub as our reader model. We load this model into a \"question-answering\" pipeline from HuggingFace transformers and feed it our questions and context passages individually. The model gives a prediction for each context we pass through the pipeline." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "hg9XTDkIJzH_", + "metadata": { + "id": "hg9XTDkIJzH_" + }, + "outputs": [], + "source": [ + "from transformers import pipeline\n", + "\n", + "model_name = 'deepset/electra-base-squad2'\n", + "# load the reader model into a question-answering pipeline\n", + "\n", + "device_id = 0 if torch.cuda.is_available() else -1\n", + "reader = pipeline(tokenizer=model_name, model=model_name, task='question-answering', device=device_id)\n", + "\n", + "reader" + ] + }, + { + "cell_type": "markdown", + "id": "e14d89d6", + "metadata": { + "id": "e14d89d6" + }, + "source": [ + "Now all the components we need are ready. Let's write some helper functions to execute our queries. The `get_context` function retrieves the context embeddings containing answers to our question from the Pinecone index, and the `extract_answer` function extracts the answers from these context passages." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "lyYaY3QEQiHZ", + "metadata": { + "id": "lyYaY3QEQiHZ" + }, + "outputs": [], + "source": [ + "# gets context passages from the pinecone index\n", + "def get_context(question, top_k):\n", + " # generate embeddings for the question\n", + " xq = retriever.encode(question).tolist()\n", + "\n", + " # search pinecone index for context passages with the answer\n", + " xc = index.query(vector=xq, top_k=top_k, include_metadata=True)\n", + "\n", + " # extract the context passages from pinecone search result\n", + " c = [match[\"metadata\"][\"context\"] for match in xc[\"matches\"]]\n", + " return c\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "Dc9VYOiUQA7B", + "metadata": { + "id": "Dc9VYOiUQA7B" + }, + "outputs": [], + "source": [ + "from pprint import pprint\n", + "\n", + "# extracts answer from the context passage\n", + "def extract_answer(question, context):\n", + " results = []\n", + " for c in context:\n", + " # feed the reader the question and contexts to extract answers\n", + " answer = reader(question=question, context=c)\n", + " # add the context to answer dict for printing both together\n", + " answer[\"context\"] = c\n", + " results.append(answer)\n", + " # sort the result based on the score from reader model\n", + " sorted_result = pprint(sorted(results, key=lambda x: x['score'], reverse=True))\n", + " return sorted_result" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5E3a3dkJ5ZQD", + "metadata": { + "id": "5E3a3dkJ5ZQD" + }, + "outputs": [], + "source": [ + "question = \"How much oil is Egypt producing in a day?\"\n", + "context = get_context(question, top_k = 1)\n", + "context" + ] + }, + { + "cell_type": "markdown", + "id": "heKNVbWQ_LtC", + "metadata": { + "id": "heKNVbWQ_LtC" + }, + "source": [ + "As we can see, the retiever is working fine and gets us the context passage that contains the answer to our question. Now let's use the reader to extract the exact answer from the context passage." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "DQ4GWdbMSjPl", + "metadata": { + "id": "DQ4GWdbMSjPl" + }, + "outputs": [], + "source": [ + "extract_answer(question, context)" + ] + }, + { + "cell_type": "markdown", + "id": "fMD_ABuDAyhN", + "metadata": { + "id": "fMD_ABuDAyhN" + }, + "source": [ + "The reader model predicted with 99% accuracy the correct answer *691,000 bbl/d* as seen from the context passage. Let's run few more queries." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "_4NRgV4mGWoj", + "metadata": { + "id": "_4NRgV4mGWoj" + }, + "outputs": [], + "source": [ + "question = \"What are the first names of the men that invented youtube?\"\n", + "context = get_context(question, top_k=1)\n", + "extract_answer(question, context)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "juXlctWgJgMF", + "metadata": { + "id": "juXlctWgJgMF" + }, + "outputs": [], + "source": [ + "question = \"What is Albert Eistein famous for?\"\n", + "context = get_context(question, top_k=1)\n", + "extract_answer(question, context)" + ] + }, + { + "cell_type": "markdown", + "id": "OhCgeny_BVno", + "metadata": { + "id": "OhCgeny_BVno" + }, + "source": [ + "Let's run another question. This time for top 3 context passages from the retriever." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "iXACn71xmett", + "metadata": { + "id": "iXACn71xmett" + }, + "outputs": [], + "source": [ + "question = \"Who was the first person to step foot on the moon?\"\n", + "context = get_context(question, top_k=3)\n", + "extract_answer(question, context)" + ] + }, + { + "cell_type": "markdown", + "id": "27fecadc-6976-43b2-90e1-788a8633ecc7", + "metadata": { + "id": "27fecadc-6976-43b2-90e1-788a8633ecc7" + }, + "source": [ + "
\n", + "\n", + "### \ud83d\ude80 Challenge\n", + "\n", + "Try adding a few more questions of your own in new cells below (reusing `get_context` and `extract_answer`). Experiment with different `top_k` values and observe:\n", + "\n", + "- Does the retriever always return a context that actually contains the answer?\n", + "- How does the reader's confidence score change with harder or more ambiguous questions?\n", + "- What happens with questions that have no good answer in the dataset?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "898daa2f", + "metadata": {}, + "outputs": [], + "source": [ + "# Try a few extra questions with different top_k values\n", + "questions = [\n", + " (\"Who developed the theory of relativity?\", 2),\n", + " (\"Where is the Eiffel Tower located?\", 3),\n", + " (\"Who won the FIFA World Cup in 2050?\", 2),\n", + "]\n", + "\n", + "for question, top_k in questions:\n", + " print(f\"\\nQuestion: {question}\")\n", + " context = get_context(question, top_k=top_k)\n", + " extract_answer(question, context)\n" + ] + }, + { + "cell_type": "markdown", + "id": "d4ed6384", + "metadata": {}, + "source": [ + "
\n", + "\n", + "---\n", + "\n", + "### Clean Up\n", + "\n", + "Now that we're done experimenting, let's delete the Pinecone index we created. This frees up the resources it was using (and, when using a paid plan, avoids being billed for an index we no longer need)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c53493d1", + "metadata": {}, + "outputs": [], + "source": [ + "# To delete the Pinecone index, \n", + "# uncomment the line below + run this code cell\n", + "\n", + "# pc.delete_index(index_name)" + ] + } + ], + "metadata": { + "accelerator": "GPU", + "colab": { + "gpuType": "T4", + "machine_shape": "hm", + "provenance": [] + }, + "environment": { + "name": "tf2-gpu.2-3.m65", + "type": "gcloud", + "uri": "gcr.io/deeplearning-platform-release/tf2-gpu.2-3:m65" + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.8" + }, + "papermill": { + "default_parameters": {}, + "duration": 333.240754, + "end_time": "2021-04-15T21:12:11.363566", + "environment_variables": {}, + "exception": null, + "input_path": "/notebooks/question_answering/question_answering.ipynb", + "output_path": "/notebooks/tmp/question_answering/question_answering.ipynb", + "parameters": {}, + "start_time": "2021-04-15T21:06:38.122812", + "version": "2.3.3" + }, + "vscode": { + "interpreter": { + "hash": "5fe10bf018ef3e697f9035d60bf60847932a12bface18908407fd371fe880db9" + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} \ No newline at end of file