diff --git a/.ipynb_checkpoints/2.5_apis (1)-checkpoint.ipynb b/.ipynb_checkpoints/2.5_apis (1)-checkpoint.ipynb new file mode 100644 index 0000000..8518b58 --- /dev/null +++ b/.ipynb_checkpoints/2.5_apis (1)-checkpoint.ipynb @@ -0,0 +1,2016 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "35870f3e-1791-4fc3-ba77-6b01a78bca50", + "metadata": {}, + "source": [ + "# APIS " + ] + }, + { + "cell_type": "markdown", + "id": "34841864-ea2a-436e-b85a-5cc871472849", + "metadata": {}, + "source": [ + "![apis_gif](https://media.giphy.com/media/wepUQluC5smgEd4Qz4/giphy.gif)" + ] + }, + { + "cell_type": "markdown", + "id": "a6795d4c-fdfe-4f6d-ad6b-2479d9b0826d", + "metadata": { + "toc": true + }, + "source": [ + "

Table of Contents

\n", + "
" + ] + }, + { + "cell_type": "markdown", + "id": "78337d49-bd91-4990-b1d6-fd2d4c03e6df", + "metadata": {}, + "source": [ + "## Web Communication Fundamentals" + ] + }, + { + "cell_type": "markdown", + "id": "1f5600c7-4e6b-4b57-986e-740538741ca9", + "metadata": {}, + "source": [ + "![enjuto](https://www.publico.es/files/article_main/uploads//2014/12/13/548bb18f1d9ef.jpg)" + ] + }, + { + "cell_type": "markdown", + "id": "582db4b0-9f2d-4847-b52c-f11eb6ae8933", + "metadata": {}, + "source": [ + "### DNS, IP" + ] + }, + { + "cell_type": "markdown", + "id": "1e7a9565-1961-4631-a6a1-71735dae6f16", + "metadata": {}, + "source": [ + "[How do we connect to www.google.com?](https://www.youtube.com/watch?v=sUhEqT_HSBI&ab_channel=ProfeSang)\n", + "* **DNS (Domain Name System)**: it's essentially the phonebook of the internet. Humans access information online through domain names, like \"google.com\". Web browsers, however, interact through Internet Protocol (IP) addresses. In this example, the DNS maps the internet address www.google.com to the server's IP: 216.58.222.196\n", + " * **IP**: Server identification. A code that allows information to be sent and received by the correct parties\n", + " * **Domain Providers**: Sell and purchase Internet domains" + ] + }, + { + "cell_type": "markdown", + "id": "7d42f993-6f37-42c7-8f04-4c5526c0825e", + "metadata": {}, + "source": [ + "*When making API calls to specific domains, the DNS translates the human-readable domain name into an IP address.*" + ] + }, + { + "cell_type": "markdown", + "id": "216e4441-45eb-4a0a-b578-f4f056a724b1", + "metadata": {}, + "source": [ + "### HTTP" + ] + }, + { + "cell_type": "markdown", + "id": "6d120e37-ac79-44c9-8fc5-3fe07f4ad7f6", + "metadata": {}, + "source": [ + "**H**yper **T**ext **T**ransfer **P**rotocol\n", + "- HTTP is a communications protocol that provides a structure for requests between the client and the server on a network.\n", + "- For example, the web browser on the user's computer (the client) uses the HTTP protocol to request information from a website on a server.\n", + "\n", + "*We will use APIs that employ the **HTTP** protocol as transport.*" + ] + }, + { + "cell_type": "markdown", + "id": "1b9f3b1b-af11-419a-bd55-771a937b72dc", + "metadata": {}, + "source": [ + "### URL\n", + "Contains information about the resource being requested from the **server**." + ] + }, + { + "cell_type": "markdown", + "id": "c6a7522a-6679-4d2d-9587-6b583c41a55f", + "metadata": {}, + "source": [ + "![](https://github.com/data-bootcamp-v4/lessons/blob/main/img/http.png?raw=true)" + ] + }, + { + "cell_type": "markdown", + "id": "246020a7-6a89-4857-98c4-8f29c1d3c827", + "metadata": {}, + "source": [ + "Examples:\n", + "\n", + "- https://www.google.com/webhp?authuser=2\n", + "- https://www.towardsdatascience.com\n", + "- https://www.ironhack.com/\n", + " - Protocol: https (https == http is the same, but https is encrypted)\n", + " - Domain Name --> ironhack\n", + " - TLD (Top-Level Domain) --> .com" + ] + }, + { + "cell_type": "markdown", + "id": "dd726622-8824-4e1f-a166-bb0d7e94fd08", + "metadata": {}, + "source": [ + "*When calling an API, you specify the URL of the API endpoint, which tells the system where to send the request.*" + ] + }, + { + "cell_type": "markdown", + "id": "5fd2c3c5-1e3e-44a2-923b-c6dc1bb19f7c", + "metadata": {}, + "source": [ + "## HTTP Requests and Responses" + ] + }, + { + "cell_type": "markdown", + "id": "1b658ec7-5230-44ab-b10b-f8cc1c65edab", + "metadata": {}, + "source": [ + "![](https://github.com/data-bootcamp-v4/lessons/blob/main/img/request-response.png?raw=true)" + ] + }, + { + "cell_type": "markdown", + "id": "4941c601-e2f2-41bd-a0f3-c5e33fc8e282", + "metadata": {}, + "source": [ + "**Requests** and **responses** are fundamental components of the client-server communication model." + ] + }, + { + "cell_type": "markdown", + "id": "b75d6570-aae7-492a-bcd0-86da6d102545", + "metadata": {}, + "source": [ + "### Requests" + ] + }, + { + "cell_type": "markdown", + "id": "6d39d707-2230-4f72-af04-8c49abeb413e", + "metadata": {}, + "source": [ + "These are queries or calls **sent by the client** (such as a web browser or other software) **to the server** in order to receive information (a **response**). \n", + "\n", + "A request typically consists of:\n", + "- A method (such as GET, POST, PUT, DELETE) that defines the action to be performed\n", + " * GET: read the information of the resource, without modifying it in any way. Accessing the website from the browser **gets** information.\n", + "- The URL or endpoint specifying the resource\n", + "- Optional additional information such as:\n", + " - Headers (metadata): User-Agent, Accept-Language\n", + " - Parameters\n", + " - Body content. \n", + " \n", + "For example, a client might send a GET request to retrieve information from a web page or a POST request to submit form data." + ] + }, + { + "cell_type": "markdown", + "id": "03152bb9-ab13-458c-94d2-f5cb53117056", + "metadata": {}, + "source": [ + "### Response" + ] + }, + { + "cell_type": "markdown", + "id": "fc180f91-d6bf-4cb9-bf07-443d47ba0c4f", + "metadata": {}, + "source": [ + "These are the answers or data sent **by the server back to the client in reply to a request**. \n", + "\n", + "A response typically includes:\n", + "- A status code that indicates the success or failure of the request\n", + "- Headers with meta-information about the server's behavior\n", + "- The actual content or data (if applicable), such as HTML, JSON (similar to a Python dictionary), images, or other media types.\n", + "\n", + "An important part of the **header** is the **status code**. This code is a numerical value that indicates the server's result. There are different status codes depending on whether the server has managed to carry out the request or has not managed to do anything. These are some groups of status codes:\n", + "\n", + "- **2xx successful**: the request was successfully received, understood, and accepted\n", + "- **3xx redirection**: more actions are required to complete the request\n", + "- **4xx client error**: the request contains incorrect syntax or cannot be fulfilled\n", + "- **5xx server error**: the server has failed to complete an apparently valid request\n", + "\n", + "Complete list:\n", + "https://en.wikipedia.org/wiki/List_of_HTTP_status_codes\n", + "\n", + "Much more fun:\n", + "https://http.cat/" + ] + }, + { + "cell_type": "markdown", + "id": "1f1ad409-99e9-4701-b879-f81d13ece283", + "metadata": { + "tags": [] + }, + "source": [ + "## APIs" + ] + }, + { + "cell_type": "markdown", + "id": "492c462e-3d81-4330-828f-ff2b027989d8", + "metadata": {}, + "source": [ + "**A**pplication **P**rogramming **I**nterface\n", + "\n", + "- APIs define a set of rules and protocols that allow different software applications to communicate with each other.\n", + "- The client calls the server through an API, and the server responds.\n", + "- Within a project, the backend may want to share information with the frontend without access to the database." + ] + }, + { + "cell_type": "markdown", + "id": "0849403b-d633-4756-82d2-cb74ac4d254b", + "metadata": {}, + "source": [ + "![](https://github.com/data-bootcamp-v4/lessons/blob/main/img/api.png?raw=true)" + ] + }, + { + "cell_type": "markdown", + "id": "b86f18cb-8590-4a6e-896b-c5e465e7719f", + "metadata": {}, + "source": [ + "### RESTful APIs" + ] + }, + { + "cell_type": "markdown", + "id": "62891457-6b5c-4443-9fb5-77f901ea7571", + "metadata": {}, + "source": [ + "As a data analyst, you often need to access data from various sources. RESTful APIs provide a **standardized way** to retrieve, update, or delete data from other systems." + ] + }, + { + "cell_type": "markdown", + "id": "5c2bea7b-f629-45f3-94b3-6cb6eac1272e", + "metadata": {}, + "source": [ + "- Usually, **we (the client) send a request, and they (the server) return a response, often in JSON - JavaScript Object Notation - format**. While JSON looks similar to the dictionaries we're used to in Python, it's worth noting that when represented as raw text (e.g., in an API response), the entire JSON structure is a *string*. However, within that structure, JSON can represent various data types like numbers, booleans, and arrays, not just strings.\n", + "\n", + "- APIs always need to provide documentation for their various services: **endpoints**. Each endpoint is a different URL. \n", + "- Sometimes we have to pass **parameters** to an API endpoint, similar to when we pass parameters to a Python function.\n", + "\n", + "At this point in the Bootcamp, **reading documentation becomes essential**, as each API will \"work\" differently, and to use it, we will need to know what it requires.\n", + "\n", + "[Here](https://github.com/public-apis/public-apis) is a repository where you can find several free APIs if you feel like exploring a bit." + ] + }, + { + "cell_type": "markdown", + "id": "33c6c206-715a-42b8-929c-691d9c66fe4d", + "metadata": {}, + "source": [ + "## Requests in Python" + ] + }, + { + "cell_type": "markdown", + "id": "515484d8-81ec-48c1-abad-c778232b583f", + "metadata": {}, + "source": [ + "You can make requests to RESTful APIs using libraries like `requests`. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "54d23d14-7425-45c6-8ba5-74e5db2c047a", + "metadata": {}, + "outputs": [], + "source": [ + "# If you don't have it installed you can do use using pip or pip3\n", + "\n", + "#!pip install requests" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9ddac917-8539-46e8-abc9-37111e2fa74c", + "metadata": {}, + "outputs": [], + "source": [ + "import requests" + ] + }, + { + "cell_type": "markdown", + "id": "da5aff9d-8a79-4171-96b1-b7f0d2daf1cd", + "metadata": {}, + "source": [ + "### GET" + ] + }, + { + "cell_type": "markdown", + "id": "2018fa77-7bce-437e-a806-69844737210b", + "metadata": {}, + "source": [ + "Making a `GET request` to the API is simply a call to a `URL` that returns information when provided with the appropriate `parameters`. We will only perform **GET** requests." + ] + }, + { + "cell_type": "markdown", + "id": "718172e2-5654-4536-bd72-594d80b31f8c", + "metadata": {}, + "source": [ + "Here is an example on how to make a request with Python:\n", + "\n", + "```python\n", + "url = \"https://api.example.com/products\"\n", + "response = requests.get(url)\n", + "\n", + "if response.status_code == 200:\n", + " products = response.json()\n", + " # Now you can analyze and work with the products data in Python\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "307395ef-cfd3-4175-99d5-a6ce6213adb1", + "metadata": {}, + "source": [ + "#### Accessing an endpoint\n", + "\n", + "As we mentioned above, APIs always need to provide documentation for their various services: **endpoints**. Each endpoint is a different URL. \n", + "\n", + "**Example: ISS API**" + ] + }, + { + "cell_type": "markdown", + "id": "792140dd-3cda-4355-86bd-dd6732880803", + "metadata": {}, + "source": [ + "Let's get information from ISS (International Space Station)! We'll start looking at the [ISS API documentation](https://wheretheiss.at/w/developer)" + ] + }, + { + "cell_type": "markdown", + "id": "a2be4164-137b-411d-a78c-38bdd94ce9ed", + "metadata": {}, + "source": [ + "This API allows you to access various data related to the International Space Station (ISS), including its current, past, or future position, timezone information for specific coordinates, and more.\n", + "\n", + "**Key Features**\n", + "- **Authentication**: No authentication is currently required, but future endpoints may include this.\n", + "- **Rate Limiting**: Limited to approximately 1 request per second.\n", + "- **Responses**: Default to JSON format, with optional parameters to modify response appearance.\n", + "- **Endpoints**: Several endpoints provide different types of information:\n", + " - **satellites**: Information about satellites, including the ISS.\n", + " - **satellites/[id]**: Position, velocity, and related information for a satellite.\n", + " - **satellites/[id]/positions**: Position data for specific timestamps.\n", + " - **satellites/[id]/tles**: TLE (Two-Line Element Set) data in either JSON or text format.\n", + " - **coordinates/[lat,lon]**: Timezone information for specific coordinates.\n" + ] + }, + { + "cell_type": "markdown", + "id": "f1272582-bc6d-4f43-9e39-47d39ffd1bfd", + "metadata": {}, + "source": [ + "**Examples**\n", + "- Satellite details: `https://api.wheretheiss.at/v1/satellites`\n", + "- ISS position: `https://api.wheretheiss.at/v1/satellites/25544`\n", + "- Coordinates information: `https://api.wheretheiss.at/v1/coordinates/37.795517,-122.393693`\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "5ca5b7da-eec0-41ca-924b-67687f794e23", + "metadata": {}, + "source": [ + "**Endpoint satellites**" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "05fad64c-5f20-4b7f-a865-9659b9cc8783", + "metadata": {}, + "outputs": [], + "source": [ + "# We'll use the endpoint satellites, we read it gives us information about satellites\n", + "\n", + "url = \"https://api.wheretheiss.at/v1/satellites\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "68f33b09-b943-486c-98e0-2d18b25640c5", + "metadata": {}, + "outputs": [], + "source": [ + "response = requests.get(url) # We use get method to make the request and get information from the API" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "67938f79-0981-471e-9201-89352e99775b", + "metadata": {}, + "outputs": [], + "source": [ + "type(response) # Lets look at the type of the response" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "310b2c20-4a51-47de-9983-1e266496c951", + "metadata": {}, + "outputs": [], + "source": [ + "response.status_code # We can access status code. 200 is OK" + ] + }, + { + "cell_type": "markdown", + "id": "488cbcbe-2d4d-42c2-b92a-ec76b70b705b", + "metadata": {}, + "source": [ + "We can access the response content, which returns a string" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5196a512-6690-49c9-971a-bc9cb2383932", + "metadata": {}, + "outputs": [], + "source": [ + "response.content" + ] + }, + { + "cell_type": "markdown", + "id": "820ad33e-4664-46b6-a51a-0b277ba04c4d", + "metadata": {}, + "source": [ + "If response Content-Type is json, we can access it better with `.json()`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7f44e27c-7866-4e9c-8965-a20ccaef0e43", + "metadata": {}, + "outputs": [], + "source": [ + "info = response.json()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cd406dbc-74f9-490b-b8c6-73ef02e9b9d4", + "metadata": {}, + "outputs": [], + "source": [ + "info" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "992d8cf6-8e5c-448d-bb1c-e9416aefe428", + "metadata": {}, + "outputs": [], + "source": [ + "type(info) # It returns a list of dictionaries" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1a7d7dc2-f2d7-44a7-9bda-e5cc0490dffe", + "metadata": {}, + "outputs": [], + "source": [ + "info[0][\"id\"] # So we can access the id like this" + ] + }, + { + "cell_type": "markdown", + "id": "903f3024-0998-459d-bb20-60a639def27e", + "metadata": {}, + "source": [ + "#### Response headers \n", + "\n", + "Response headers are part of the HTTP response that a server sends back to the client after a request has been made. These headers provide meta-information about the response and can affect how the client handles the response.\n", + "\n", + "Here are some common response headers and what they typically represent:\n", + "\n", + "1. **`Date`**: Represents the date and time at which the response was sent.\n", + "\n", + "2. **`Server`**: Provides information about the software used by the originating server.\n", + "\n", + "3. **`X-Rate-Limit`**: Sometimes used in APIs to inform the client about rate limiting policies, such as the number of allowed requests in a given time frame.\n", + "\n", + "4. **`Content-Type`**: Specifies the media type of the resource or data the server is sending back. For example, it could be `application/json` for a JSON object, `text/html` for an HTML page, or `image/png` for an image.\n", + "\n", + "And more." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a53d51bc-2346-4139-a23f-49a34dc3b475", + "metadata": {}, + "outputs": [], + "source": [ + "response.headers # We can also access the response headers" + ] + }, + { + "cell_type": "markdown", + "id": "a46967a4-763f-4d4e-bca9-c3a37742f65e", + "metadata": {}, + "source": [ + "#### Parameters" + ] + }, + { + "cell_type": "markdown", + "id": "69017232-6063-4ee5-a13c-0eb40f182b1e", + "metadata": {}, + "source": [ + "As we mentioned above, sometimes we can pass **parameters** to an API endpoint, similar to when we pass parameters to a Python function." + ] + }, + { + "cell_type": "markdown", + "id": "4dbf1646-5953-44b9-a2e3-a7dbee7f1a4e", + "metadata": {}, + "source": [ + "In the example above, we didn't use any parameters in the endpoint `satellites` since the [documentation](\"https://wheretheiss.at/w/developer\") said *Parameters: None*." + ] + }, + { + "cell_type": "markdown", + "id": "ed0d4cf7-fa4d-4c83-8d56-4a25accc3d01", + "metadata": {}, + "source": [ + "API parameters are specific values that you include in a request to an API endpoint to filter, sort, or detail the data that you want to retrieve. They allow you to customize the request to get exactly the information you need." + ] + }, + { + "cell_type": "markdown", + "id": "dbf6baed-1967-46ea-908f-bff2fd59dc46", + "metadata": {}, + "source": [ + "There are several types of parameters that can be used in API requests, such as Path Parameters, Query Parameters, Header Parameters and Request Body Parameters. Let's look at them with an example using another endpoint." + ] + }, + { + "cell_type": "markdown", + "id": "f854df58-1a0a-4759-9991-30816cf45c92", + "metadata": {}, + "source": [ + "##### 1. **Path Parameters**\n", + "\n", + "These are embedded in the URL path and are used to identify a specific resource. For example, in the URL above `https://api.wheretheiss.at/v1/satellites/25544`, the number `25544` is a path parameter that identifies a specific sattelite." + ] + }, + { + "cell_type": "markdown", + "id": "df9a0f8c-b363-4ceb-bcc6-04128ec2bffc", + "metadata": { + "tags": [] + }, + "source": [ + "**Endpoint satellites/id**" + ] + }, + { + "cell_type": "markdown", + "id": "f7f1892c-1908-438d-a072-a77fc2214e52", + "metadata": {}, + "source": [ + "Lets use endpoint satellites/[id] with the id we got from the previous endpoint.\n", + "Important, we need to provide the whole url for the endpoint as a `string`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "60e19b4e-1476-456c-8272-a3175708368e", + "metadata": {}, + "outputs": [], + "source": [ + "url_iss_position = \"https://api.wheretheiss.at/v1/satellites/\"+str(info[0][\"id\"])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "42cb7357-ff48-4f7d-81df-8060c7ed794f", + "metadata": {}, + "outputs": [], + "source": [ + "response = requests.get(url_iss_position)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "340a190c-6010-4134-bfd5-f4acc5484e8b", + "metadata": {}, + "outputs": [], + "source": [ + "response.status_code" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a9ddae28-476f-4133-8a25-4e35c3fac891", + "metadata": {}, + "outputs": [], + "source": [ + "response.json()" + ] + }, + { + "cell_type": "markdown", + "id": "7c21289e-ed73-4fdf-8ff2-8ec53d1bedbb", + "metadata": {}, + "source": [ + "Every time we call the endpoint, the information changes since it gives us the current position of the satellite. Let's check that with a loop to gather information at different times." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c02e4902-90b6-4f31-8326-2dcee2478aa8", + "metadata": {}, + "outputs": [], + "source": [ + "import time\n", + "\n", + "positions = []\n", + "\n", + "for i in range(10):\n", + " response = requests.get(url_iss_position)\n", + " data = response.json()\n", + " \n", + " positions.append(data)\n", + " time.sleep(0.5)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3801506d-82c7-4c1b-b4ab-81fce6a73b36", + "metadata": {}, + "outputs": [], + "source": [ + "type(positions) # We created a list with responses" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5f3da8cc-7ad5-4313-8bbd-71e733359747", + "metadata": {}, + "outputs": [], + "source": [ + "type(positions[0]) # Is a list of dictionaries" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8d24d4c8-860c-4a6f-a5ec-db44ed9d4c65", + "metadata": {}, + "outputs": [], + "source": [ + "positions[0] # So we can access each dictionary by its position in the list" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b588249b-ef73-4cd0-8c3f-4a7a46c9156f", + "metadata": {}, + "outputs": [], + "source": [ + "positions[9]" + ] + }, + { + "cell_type": "markdown", + "id": "57c55b8a-1b23-4e9e-8f7b-d24b7a1b375e", + "metadata": {}, + "source": [ + "Let's extract 10 latitudes from the 10 dictionaries in the list, with 2 decimals." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3371418e-6637-4857-913f-01fd244bd687", + "metadata": {}, + "outputs": [], + "source": [ + "latitudes = [round(p[\"latitude\"], 2) for p in positions]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "256a7492-b9de-416b-9cad-3b4155f767e4", + "metadata": {}, + "outputs": [], + "source": [ + "latitudes" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7b841d34-6a6f-4f31-9764-1d5ac4d2f35f", + "metadata": {}, + "outputs": [], + "source": [ + "import pandas as pd\n", + "pd.DataFrame(positions) # We can create a dataframe from the API response to work with it better" + ] + }, + { + "cell_type": "markdown", + "id": "204abde8-009b-4217-8295-51fbde187742", + "metadata": { + "tags": [] + }, + "source": [ + "\n", + "##### 2. **Query Parameters**\n", + "\n", + "These are added to the end of the URL after a question mark (`?`) and are often used to filter or sort the response. \n", + "\n", + "For example, in the ISS documentation, it mentions that there is a parameter `units` that can take values `miles` or `kilometers`. So in the URL `https://api.wheretheiss.at/v1/satellites/25544?units=miles`, we added `units=miles` which is a query parameter that shows the data in \"miles\". \n", + "\n", + "In general, we add the parameters like this `?param1=value1¶m2=value2...` at the end of the URL." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b252f4b2-f739-4d78-8a14-7933b946c309", + "metadata": {}, + "outputs": [], + "source": [ + "url_iss_position" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b8bfc174-f81f-41cc-92f4-eec321e1961a", + "metadata": {}, + "outputs": [], + "source": [ + "# We saw in the documentation that we can add a parameter units to use miles or kilometers\n", + "\n", + "url_iss_position2 = 'https://api.wheretheiss.at/v1/satellites/25544?units=miles'" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "47ead56f-1882-41f4-8850-67a6611fe36d", + "metadata": {}, + "outputs": [], + "source": [ + "response = requests.get(url_iss_position2)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5a032e2c-abab-4c49-97d1-341d74a22c68", + "metadata": {}, + "outputs": [], + "source": [ + "response.status_code" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "395d4fed-6f83-4c96-8cf5-8523ce297e92", + "metadata": {}, + "outputs": [], + "source": [ + "response.json()" + ] + }, + { + "cell_type": "markdown", + "id": "9c50844d-2d2c-4768-8d8b-703a2cb1875c", + "metadata": {}, + "source": [ + "We can also do it by passing to the argument `params` a dictionary with the parameters in the `get` method." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cdcdd6f7-628b-4978-8656-23f72ecd5cac", + "metadata": {}, + "outputs": [], + "source": [ + "parameters = {\"units\": \"miles\"}" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "31d68523-2258-408f-ac24-0836af707cd9", + "metadata": {}, + "outputs": [], + "source": [ + "url_iss_position" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2688ca3e-e9ef-44cf-8e28-7c575f2d61be", + "metadata": {}, + "outputs": [], + "source": [ + "response = requests.get(\n", + " url=url_iss_position,\n", + " params=parameters\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "87e3afb1-2e29-4c62-8162-e8b27756be0c", + "metadata": {}, + "outputs": [], + "source": [ + "response.json()" + ] + }, + { + "cell_type": "markdown", + "id": "6aea75ed-be20-475d-98c1-86f4e9d2b1fd", + "metadata": {}, + "source": [ + "\n", + "##### 3. **Header Parameters**\n", + "\n", + "These are included in the request header and can be used for various purposes, such as authentication (e.g., sending an API key or token), content negotiation (e.g., defining the response format), or custom settings defined by the API. If the API requires authentication, you might include an `Authorization` header with your API key." + ] + }, + { + "cell_type": "markdown", + "id": "78131fa0-a203-4d64-8fc2-939ee0ebf064", + "metadata": {}, + "source": [ + "ISS API mentions that *currently there is no authentication required*. So we'll look at an example with a different API of `authentication` in `header parameters`." + ] + }, + { + "cell_type": "markdown", + "id": "69484a90-65de-44b0-87bb-b591274f27b5", + "metadata": {}, + "source": [ + "#### Request Headers" + ] + }, + { + "cell_type": "markdown", + "id": "a5782c9a-84ad-434e-b4a5-5657f17592f1", + "metadata": {}, + "source": [ + "Request headers are key-value pairs sent in an HTTP request to provide information about the request itself." + ] + }, + { + "cell_type": "markdown", + "id": "70c5dec7-204a-4c3b-a9e4-8c7bbf516b10", + "metadata": {}, + "source": [ + "Some request headers are:\n", + "\n", + "1. **Content-Type**: Specifies the media type of the resource or data. Common examples include \"application/json\" for JSON data, \"text/html\" for HTML content, and \"application/xml\" for XML data.\n", + "\n", + "3. **Authorization**: Contains credentials for authenticating the client with the server, often used with tokens or other forms of authentication.\n", + "\n", + "4. **User-Agent**: Provides information about the client (browser or other client), including its version and operating system.\n", + "\n", + "\n", + "In Python's `requests` library, you can include headers in a request by using the `headers` argument, like this:\n", + "\n", + "```python\n", + "headers = {'Content-Type': 'application/json', 'Authorization': 'Bearer YOUR_TOKEN'}\n", + "response = requests.get(url, headers=headers)\n", + "```\n" + ] + }, + { + "cell_type": "markdown", + "id": "a5311048-ef5e-4b04-9e4b-8ed700ab3076", + "metadata": {}, + "source": [ + "⚠️🚨 ¡Careful with the authentication token! ⚠️🚨 A token is a personal credential for accessing an API, through which your request quota to the API is managed. Therefore, the ideal procedure is to ensure security by storing the token as a variable in an .env file. This way, you can call the environment variable without having to publicly display the token." + ] + }, + { + "cell_type": "markdown", + "id": "b3cf22bc-d4e1-4e51-876e-f4df41b70f05", + "metadata": {}, + "source": [ + "**Example: News API**" + ] + }, + { + "cell_type": "markdown", + "id": "a6b35b6f-4cb4-4b89-af23-3c9c65fb2686", + "metadata": {}, + "source": [ + "The *News API* lets you locate articles and breaking news headlines from news sources and blogs across the web." + ] + }, + { + "cell_type": "markdown", + "id": "58123f1a-b5e4-4b20-a02d-47077a4a3635", + "metadata": {}, + "source": [ + "- Let's look at the [NewsAPI Authentication documentation](https://newsapi.org/docs/authentication). It mentions: \n", + "```\n", + "You can attach your API key to a request in one of three ways:\n", + "\n", + "- Via the apiKey querystring parameter.\n", + "- Via the X-Api-Key HTTP header.\n", + "- Via the Authorization HTTP header. Including Bearer is optional, and be sure not to base 64 encode it like you may have seen in other authentication tutorials.\n", + "We strongly recommend the either of last two so that your API key isn't visible to others in logs or via request sniffing.\n", + "\n", + "If you don't append your API key correctly, or your API key is invalid, you will receive a 401 - Unauthorized HTTP error.\n", + "```\n", + "\n", + "- Let's look at the documentation for the endpoints to see what **parameters** they accept: [NewsAPI endpoints](https://newsapi.org/docs/endpoints). We see we have the endpoint `/v2/top-headlines` – *returns breaking news headlines for countries, categories, and singular publishers. This is perfect for use with news tickers or anywhere you want to use live up-to-date news headlines.*\n", + "If we look at the [documentation of that endpoint](https://newsapi.org/docs/endpoints/top-headlines) we see we have more parameters we can add such as `country` or `category`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e2853b7d-09f0-46e3-83df-c1a1b03f6b61", + "metadata": {}, + "outputs": [], + "source": [ + "url = \"http://newsapi.org/v2/top-headlines\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ab2d561c-97fc-4750-9334-f19ce984b2ae", + "metadata": {}, + "outputs": [], + "source": [ + "response = requests.get(url)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1d44b74a-8282-4f74-bcbb-e030a2d0a26a", + "metadata": {}, + "outputs": [], + "source": [ + "response.status_code" + ] + }, + { + "cell_type": "markdown", + "id": "6c01205a-e732-44bc-9fc0-cb3cefdfc76f", + "metadata": {}, + "source": [ + "Let's try a made up key, using the apiKey querystring parameter. As we read in the documentation, this is not the recommended way." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "24d14d50-c5dc-4c20-ab4b-4f65d5e61a61", + "metadata": {}, + "outputs": [], + "source": [ + "url = f\"http://newsapi.org/v2/top-headlines?country=us&apiKey=manolonodoesnthavekey\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2d106abb-06ac-4432-b4ad-2c51fbd90e28", + "metadata": {}, + "outputs": [], + "source": [ + "url" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "304ff5dd-8d38-4641-9799-8cd7978434d0", + "metadata": {}, + "outputs": [], + "source": [ + "response = requests.get(url)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "bb9dc34b-21de-4d00-8544-fc9ea2c23f70", + "metadata": {}, + "outputs": [], + "source": [ + "response.status_code" + ] + }, + { + "cell_type": "markdown", + "id": "e829611f-a4a8-4b3d-90a2-31c1bb04fb2b", + "metadata": {}, + "source": [ + "**4xx**: client error. This means, it is our error." + ] + }, + { + "cell_type": "markdown", + "id": "71322103-4234-4105-be5d-037cb6107e26", + "metadata": {}, + "source": [ + "Lets do with a correct key. You should get your own API key through their website. Once we've saved our key, we'll send it via the X-Api-Key HTTP header." + ] + }, + { + "cell_type": "markdown", + "id": "a0ae416d-0215-4acd-9479-2ba49dab313b", + "metadata": {}, + "source": [ + "##### Saving the API Key" + ] + }, + { + "cell_type": "markdown", + "id": "2c509a90-72f3-4bba-9b29-bc4316bd3111", + "metadata": {}, + "source": [ + "Storing an API key directly in your code can expose sensitive information, especially if your code is publicly available (e.g., on a public GitHub repository). The best practice for saving and loading an API key in your code involves the following steps:\n", + "\n", + "1. **Storing the API Key**:\n", + " - **Use Environment Variables**: Store your API key in an environment variable on your system. This keeps the key out of your codebase and allows you to change it without altering your code.\n", + "\n", + "\n", + " - **Create a .env File**: If you prefer, you can create a `.env` file (just call it `.env` nothing else before the `.`) in your project directory to store the API key. Inside this file, you would have something like:\n", + " \n", + " ```\n", + " API_KEY=your-api-key-here\n", + " ```\n", + "\n", + " - **Add .env to .gitignore**: If you're using a version control system like Git, make sure to add the `.env` file to your `.gitignore` file. This prevents the `.env` file (and therefore your API key) from being uploaded to any public repositories. We'll talk about Git and .gitignore in more detail soon.\n", + "\n", + "2. **Load the Key in Your Code**: \n", + "\n", + " You can use libraries like `python-dotenv` to load the key into your code. \n", + " You would need to install `python-dotenv` first.\n", + " ```python \n", + " !pip install python-dotenv\n", + "\n", + " from dotenv import load_dotenv\n", + " import os\n", + "\n", + " load_dotenv()\n", + " api_key = os.getenv(\"API_KEY\")\n", + " ```\n", + "\n", + " \n", + " Now, `api_key` contains the value of your API key, and you can use it to authenticate your requests to the API.\n", + " \n" + ] + }, + { + "cell_type": "markdown", + "id": "98b7f771-e230-4223-a5a7-4353ee337934", + "metadata": {}, + "source": [ + "Lets do the second approach. Lets save it in an env file and save it in a variable." + ] + }, + { + "cell_type": "markdown", + "id": "3766f8a4-3e0a-4b18-8b16-d5e0d64c4dbd", + "metadata": {}, + "source": [ + "`Make sure your file is named .env and not .env.txt! Is one of the most common errors. If you need, look at the document properties to make sure it doesn't have .txt at the end even if its not showing the .txt when looking at your folder.`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "12fa7d1f-37c7-40a7-b60f-e4e264a43574", + "metadata": {}, + "outputs": [], + "source": [ + "#!pip install python-dotenv" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "65104c73-28cb-4b84-949b-013d2f0fd1b0", + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "from dotenv import load_dotenv, find_dotenv\n", + "\n", + "load_dotenv()\n", + "\n", + "my_key = os.getenv(\"API_KEY\")" + ] + }, + { + "cell_type": "markdown", + "id": "c2b8daee-8bbe-4dba-b302-4424bff542ee", + "metadata": {}, + "source": [ + "If you get an error, make sure you saved the .env file in the same directory as your jupyter notebook. You can use `os.getcwd()` to do so." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2edcbffe-e4b5-464b-8e5b-9efd3a41ed59", + "metadata": {}, + "outputs": [], + "source": [ + "os.getcwd()" + ] + }, + { + "cell_type": "markdown", + "id": "f1e87da8-be8c-4f58-94aa-354b879a7ed6", + "metadata": {}, + "source": [ + "##### Using the API Key in the Header Parameters" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c2c03dbc-e8b1-4649-9c71-18e61cc61123", + "metadata": {}, + "outputs": [], + "source": [ + "url_top_headlines = \"http://newsapi.org/v2/top-headlines\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "262b26fd-e3a8-4bc3-a7cb-38901f474817", + "metadata": {}, + "outputs": [], + "source": [ + "parameters = {\"country\": \"us\", \"category\": \"science\"}" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d3f8f790-1a58-4655-99bb-729fccd803eb", + "metadata": {}, + "outputs": [], + "source": [ + "headers = {\"X-Api-Key\": my_key}" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "85034bf8-bcc1-4e4e-bcb1-a442b2d65711", + "metadata": {}, + "outputs": [], + "source": [ + "response = requests.get(\n", + " url=url_top_headlines, \n", + " params=parameters, \n", + " headers=headers\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ad8de9aa-736e-480a-8a4d-1de84dbade07", + "metadata": {}, + "outputs": [], + "source": [ + "response.status_code" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "56058048-233a-41f5-a1da-edb8092730de", + "metadata": {}, + "outputs": [], + "source": [ + "data = response.json()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "eb07a572-50ca-41b0-b3b3-ec82d0cd3f5f", + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "data" + ] + }, + { + "cell_type": "markdown", + "id": "b6e5c398-c333-4831-b15c-230c63e7ac9c", + "metadata": {}, + "source": [ + "## APIs examples" + ] + }, + { + "cell_type": "markdown", + "id": "99d3ab93-1f0f-4c3d-9a4d-ebdf7337641b", + "metadata": {}, + "source": [ + "### NewsAPI" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f9720542-bc2e-4bd5-af7d-395f6aa420a0", + "metadata": {}, + "outputs": [], + "source": [ + "type(data) # response.json() returns a dictionary" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "541f1c98-0837-4e79-b3bc-8266486acf2f", + "metadata": {}, + "outputs": [], + "source": [ + "data.keys() # we can get the keys with method keys()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b536335a-008b-41d9-af92-6ff4466da5e9", + "metadata": {}, + "outputs": [], + "source": [ + "data.get(\"status\") # we can get a key's value by using get method" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "eee49271-cd78-42f5-972d-dfe9728ea730", + "metadata": {}, + "outputs": [], + "source": [ + "data[\"totalResults\"] # or by using dictionary notation to access values" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f26b202a-2d95-4e36-87e4-9a52f32a4be9", + "metadata": {}, + "outputs": [], + "source": [ + "# this is the same as above\n", + "data.get(\"totalResults\")" + ] + }, + { + "cell_type": "markdown", + "id": "61e9315e-c94e-41f8-8794-f255eb5220cc", + "metadata": {}, + "source": [ + "`dict[key]` raises error if the key does not exits \n", + "`dict.get(key)` does not raise an error if the key does not exist" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cc234036-e6d5-450e-9183-76e960220a7f", + "metadata": {}, + "outputs": [], + "source": [ + "# lets look at the articles\n", + "articles = data.get(\"articles\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0fac4939-7f00-4fe8-b758-4dc13e2c6b52", + "metadata": {}, + "outputs": [], + "source": [ + "type(articles) # lets see what type it is to know how to access it" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5988fdf2-5d2a-4236-927f-e8abd9c0fe93", + "metadata": {}, + "outputs": [], + "source": [ + "len(articles)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2353c5e3-c200-40b5-9529-a4f531dc4d77", + "metadata": {}, + "outputs": [], + "source": [ + "articles[0] # since articles is a list, we can access its elements by the index" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a98e38bb-4364-4d3a-817d-6993049f9408", + "metadata": {}, + "outputs": [], + "source": [ + "type(articles[0]) # each element inside the list is a dictionary" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "18eb15ba-18e3-49c5-997d-050815814e21", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "[art[\"title\"] for art in articles] # lets get all the titles using list comprehension" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e8422609-6ccf-4ddc-b044-b5a951f49ab6", + "metadata": {}, + "outputs": [], + "source": [ + "df = pd.DataFrame(articles) # lets create a dataframe with the articles" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9069dc15-b0e6-4db9-acba-f1b5c0c37a5b", + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "df.head()" + ] + }, + { + "cell_type": "markdown", + "id": "2dc2bab1-1cdc-4a3f-9956-b5d5eb832103", + "metadata": {}, + "source": [ + "#### 💡 Check for understanding" + ] + }, + { + "cell_type": "markdown", + "id": "3e21fbb4-6e2d-43ef-a61e-38da7eedc5fe", + "metadata": {}, + "source": [ + "1. What data type is the column *source*?\n", + "2. Since the column *source* is not very useful as it is, create a column called *name* that contains only the name inside the column *source*.\n", + "3. How many times each unique name appears in the \"name\" column?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f906ed21-be98-436b-816b-c73f5dd5c066", + "metadata": {}, + "outputs": [], + "source": [ + "# Your answer goes here" + ] + }, + { + "cell_type": "markdown", + "id": "48ca730a-10fe-4a69-ad11-01bf8759a938", + "metadata": {}, + "source": [ + "### Pokemon API" + ] + }, + { + "cell_type": "markdown", + "id": "be8f6d65-359d-408b-aeb4-64379614377f", + "metadata": {}, + "source": [ + "[PokeAPI](https://pokeapi.co/) is a full RESTful API linked to an extensive database detailing everything about the Pokémon main game series.\n", + "\n", + "In the documentation we see that we can get lot's of data. Let's look at the endpoint *pokemon*: ```https://pokeapi.co/api/v2/pokemon/{id or name}/``` " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b59992ec-a9de-4c14-88ef-177df49393a4", + "metadata": {}, + "outputs": [], + "source": [ + "res=requests.get('https://pokeapi.co/api/v2/pokemon/25')\n", + "res.status_code" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b180faa9-b8b2-4ec5-804e-d46941f6132c", + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "data=res.json()\n", + "data" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "faa82a7d-35f8-441c-9c65-c4cf34c8ba99", + "metadata": {}, + "outputs": [], + "source": [ + "type(data)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a4745456-3fa4-4511-a7be-c6eea20db9df", + "metadata": {}, + "outputs": [], + "source": [ + "data.keys()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9fa671b3-9a78-49e8-bf57-0781a1fa6b70", + "metadata": {}, + "outputs": [], + "source": [ + "# Lets look at the following attributes\n", + "print(data[\"name\"])\n", + "print(data[\"weight\"])\n", + "print(data[\"sprites\"])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b4ceecb0-9e17-4cfe-a09e-67ae3026652d", + "metadata": {}, + "outputs": [], + "source": [ + "# Lets get the pokemon image from sprites - front_default.\n", + "# Remember we can read the documentation to understand how to get all the information\n", + "data['sprites']['front_default']" + ] + }, + { + "cell_type": "markdown", + "id": "5bdb9958-a049-42b2-8c5c-475c7b90601d", + "metadata": {}, + "source": [ + "We will define a function, `get_pokemon`, that takes an ID number as input. This function fetches details about a Pokémon from the PokeAPI using the given ID, extracts its name, weight, and front-facing image, and then returns this data in a dictionary format. If the ID is invalid or there's an issue with the request, a ValueError is raised." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "04981f9a-157d-49ae-9e5d-0b382d5ab2fd", + "metadata": {}, + "outputs": [], + "source": [ + "def get_pokemon(id_number):\n", + " res=requests.get('https://pokeapi.co/api/v2/pokemon/{}'.format(id_number))\n", + " \n", + " if(res.status_code==200):\n", + " data = res.json()\n", + " return {\"id\": id_number,'name':data['name'], \"weight\":data[\"weight\"],'image':data['sprites']['front_default']}\n", + " else:\n", + " raise ValueError(\"Cannot get pokemon, id does not exist\")\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "96dbff41-1e63-40ee-b409-aed7cce7e7a7", + "metadata": {}, + "outputs": [], + "source": [ + "get_pokemon(25) # Lets use the function for pokemon id 25" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ebf5de4c-1150-4cab-812a-7b6d9f1e807c", + "metadata": {}, + "outputs": [], + "source": [ + "[get_pokemon(i+1) for i in range(5)] # Lets use list comprehension to gather information for the first 5 pokemons" + ] + }, + { + "cell_type": "markdown", + "id": "5d419697-cb8a-4737-8b0b-3a742615f16a", + "metadata": {}, + "source": [ + "We will now define a function, `print_pokemon`, that takes in a Pokémon dictionary and displays its ID, name, weight, and its image. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "da875adb-e2d1-4e59-8b2e-ce3e00da5f59", + "metadata": {}, + "outputs": [], + "source": [ + "from IPython.display import Image\n", + "\n", + "def print_pokemon(poke):\n", + " print(str(poke[\"id\"])+\" \" +poke[\"name\"]+ \" \"+str(poke[\"weight\"]) + \" kg\")\n", + " display(Image(url=poke['image']))\n", + " " + ] + }, + { + "cell_type": "markdown", + "id": "d284e0ef-4d49-42a3-bc50-c1a859a7e41a", + "metadata": {}, + "source": [ + "After defining this function, we will retrieve and display details for the first 10 Pokémon by iterating through their IDs and using both the `get_pokemon` and `print_pokemon` functions. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9e8a778b-bf09-4351-b9d6-6adc759566e5", + "metadata": {}, + "outputs": [], + "source": [ + "[print_pokemon(get_pokemon(i+1)) for i in range(10)]" + ] + }, + { + "cell_type": "markdown", + "id": "8ee586a7-a55f-4c14-b3cf-55745308fe84", + "metadata": {}, + "source": [ + "#### json_normalize()" + ] + }, + { + "cell_type": "markdown", + "id": "8ca157cb-530d-49de-9286-f072b3e295e4", + "metadata": {}, + "source": [ + "Want the data as a DataFrame instead of a dictionary?" + ] + }, + { + "cell_type": "markdown", + "id": "9672696a-3e67-4af5-af41-10270e1209b1", + "metadata": {}, + "source": [ + "The following line creates a DataFrame from the items of the dictionary data. Each key-value pair will be treated as a row, with two columns: the first for the key and the second for the value." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5455a1bd-a158-4111-9841-750add5b2708", + "metadata": {}, + "outputs": [], + "source": [ + "df = pd.DataFrame(data.items())\n", + "df" + ] + }, + { + "cell_type": "markdown", + "id": "175094a5-2fcd-4956-8fb4-e0d59f701ffe", + "metadata": {}, + "source": [ + "That format is still very hard to work with. Lets use `json_normalize` instead. `json_normalize` is used to normalize semi-structured JSON data into a flat table (DataFrame). It's particularly useful when dealing with nested dictionaries or lists inside JSON." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "def72654-66aa-41cd-900b-d55fae54894d", + "metadata": {}, + "outputs": [], + "source": [ + "df = pd.json_normalize(data)\n", + "df" + ] + }, + { + "cell_type": "markdown", + "id": "a879ab4b-1a2e-42f7-8a4b-4bf2c4e8687c", + "metadata": {}, + "source": [ + "A lot better, even though we still have nested dictionaries or lists inside the dataframe.\n", + "\n", + "Let's see again the difference between creating a dataframe directly or using `json_normalize` by using `data[\"abilities\"]`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "96389cbb-ee4f-4495-ae45-91de38ccc1fa", + "metadata": {}, + "outputs": [], + "source": [ + "pd.DataFrame(data[\"abilities\"])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "04997d49-0438-40e5-b0cf-6244405aa55e", + "metadata": {}, + "outputs": [], + "source": [ + "pd.json_normalize(data[\"abilities\"])" + ] + }, + { + "cell_type": "markdown", + "id": "4680fda5-270d-45e9-ab0f-abebcfe09088", + "metadata": {}, + "source": [ + "### Coincap API" + ] + }, + { + "cell_type": "markdown", + "id": "d43ce6fe-baf5-42da-9e40-d31f2c8e3b27", + "metadata": {}, + "source": [ + "CoinCap is a useful tool for real-time pricing and market activity for over 1,000 cryptocurrencies. Check the documentation at https://docs.coincap.io/" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "bdfda195-9de8-4e2c-8b01-63161db18060", + "metadata": {}, + "outputs": [], + "source": [ + "url = \"http://api.coincap.io/v2/assets\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8cdd990d-cf9b-4bdc-bbd9-cb038685efd5", + "metadata": {}, + "outputs": [], + "source": [ + "response = requests.get(url)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1ea53c95-0868-4440-80ba-13f492e4da6f", + "metadata": {}, + "outputs": [], + "source": [ + "print(response)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8873ba49-0849-40f9-903d-9e49451de9ee", + "metadata": {}, + "outputs": [], + "source": [ + "res = response.json()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "17605693-433c-4581-813a-136209c382e3", + "metadata": {}, + "outputs": [], + "source": [ + "res # it returns a dictionary with all the data" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b56de457-e978-468c-b752-97000ce71baa", + "metadata": {}, + "outputs": [], + "source": [ + "# Again, let's try to have it as a dataframe\n", + "data = pd.DataFrame(res[\"data\"])\n", + "data.head()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2e264ce5-5f90-49f2-8888-78b64808a3ba", + "metadata": {}, + "outputs": [], + "source": [ + "# Lets see if the result would differ in this case using json_normalize\n", + "pd.json_normalize(res[\"data\"])" + ] + }, + { + "cell_type": "markdown", + "id": "52dfaadb-dd5f-4b7d-a832-c101e4d7c80a", + "metadata": {}, + "source": [ + "### Api Jokes\n", + "\n", + "JokeAPI is a REST API that serves uniformly and well formatted jokes.\n", + "It can be used without any API token, membership, registration or payment.\n", + "It supports a variety of filters that can be applied to get just the right jokes you need.\n", + "Check the documentation here: https://jokeapi.dev/ " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e897b2d9-088c-47fd-9d58-fefec4332150", + "metadata": {}, + "outputs": [], + "source": [ + "url_random_joke = \"https://v2.jokeapi.dev/joke/dark?amount=3\"\n", + "request = requests.get(url_random_joke).json()\n", + "request" + ] + }, + { + "cell_type": "markdown", + "id": "bd07c567-1d95-428a-8ac9-9f9909457c49", + "metadata": { + "lang": "en" + }, + "source": [ + "### 💡 Check for understanding\n" + ] + }, + { + "cell_type": "markdown", + "id": "08d0de37-f0e9-4249-a1f2-3e9c091f83da", + "metadata": {}, + "source": [ + "Choose one API from the [Public APIs list](https://github.com/public-apis/public-apis). Attempt to call your selected API, either with or without parameters, and retrieve some valuable information. Document your findings." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "c8f7e8a5-31b7-4444-a80f-8a94dd85e2eb", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/Users/samvesali/Library/Python/3.9/lib/python/site-packages/urllib3/__init__.py:35: NotOpenSSLWarning: urllib3 v2 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'LibreSSL 2.8.3'. See: https://github.com/urllib3/urllib3/issues/3020\n", + " warnings.warn(\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'request': {'type': 'City', 'query': 'Amsterdam, Netherlands', 'language': 'en', 'unit': 'm'}, 'location': {'name': 'Amsterdam', 'country': 'Netherlands', 'region': 'North Holland', 'lat': '52.374', 'lon': '4.890', 'timezone_id': 'Europe/Amsterdam', 'localtime': '2026-04-07 17:33', 'localtime_epoch': 1775583180, 'utc_offset': '2.0'}, 'current': {'observation_time': '03:33 PM', 'temperature': 16, 'weather_code': 113, 'weather_icons': ['https://cdn.worldweatheronline.com/images/wsymbols01_png_64/wsymbol_0001_sunny.png'], 'weather_descriptions': ['Sunny'], 'astro': {'sunrise': '07:01 AM', 'sunset': '08:25 PM', 'moonrise': '02:09 AM', 'moonset': '08:39 AM', 'moon_phase': 'Waning Gibbous', 'moon_illumination': 79}, 'air_quality': {'co': '187.85', 'no2': '7.95', 'o3': '96', 'so2': '1.35', 'pm2_5': '8.95', 'pm10': '13.05', 'us-epa-index': '1', 'gb-defra-index': '1'}, 'wind_speed': 15, 'wind_degree': 87, 'wind_dir': 'E', 'pressure': 1026, 'precip': 0, 'humidity': 36, 'cloudcover': 0, 'feelslike': 16, 'uv_index': 2, 'visibility': 10, 'is_day': 'yes'}}\n" + ] + } + ], + "source": [ + "#I explored the Weatherstack API through its documentation and request format. The current endpoint can be used to retrieve live weather data by sending an access_key and a query parameter.\n", + "#A successful request returns useful information such as temperature, humidity, wind speed, pressure, weather description, UV index, and visibility.\n", + "import requests\n", + "import os\n", + "from dotenv import load_dotenv, find_dotenv\n", + "\n", + "load_dotenv()\n", + "\n", + "my_key = os.getenv(\"API_KEY\")\n", + "\n", + "url = \"https://api.weatherstack.com/current\"\n", + "params = {\n", + " \"access_key\": my_key,\n", + " \"query\": \"Amsterdam\",\n", + " \"units\": \"m\"\n", + "}\n", + "\n", + "response = requests.get(url, params=params)\n", + "data = response.json()\n", + "\n", + "print(data)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f5ee5eb0-0126-49c4-922a-ea5a12330e06", + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "10d38405-f1bd-464f-a319-4a5f29a4cdc2", + "metadata": {}, + "source": [ + "## API Wrappers\n", + "\n", + "A Python wrapper is a Python library or module that provides a more convenient or more \"Pythonic\" interface to another software component, such as a library in another language, a system tool, or a web API. It \"wraps\" the functionality of that component in a way that abstracts away its complexities and makes it easier to use in a Python context. 🙌🏻\n", + "\n", + "One example is the `tweepy` library, which makes obtaining data from Twitter's API relatively straightforward." + ] + }, + { + "cell_type": "markdown", + "id": "796f214d-01cc-46d1-a020-d4bb8d256799", + "metadata": {}, + "source": [ + "## Summary\n", + "- Import the `requests` library.\n", + "- Store the necessary credentials, such as API key or OAuth tokens, if the API requires them.\n", + "- Execute a `request.get` to the desired API endpoint (the API's documentation will provide details on available endpoints).\n", + "- The API returns a JSON response.\n", + "- This JSON can be converted into a dataframe, or you can further explore its elements (typically a list of dictionaries)." + ] + }, + { + "cell_type": "markdown", + "id": "092ff771-4d9a-4d84-98c6-db652fe4ff72", + "metadata": {}, + "source": [ + "## Glossary" + ] + }, + { + "cell_type": "markdown", + "id": "32ca2f95-7c09-4481-8897-0b0533dba4c9", + "metadata": {}, + "source": [ + "* DNS: domain name system.\n", + "* HTTP: is the protocol used to transfer data over the web.\n", + "* API: application programming interface.\n", + "* REST: series of rules, architectural style." + ] + }, + { + "cell_type": "markdown", + "id": "12ba250f-82d7-450b-9038-0f32013dbaf5", + "metadata": {}, + "source": [ + "* `requests`: Python library for interacting with APIs.\n", + "* URL: server name you want to ask information for.\n", + "* endpoint: server service you want to ask information for.\n", + "* parameters: extra parameters to your query.\n", + "* headers: metadata, invisible information." + ] + }, + { + "cell_type": "markdown", + "id": "df2580ce-5fa4-4716-a1f1-3d6c8edd9673", + "metadata": {}, + "source": [ + "## Further materials" + ] + }, + { + "cell_type": "markdown", + "id": "cdae6b8f-bed4-47f8-b9ae-3d8e24e8aba6", + "metadata": {}, + "source": [ + "[5 Simple-To-Use APIs for Beginners](https://dev.to/alanconstantino/5-simple-to-use-apis-for-beginners-2e0n)" + ] + }, + { + "cell_type": "markdown", + "id": "3e747420-7bb6-4ea7-a5d9-e05c90fafd32", + "metadata": {}, + "source": [ + "[RapidAPI](https://rapidapi.com/category/Sports): access thousands of APIs." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.6" + }, + "toc": { + "base_numbering": 1, + "nav_menu": {}, + "number_sections": true, + "sideBar": true, + "skip_h1_title": false, + "title_cell": "Table of Contents", + "title_sidebar": "Contents", + "toc_cell": true, + "toc_position": {}, + "toc_section_display": true, + "toc_window_display": false + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/.ipynb_checkpoints/README-checkpoint.md b/.ipynb_checkpoints/README-checkpoint.md new file mode 100644 index 0000000..936335c --- /dev/null +++ b/.ipynb_checkpoints/README-checkpoint.md @@ -0,0 +1,118 @@ +![logo_ironhack_blue 7](https://user-images.githubusercontent.com/23629340/40541063-a07a0a8a-601a-11e8-91b5-2f13e4e6b441.png) + +# LAB | APIs +
+ +

Learning Goals

+ + + This lab allows you to practice and apply the concepts and techniques taught in class. + + Upon completion of this lab, you will be able to: + +- Use Python libraries such as requests to extract data from APIs, and convert extracted data into a suitable data structure. + +
+
+ +
+ +
+ +
+ + +## Introduction + +Welcome to the world of music data exploration! In this lab, you will venture into the realm of the Spotify API using SpotiPy, a Python library that allows you to interact with Spotify's vast music catalog and playlist data. + +### **Objective** + +The objective of this lab is to acquaint you with the fundamental operations of the Spotify API using SpotiPy. You will learn how to authenticate with the API, perform searches for tracks and artists, and retrieve information about playlists, among other things. + +### **Lab Sections** + +This lab comprises two main sections: + +#### **Section 1: Discovering New Music through Your Favorite Artists** + +In this section, you will use the Spotify API to explore the top tracks of your favorite artists and discover related artists. By the end, you'll have the opportunity to create a playlist featuring your favorite tracks and those of related artists. + +#### **Section 2: Unraveling the World of Playlists** + +In the second section, you will delve into Spotify's featured playlists. You'll learn how to fetch information about these playlists, explore their details, and extract track and artist data from them. + +
+ +Let's embark on this musical journey into the world of Spotify API exploration! + +
+ +**Happy coding!** :heart: + + +## Requirements + +- Fork this repo +- Clone it to your machine + +## Getting Started + +Complete the challenges in the `Jupyter Notebook` file. Follow the instructions and add your code and explanations as necessary. + +## Submission + +- Upon completion, run the following commands: + +```bash +git add . +git commit -m "Solved lab" +git push origin master +``` + +- Paste the link of your lab in Student Portal. + + +## FAQs +
+ I am stuck in the exercise and don't know how to solve the problem or where to start. +
+ + If you are stuck in your code and don't know how to solve the problem or where to start, you should take a step back and try to form a clear question about the specific issue you are facing. This will help you narrow down the problem and come up with potential solutions. + + + For example, is it a concept that you don't understand, or are you receiving an error message that you don't know how to fix? It is usually helpful to try to state the problem as clearly as possible, including any error messages you are receiving. This can help you communicate the issue to others and potentially get help from classmates or online resources. + + + Once you have a clear understanding of the problem, you will be able to start working toward the solution. + + [Back to top](#faqs) + +
+ + +
+ I am unable to push changes to the repository. What should I do? +
+ +There are a couple of possible reasons why you may be unable to *push* changes to a Git repository: + +1. **You have not committed your changes:** Before you can push your changes to the repository, you need to commit them using the `git commit` command. Make sure you have committed your changes and try pushing again. To do this, run the following terminal commands from the project folder: + ```bash + git add . + git commit -m "Your commit message" + git push + ``` +2. **You do not have permission to push to the repository:** If you have cloned the repository directly from the main Ironhack repository without making a *Fork* first, you do not have write access to the repository. +To check which remote repository you have cloned, run the following terminal command from the project folder: + ```bash + git remote -v + ``` +If the link shown is the same as the main Ironhack repository, you will need to fork the repository to your GitHub account first and then clone your fork to your local machine to be able to push the changes. + +**Note**: You should make a copy of your local code to avoid losing it in the process. + + [Back to top](#faqs) + +
+ diff --git a/.ipynb_checkpoints/lab-apis-checkpoint.ipynb b/.ipynb_checkpoints/lab-apis-checkpoint.ipynb new file mode 100644 index 0000000..e923554 --- /dev/null +++ b/.ipynb_checkpoints/lab-apis-checkpoint.ipynb @@ -0,0 +1,566 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "27422808-8a16-4be1-897f-57b379af8991", + "metadata": {}, + "source": [ + "# Lab | APIs" + ] + }, + { + "cell_type": "markdown", + "id": "50f30950-3e31-499a-92ea-1535422c570b", + "metadata": {}, + "source": [ + "In order to use the `Spotify` API (`SpotiPy`), create an account in `Spotify` and follow [these](https://developer.spotify.com/documentation/general/guides/app-settings/) steps. " + ] + }, + { + "cell_type": "markdown", + "id": "a0479b95-6ca5-415e-b894-1f5cb17b055b", + "metadata": {}, + "source": [ + "## Authentication and initializing the API" + ] + }, + { + "cell_type": "markdown", + "id": "47d71611-c617-4972-a0c3-7090c24b399c", + "metadata": {}, + "source": [ + "Save your client ID and your client secret in your preferred way, and read it or load it into the following variables:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ea44c82a-5c07-4dbc-beb2-bba2601bb75e", + "metadata": {}, + "outputs": [], + "source": [ + "CLIENT_ID = \"\"\n", + "CLIENT_SECRET = \"\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "319cfd4e-f6fb-4419-80e0-e3983cd25e43", + "metadata": {}, + "outputs": [], + "source": [ + "CLIENT_ID = \"4109b2d9f5014671863bb2df1d1fbdd3\"\n", + "CLIENT_SECRET = \"5cb719fae80c4191a9c0b288f51bfe50\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "04e12954-fd70-4311-88a5-fb7e2c29799c", + "metadata": {}, + "outputs": [], + "source": [ + "# If you havent done so, install the spotipy wrapper\n", + "!pip install spotipy" + ] + }, + { + "cell_type": "markdown", + "id": "dc0e86da-8846-4207-84c3-cd20b9e01d0e", + "metadata": {}, + "source": [ + "Once you have done it, we will start initializing the API." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "03034bc6-9858-412a-83b7-18abdc345d7e", + "metadata": {}, + "outputs": [], + "source": [ + "import spotipy\n", + "from spotipy.oauth2 import SpotifyClientCredentials\n", + "\n", + "#Initialize SpotiPy with user credentials\n", + "sp = spotipy.Spotify(auth_manager=SpotifyClientCredentials(client_id=CLIENT_ID,\n", + " client_secret=CLIENT_SECRET))\n" + ] + }, + { + "cell_type": "markdown", + "id": "8fed9628-08d7-4290-a4be-5527696b01c5", + "metadata": {}, + "source": [ + "## Using the search method" + ] + }, + { + "cell_type": "markdown", + "id": "6575a3c6-f25a-4905-b1f3-c0efd50dcc1e", + "metadata": {}, + "source": [ + "Now, let's use the search method by introducing a \"query\". For example, let's try searching for \"Lady Gaga\":" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7772a1e0-9692-4d04-a590-76111a272d8d", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "results = sp.search(q='Lady Gaga', limit=50)\n", + "results" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "edc14c95-474b-4e2a-aea3-bdfd0a205546", + "metadata": {}, + "outputs": [], + "source": [ + "results.keys() # We can see that we only have tracks" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ad8ef934-1dbb-4008-ac8e-f5c29823fe6a", + "metadata": {}, + "outputs": [], + "source": [ + "results[\"tracks\"].keys() # Let's check the values" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "004b7814-4dd5-408e-b7ba-1da87f9250cb", + "metadata": {}, + "outputs": [], + "source": [ + "results[\"tracks\"][\"href\"] # Query we have searched " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7285cedd-fbe1-47cf-98d5-a7fdc3e5c8b8", + "metadata": { + "scrolled": true, + "tags": [] + }, + "outputs": [], + "source": [ + "results[\"tracks\"][\"items\"] #items (actual tracks)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "529fff56-47d3-4d78-8ff5-9530fe290d1d", + "metadata": {}, + "outputs": [], + "source": [ + "results[\"tracks\"][\"limit\"]#Limit we have chosen" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "92c64c57-3bd2-4d42-bbd1-84a040f1e02a", + "metadata": {}, + "outputs": [], + "source": [ + "results[\"tracks\"][\"next\"] #link to the next page (next 50 tracks)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f5ccdf79-5d9e-40de-adb9-2cc1e5b7c74a", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "results[\"tracks\"][\"offset\"] # Actual offset (starting point)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "356730c1-60a2-4ea8-bd2c-e0522bab8a4d", + "metadata": {}, + "outputs": [], + "source": [ + "results[\"tracks\"][\"previous\"] #Previous search" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7c44c8fd-63ea-45ba-94bd-5c5e8e1458b3", + "metadata": {}, + "outputs": [], + "source": [ + "results[\"tracks\"][\"total\"] # Number of matches" + ] + }, + { + "cell_type": "markdown", + "id": "7a127c64-3274-4ecc-aa0f-83ae34af4655", + "metadata": {}, + "source": [ + "## Exploring the tracks" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6c3541a2-0fd2-41e0-9b27-60a7dc36c4cb", + "metadata": { + "scrolled": true, + "tags": [] + }, + "outputs": [], + "source": [ + "results[\"tracks\"][\"items\"][0] # Explore the first song" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f2c35eb2-3ea6-4329-9f29-7c062f466638", + "metadata": {}, + "outputs": [], + "source": [ + "results[\"tracks\"][\"items\"][0].keys() # We will focus on album, artists, id, name, popularity, type and uri" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "889ca3c3-b0c8-4037-96fb-6add847f537f", + "metadata": {}, + "outputs": [], + "source": [ + "# Track artists\n", + "results[\"tracks\"][\"items\"][0][\"artists\"] " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a9c6a0b2-cea7-48ff-8c51-179d15388aa2", + "metadata": {}, + "outputs": [], + "source": [ + "# Track artists names\n", + "for artist in results[\"tracks\"][\"items\"][0][\"artists\"]:\n", + " print(artist[\"name\"])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6a826e9c-d2e7-4537-a82c-3dc3a2b80b9f", + "metadata": {}, + "outputs": [], + "source": [ + "# Track ID\n", + "results[\"tracks\"][\"items\"][0][\"id\"] " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a5bd871b-6087-4680-819c-1a1d8ba879bc", + "metadata": {}, + "outputs": [], + "source": [ + "# Track name\n", + "results[\"tracks\"][\"items\"][0][\"name\"] " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "812661f1-29db-452f-a719-cdfbe95ba9f6", + "metadata": {}, + "outputs": [], + "source": [ + "# Popularity index\n", + "results[\"tracks\"][\"items\"][0][\"popularity\"] " + ] + }, + { + "cell_type": "markdown", + "id": "0e81c762-e6c5-424e-a4eb-12ab45dffb9f", + "metadata": {}, + "source": [ + "Spotify songs are identified by either a \"url\", a \"uri\", or an \"id\". \n", + "\n", + "- The `id` is an alphanumeric code, and it's the nuclear part of the identifier.\n", + "\n", + "- The `uri` contains \"spotify:track\" before the id. An uri is useful because it can be searched manually in the Spotify app.\n", + "\n", + "- The `url` is a link to the song on the Spotify web player.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8bcdccfc-dde9-4f4b-8af5-3caa335b89b5", + "metadata": {}, + "outputs": [], + "source": [ + "results[\"tracks\"][\"items\"][0][\"uri\"]" + ] + }, + { + "cell_type": "markdown", + "id": "71c3c9c1-4ec2-42bf-a243-b21105ae1699", + "metadata": {}, + "source": [ + "## Exercise 1: Discovering New Music through Your Favorite Artists\n", + "\n", + "**Objective:** \n", + "Uncover new music by exploring the top tracks of your favorite artists and their related artists.\n", + "\n", + "**Instructions:**\n", + "\n", + "1. **List Your Favorite Artists**:\n", + " - Make a list of your three favorite artists and store it in a variable named `artists`.\n", + " - Example: `artists = [\"Los Fabulosos Cadillacs\", \"Manu Chao\", \"Muchachito Bombo Infierno\"]`.\n", + "\n", + "2. **Fetch Top Tracks**:\n", + " - Write a function named `get_top_tracks`.\n", + " - This function should accept an artist's name and return the name of the first 5 top tracks by that artist.\n", + " - Use the function `get_top_tracks` to get the first 5 top tracks for each artist in your `artists` list and store the results in a new list named `top_tracks_list`.\n", + "\n", + "3. **Discover Related Artists**:\n", + " - Write a function named `find_related_artists`.\n", + " - This function should accept an artist's name and return the names of the first 5 artists related to the provided artist.\n", + " - Store the results in a list named `related_artists_list`.\n", + "\n", + "**Challenge:** \n", + "Combine the above steps to create a playlist that includes the top tracks of your favorite artists and the top tracks of the artists related to them." + ] + }, + { + "cell_type": "markdown", + "id": "0c442378-e26f-47c8-b4f1-b4fa07089935", + "metadata": {}, + "source": [ + "**Hint Section for 3. **Discover Related Artists**:**\n", + "\n", + "1. **Getting Artist ID**:\n", + " - Remember that every artist on Spotify has a unique identifier: their `id`. To get the related artists, you first need to fetch the ID of the given artist.\n", + " - Consider using the `sp.search` method to query the artist's name. The method requires a `q` parameter, which is your query (in this case, the artist's name). It also has a `limit` parameter, which specifies the number of tracks it returns. In this case, 1 track is enough, since we just want the artist ID. \n", + " - Each track in the results has an associated 'artists' field. This field is a list containing details about all artists involved in that track.\n", + " - For most tracks, especially those by a single artist, this list will contain one artist. From this artist's details, you can extract the 'id' field, which is the unique identifier for that artist on Spotify.\n", + "\n", + "\n", + "3. **Fetching Related Artists**:\n", + " - Once you have the artist's ID, you can use another SpotiPy method to fetch related artists. Think about which SpotiPy method allows you to get related artists using an artist's ID. Here is the documentation link: https://spotipy.readthedocs.io/en/2.22.1/. \n", + " - This method will return a list of related artists. You'll need to extract the relevant details (artist names) from this list.\n", + "\n", + "4. **Iterating for Multiple Artists**:\n", + " - Once you have a function that returns related artists names for one artist, you can use a list comprehension to apply this function to a list of artist names.\n", + "\n", + "5. **Testing**:\n", + " - Always test your function with one artist name first. Once you're confident it works, then apply it to the entire list.\n", + "\n", + "Remember, the key is to break the problem down into manageable steps. Use the SpotiPy documentation as a resource to understand available methods and their return structures." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "29694252-f217-454d-8881-681b2b6eeb1e", + "metadata": {}, + "outputs": [], + "source": [ + "# Your answer here" + ] + }, + { + "cell_type": "markdown", + "id": "94ad5fdc-22e5-4521-8aa1-c6833eb7e949", + "metadata": {}, + "source": [ + "## Playlists\n", + "\n", + "The `sp.featured_playlists()` method in `spotipy` fetches a list of Spotify's featured playlists at a given moment. These are curated playlists that Spotify often highlights on the platform's homepage. The method provides a snapshot of the playlists that are being promoted or featured by Spotify at the time of the request.\n", + "\n", + "Once you've fetched the featured playlists, you can extract their IDs (and other details)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "25fb0cf8-c13a-41b0-b8f8-7e0700fd1e41", + "metadata": {}, + "outputs": [], + "source": [ + "sp.featured_playlists() # We get a playlist id of a playlist we like" + ] + }, + { + "cell_type": "markdown", + "id": "90f558f3-c638-4df4-b5a4-e24f7847d52a", + "metadata": {}, + "source": [ + "### Getting a Playlist's Details\n", + "To fetch details about a specific playlist, you can use the playlist method. You'll need the playlist's Spotify ID." + ] + }, + { + "cell_type": "markdown", + "id": "0eef529f-617f-4ea3-8156-07472ac8e6d5", + "metadata": {}, + "source": [ + "In this example, we will use the following playlist id: *37i9dQZF1DXd9zR7tdziuQ*" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "46d35121-9256-4cf4-81f5-118b87f7af32", + "metadata": {}, + "outputs": [], + "source": [ + "playlist_id = \"37i9dQZF1DXd9zR7tdziuQ\"\n", + "playlist = sp.playlist(playlist_id)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5260f67f-6024-4fee-8449-30904f03bf76", + "metadata": {}, + "outputs": [], + "source": [ + "print(playlist['name']) # Print the playlist's name\n", + "print(playlist['description']) # Print the playlist's description" + ] + }, + { + "cell_type": "markdown", + "id": "13bc8631-69f0-4b98-9cc9-5baecbaea9ba", + "metadata": {}, + "source": [ + "### Getting Tracks from a Playlist\n", + "If you want to get the tracks from a specific playlist, you can use the playlist_tracks method." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "69c78f8d-7e6a-4d15-bcbb-fc93edb82433", + "metadata": {}, + "outputs": [], + "source": [ + "tracks = sp.playlist_tracks(playlist_id)\n", + "for track in tracks['items']:\n", + " print(track['track']['name']) # Print each track's name" + ] + }, + { + "cell_type": "markdown", + "id": "2775714d-acc7-4555-96bd-2c541ab0855e", + "metadata": {}, + "source": [ + "### Getting Artists from a Playlist\n", + "\n", + "To extract all the artists from the tracks in a playlist, you'd typically follow these steps:\n", + "\n", + "1. Fetch the playlist's tracks.\n", + "2. Iterate through each track.\n", + "3. For each track, extract the associated artists." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "65c5e5c4-f186-42c6-b136-4ef02b0b01ff", + "metadata": {}, + "outputs": [], + "source": [ + "# List to store unique artist names\n", + "artists_list = []\n", + "\n", + "for track_item in tracks['items']:\n", + " track = track_item['track']\n", + " for artist in track['artists']:\n", + " artist_name = artist['name']\n", + " if artist_name not in artists_list: # This ensures each artist is added only once\n", + " artists_list.append(artist_name)\n", + "\n", + "print(artists_list)" + ] + }, + { + "cell_type": "markdown", + "id": "7b52207e-a4f0-4f90-9f4e-3170d7f0f3fe", + "metadata": {}, + "source": [ + "## Exercise 2: Unraveling the World of Playlists\n", + "\n", + "\n", + "1. **Featured Exploration**: \n", + " - Fetch the list of Spotify's current featured playlists. \n", + " - Extract and display the names and IDs of the top 5 featured playlists.\n", + " \n", + "2. **Deep Dive**:\n", + " - Choose any one of the top 5 featured playlists (you can choose the one you personally find most interesting or simply pick one randomly).\n", + " - Fetch and display its name, description, and total track count.\n", + "\n", + "3. **Track-tastic**:\n", + " - Extract and display the names of the first 10 tracks in the chosen playlist.\n", + "\n", + "4. **Artistic Flair**:\n", + " - Create a dictionary where the keys are the names of the first 10 tracks, and the values are lists containing the names of the artists associated with each track.\n", + " - For example: `{\"TrackName1\": [\"Artist1\", \"Artist2\"], \"TrackName2\": [\"Artist3\"]}`\n", + " " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ed92d961-9646-4375-a386-ccc320a958f5", + "metadata": {}, + "outputs": [], + "source": [ + "# Your answer here" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/2.5_apis (1).ipynb b/2.5_apis (1).ipynb new file mode 100644 index 0000000..8518b58 --- /dev/null +++ b/2.5_apis (1).ipynb @@ -0,0 +1,2016 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "35870f3e-1791-4fc3-ba77-6b01a78bca50", + "metadata": {}, + "source": [ + "# APIS " + ] + }, + { + "cell_type": "markdown", + "id": "34841864-ea2a-436e-b85a-5cc871472849", + "metadata": {}, + "source": [ + "![apis_gif](https://media.giphy.com/media/wepUQluC5smgEd4Qz4/giphy.gif)" + ] + }, + { + "cell_type": "markdown", + "id": "a6795d4c-fdfe-4f6d-ad6b-2479d9b0826d", + "metadata": { + "toc": true + }, + "source": [ + "

Table of Contents

\n", + "
" + ] + }, + { + "cell_type": "markdown", + "id": "78337d49-bd91-4990-b1d6-fd2d4c03e6df", + "metadata": {}, + "source": [ + "## Web Communication Fundamentals" + ] + }, + { + "cell_type": "markdown", + "id": "1f5600c7-4e6b-4b57-986e-740538741ca9", + "metadata": {}, + "source": [ + "![enjuto](https://www.publico.es/files/article_main/uploads//2014/12/13/548bb18f1d9ef.jpg)" + ] + }, + { + "cell_type": "markdown", + "id": "582db4b0-9f2d-4847-b52c-f11eb6ae8933", + "metadata": {}, + "source": [ + "### DNS, IP" + ] + }, + { + "cell_type": "markdown", + "id": "1e7a9565-1961-4631-a6a1-71735dae6f16", + "metadata": {}, + "source": [ + "[How do we connect to www.google.com?](https://www.youtube.com/watch?v=sUhEqT_HSBI&ab_channel=ProfeSang)\n", + "* **DNS (Domain Name System)**: it's essentially the phonebook of the internet. Humans access information online through domain names, like \"google.com\". Web browsers, however, interact through Internet Protocol (IP) addresses. In this example, the DNS maps the internet address www.google.com to the server's IP: 216.58.222.196\n", + " * **IP**: Server identification. A code that allows information to be sent and received by the correct parties\n", + " * **Domain Providers**: Sell and purchase Internet domains" + ] + }, + { + "cell_type": "markdown", + "id": "7d42f993-6f37-42c7-8f04-4c5526c0825e", + "metadata": {}, + "source": [ + "*When making API calls to specific domains, the DNS translates the human-readable domain name into an IP address.*" + ] + }, + { + "cell_type": "markdown", + "id": "216e4441-45eb-4a0a-b578-f4f056a724b1", + "metadata": {}, + "source": [ + "### HTTP" + ] + }, + { + "cell_type": "markdown", + "id": "6d120e37-ac79-44c9-8fc5-3fe07f4ad7f6", + "metadata": {}, + "source": [ + "**H**yper **T**ext **T**ransfer **P**rotocol\n", + "- HTTP is a communications protocol that provides a structure for requests between the client and the server on a network.\n", + "- For example, the web browser on the user's computer (the client) uses the HTTP protocol to request information from a website on a server.\n", + "\n", + "*We will use APIs that employ the **HTTP** protocol as transport.*" + ] + }, + { + "cell_type": "markdown", + "id": "1b9f3b1b-af11-419a-bd55-771a937b72dc", + "metadata": {}, + "source": [ + "### URL\n", + "Contains information about the resource being requested from the **server**." + ] + }, + { + "cell_type": "markdown", + "id": "c6a7522a-6679-4d2d-9587-6b583c41a55f", + "metadata": {}, + "source": [ + "![](https://github.com/data-bootcamp-v4/lessons/blob/main/img/http.png?raw=true)" + ] + }, + { + "cell_type": "markdown", + "id": "246020a7-6a89-4857-98c4-8f29c1d3c827", + "metadata": {}, + "source": [ + "Examples:\n", + "\n", + "- https://www.google.com/webhp?authuser=2\n", + "- https://www.towardsdatascience.com\n", + "- https://www.ironhack.com/\n", + " - Protocol: https (https == http is the same, but https is encrypted)\n", + " - Domain Name --> ironhack\n", + " - TLD (Top-Level Domain) --> .com" + ] + }, + { + "cell_type": "markdown", + "id": "dd726622-8824-4e1f-a166-bb0d7e94fd08", + "metadata": {}, + "source": [ + "*When calling an API, you specify the URL of the API endpoint, which tells the system where to send the request.*" + ] + }, + { + "cell_type": "markdown", + "id": "5fd2c3c5-1e3e-44a2-923b-c6dc1bb19f7c", + "metadata": {}, + "source": [ + "## HTTP Requests and Responses" + ] + }, + { + "cell_type": "markdown", + "id": "1b658ec7-5230-44ab-b10b-f8cc1c65edab", + "metadata": {}, + "source": [ + "![](https://github.com/data-bootcamp-v4/lessons/blob/main/img/request-response.png?raw=true)" + ] + }, + { + "cell_type": "markdown", + "id": "4941c601-e2f2-41bd-a0f3-c5e33fc8e282", + "metadata": {}, + "source": [ + "**Requests** and **responses** are fundamental components of the client-server communication model." + ] + }, + { + "cell_type": "markdown", + "id": "b75d6570-aae7-492a-bcd0-86da6d102545", + "metadata": {}, + "source": [ + "### Requests" + ] + }, + { + "cell_type": "markdown", + "id": "6d39d707-2230-4f72-af04-8c49abeb413e", + "metadata": {}, + "source": [ + "These are queries or calls **sent by the client** (such as a web browser or other software) **to the server** in order to receive information (a **response**). \n", + "\n", + "A request typically consists of:\n", + "- A method (such as GET, POST, PUT, DELETE) that defines the action to be performed\n", + " * GET: read the information of the resource, without modifying it in any way. Accessing the website from the browser **gets** information.\n", + "- The URL or endpoint specifying the resource\n", + "- Optional additional information such as:\n", + " - Headers (metadata): User-Agent, Accept-Language\n", + " - Parameters\n", + " - Body content. \n", + " \n", + "For example, a client might send a GET request to retrieve information from a web page or a POST request to submit form data." + ] + }, + { + "cell_type": "markdown", + "id": "03152bb9-ab13-458c-94d2-f5cb53117056", + "metadata": {}, + "source": [ + "### Response" + ] + }, + { + "cell_type": "markdown", + "id": "fc180f91-d6bf-4cb9-bf07-443d47ba0c4f", + "metadata": {}, + "source": [ + "These are the answers or data sent **by the server back to the client in reply to a request**. \n", + "\n", + "A response typically includes:\n", + "- A status code that indicates the success or failure of the request\n", + "- Headers with meta-information about the server's behavior\n", + "- The actual content or data (if applicable), such as HTML, JSON (similar to a Python dictionary), images, or other media types.\n", + "\n", + "An important part of the **header** is the **status code**. This code is a numerical value that indicates the server's result. There are different status codes depending on whether the server has managed to carry out the request or has not managed to do anything. These are some groups of status codes:\n", + "\n", + "- **2xx successful**: the request was successfully received, understood, and accepted\n", + "- **3xx redirection**: more actions are required to complete the request\n", + "- **4xx client error**: the request contains incorrect syntax or cannot be fulfilled\n", + "- **5xx server error**: the server has failed to complete an apparently valid request\n", + "\n", + "Complete list:\n", + "https://en.wikipedia.org/wiki/List_of_HTTP_status_codes\n", + "\n", + "Much more fun:\n", + "https://http.cat/" + ] + }, + { + "cell_type": "markdown", + "id": "1f1ad409-99e9-4701-b879-f81d13ece283", + "metadata": { + "tags": [] + }, + "source": [ + "## APIs" + ] + }, + { + "cell_type": "markdown", + "id": "492c462e-3d81-4330-828f-ff2b027989d8", + "metadata": {}, + "source": [ + "**A**pplication **P**rogramming **I**nterface\n", + "\n", + "- APIs define a set of rules and protocols that allow different software applications to communicate with each other.\n", + "- The client calls the server through an API, and the server responds.\n", + "- Within a project, the backend may want to share information with the frontend without access to the database." + ] + }, + { + "cell_type": "markdown", + "id": "0849403b-d633-4756-82d2-cb74ac4d254b", + "metadata": {}, + "source": [ + "![](https://github.com/data-bootcamp-v4/lessons/blob/main/img/api.png?raw=true)" + ] + }, + { + "cell_type": "markdown", + "id": "b86f18cb-8590-4a6e-896b-c5e465e7719f", + "metadata": {}, + "source": [ + "### RESTful APIs" + ] + }, + { + "cell_type": "markdown", + "id": "62891457-6b5c-4443-9fb5-77f901ea7571", + "metadata": {}, + "source": [ + "As a data analyst, you often need to access data from various sources. RESTful APIs provide a **standardized way** to retrieve, update, or delete data from other systems." + ] + }, + { + "cell_type": "markdown", + "id": "5c2bea7b-f629-45f3-94b3-6cb6eac1272e", + "metadata": {}, + "source": [ + "- Usually, **we (the client) send a request, and they (the server) return a response, often in JSON - JavaScript Object Notation - format**. While JSON looks similar to the dictionaries we're used to in Python, it's worth noting that when represented as raw text (e.g., in an API response), the entire JSON structure is a *string*. However, within that structure, JSON can represent various data types like numbers, booleans, and arrays, not just strings.\n", + "\n", + "- APIs always need to provide documentation for their various services: **endpoints**. Each endpoint is a different URL. \n", + "- Sometimes we have to pass **parameters** to an API endpoint, similar to when we pass parameters to a Python function.\n", + "\n", + "At this point in the Bootcamp, **reading documentation becomes essential**, as each API will \"work\" differently, and to use it, we will need to know what it requires.\n", + "\n", + "[Here](https://github.com/public-apis/public-apis) is a repository where you can find several free APIs if you feel like exploring a bit." + ] + }, + { + "cell_type": "markdown", + "id": "33c6c206-715a-42b8-929c-691d9c66fe4d", + "metadata": {}, + "source": [ + "## Requests in Python" + ] + }, + { + "cell_type": "markdown", + "id": "515484d8-81ec-48c1-abad-c778232b583f", + "metadata": {}, + "source": [ + "You can make requests to RESTful APIs using libraries like `requests`. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "54d23d14-7425-45c6-8ba5-74e5db2c047a", + "metadata": {}, + "outputs": [], + "source": [ + "# If you don't have it installed you can do use using pip or pip3\n", + "\n", + "#!pip install requests" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9ddac917-8539-46e8-abc9-37111e2fa74c", + "metadata": {}, + "outputs": [], + "source": [ + "import requests" + ] + }, + { + "cell_type": "markdown", + "id": "da5aff9d-8a79-4171-96b1-b7f0d2daf1cd", + "metadata": {}, + "source": [ + "### GET" + ] + }, + { + "cell_type": "markdown", + "id": "2018fa77-7bce-437e-a806-69844737210b", + "metadata": {}, + "source": [ + "Making a `GET request` to the API is simply a call to a `URL` that returns information when provided with the appropriate `parameters`. We will only perform **GET** requests." + ] + }, + { + "cell_type": "markdown", + "id": "718172e2-5654-4536-bd72-594d80b31f8c", + "metadata": {}, + "source": [ + "Here is an example on how to make a request with Python:\n", + "\n", + "```python\n", + "url = \"https://api.example.com/products\"\n", + "response = requests.get(url)\n", + "\n", + "if response.status_code == 200:\n", + " products = response.json()\n", + " # Now you can analyze and work with the products data in Python\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "307395ef-cfd3-4175-99d5-a6ce6213adb1", + "metadata": {}, + "source": [ + "#### Accessing an endpoint\n", + "\n", + "As we mentioned above, APIs always need to provide documentation for their various services: **endpoints**. Each endpoint is a different URL. \n", + "\n", + "**Example: ISS API**" + ] + }, + { + "cell_type": "markdown", + "id": "792140dd-3cda-4355-86bd-dd6732880803", + "metadata": {}, + "source": [ + "Let's get information from ISS (International Space Station)! We'll start looking at the [ISS API documentation](https://wheretheiss.at/w/developer)" + ] + }, + { + "cell_type": "markdown", + "id": "a2be4164-137b-411d-a78c-38bdd94ce9ed", + "metadata": {}, + "source": [ + "This API allows you to access various data related to the International Space Station (ISS), including its current, past, or future position, timezone information for specific coordinates, and more.\n", + "\n", + "**Key Features**\n", + "- **Authentication**: No authentication is currently required, but future endpoints may include this.\n", + "- **Rate Limiting**: Limited to approximately 1 request per second.\n", + "- **Responses**: Default to JSON format, with optional parameters to modify response appearance.\n", + "- **Endpoints**: Several endpoints provide different types of information:\n", + " - **satellites**: Information about satellites, including the ISS.\n", + " - **satellites/[id]**: Position, velocity, and related information for a satellite.\n", + " - **satellites/[id]/positions**: Position data for specific timestamps.\n", + " - **satellites/[id]/tles**: TLE (Two-Line Element Set) data in either JSON or text format.\n", + " - **coordinates/[lat,lon]**: Timezone information for specific coordinates.\n" + ] + }, + { + "cell_type": "markdown", + "id": "f1272582-bc6d-4f43-9e39-47d39ffd1bfd", + "metadata": {}, + "source": [ + "**Examples**\n", + "- Satellite details: `https://api.wheretheiss.at/v1/satellites`\n", + "- ISS position: `https://api.wheretheiss.at/v1/satellites/25544`\n", + "- Coordinates information: `https://api.wheretheiss.at/v1/coordinates/37.795517,-122.393693`\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "5ca5b7da-eec0-41ca-924b-67687f794e23", + "metadata": {}, + "source": [ + "**Endpoint satellites**" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "05fad64c-5f20-4b7f-a865-9659b9cc8783", + "metadata": {}, + "outputs": [], + "source": [ + "# We'll use the endpoint satellites, we read it gives us information about satellites\n", + "\n", + "url = \"https://api.wheretheiss.at/v1/satellites\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "68f33b09-b943-486c-98e0-2d18b25640c5", + "metadata": {}, + "outputs": [], + "source": [ + "response = requests.get(url) # We use get method to make the request and get information from the API" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "67938f79-0981-471e-9201-89352e99775b", + "metadata": {}, + "outputs": [], + "source": [ + "type(response) # Lets look at the type of the response" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "310b2c20-4a51-47de-9983-1e266496c951", + "metadata": {}, + "outputs": [], + "source": [ + "response.status_code # We can access status code. 200 is OK" + ] + }, + { + "cell_type": "markdown", + "id": "488cbcbe-2d4d-42c2-b92a-ec76b70b705b", + "metadata": {}, + "source": [ + "We can access the response content, which returns a string" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5196a512-6690-49c9-971a-bc9cb2383932", + "metadata": {}, + "outputs": [], + "source": [ + "response.content" + ] + }, + { + "cell_type": "markdown", + "id": "820ad33e-4664-46b6-a51a-0b277ba04c4d", + "metadata": {}, + "source": [ + "If response Content-Type is json, we can access it better with `.json()`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7f44e27c-7866-4e9c-8965-a20ccaef0e43", + "metadata": {}, + "outputs": [], + "source": [ + "info = response.json()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cd406dbc-74f9-490b-b8c6-73ef02e9b9d4", + "metadata": {}, + "outputs": [], + "source": [ + "info" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "992d8cf6-8e5c-448d-bb1c-e9416aefe428", + "metadata": {}, + "outputs": [], + "source": [ + "type(info) # It returns a list of dictionaries" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1a7d7dc2-f2d7-44a7-9bda-e5cc0490dffe", + "metadata": {}, + "outputs": [], + "source": [ + "info[0][\"id\"] # So we can access the id like this" + ] + }, + { + "cell_type": "markdown", + "id": "903f3024-0998-459d-bb20-60a639def27e", + "metadata": {}, + "source": [ + "#### Response headers \n", + "\n", + "Response headers are part of the HTTP response that a server sends back to the client after a request has been made. These headers provide meta-information about the response and can affect how the client handles the response.\n", + "\n", + "Here are some common response headers and what they typically represent:\n", + "\n", + "1. **`Date`**: Represents the date and time at which the response was sent.\n", + "\n", + "2. **`Server`**: Provides information about the software used by the originating server.\n", + "\n", + "3. **`X-Rate-Limit`**: Sometimes used in APIs to inform the client about rate limiting policies, such as the number of allowed requests in a given time frame.\n", + "\n", + "4. **`Content-Type`**: Specifies the media type of the resource or data the server is sending back. For example, it could be `application/json` for a JSON object, `text/html` for an HTML page, or `image/png` for an image.\n", + "\n", + "And more." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a53d51bc-2346-4139-a23f-49a34dc3b475", + "metadata": {}, + "outputs": [], + "source": [ + "response.headers # We can also access the response headers" + ] + }, + { + "cell_type": "markdown", + "id": "a46967a4-763f-4d4e-bca9-c3a37742f65e", + "metadata": {}, + "source": [ + "#### Parameters" + ] + }, + { + "cell_type": "markdown", + "id": "69017232-6063-4ee5-a13c-0eb40f182b1e", + "metadata": {}, + "source": [ + "As we mentioned above, sometimes we can pass **parameters** to an API endpoint, similar to when we pass parameters to a Python function." + ] + }, + { + "cell_type": "markdown", + "id": "4dbf1646-5953-44b9-a2e3-a7dbee7f1a4e", + "metadata": {}, + "source": [ + "In the example above, we didn't use any parameters in the endpoint `satellites` since the [documentation](\"https://wheretheiss.at/w/developer\") said *Parameters: None*." + ] + }, + { + "cell_type": "markdown", + "id": "ed0d4cf7-fa4d-4c83-8d56-4a25accc3d01", + "metadata": {}, + "source": [ + "API parameters are specific values that you include in a request to an API endpoint to filter, sort, or detail the data that you want to retrieve. They allow you to customize the request to get exactly the information you need." + ] + }, + { + "cell_type": "markdown", + "id": "dbf6baed-1967-46ea-908f-bff2fd59dc46", + "metadata": {}, + "source": [ + "There are several types of parameters that can be used in API requests, such as Path Parameters, Query Parameters, Header Parameters and Request Body Parameters. Let's look at them with an example using another endpoint." + ] + }, + { + "cell_type": "markdown", + "id": "f854df58-1a0a-4759-9991-30816cf45c92", + "metadata": {}, + "source": [ + "##### 1. **Path Parameters**\n", + "\n", + "These are embedded in the URL path and are used to identify a specific resource. For example, in the URL above `https://api.wheretheiss.at/v1/satellites/25544`, the number `25544` is a path parameter that identifies a specific sattelite." + ] + }, + { + "cell_type": "markdown", + "id": "df9a0f8c-b363-4ceb-bcc6-04128ec2bffc", + "metadata": { + "tags": [] + }, + "source": [ + "**Endpoint satellites/id**" + ] + }, + { + "cell_type": "markdown", + "id": "f7f1892c-1908-438d-a072-a77fc2214e52", + "metadata": {}, + "source": [ + "Lets use endpoint satellites/[id] with the id we got from the previous endpoint.\n", + "Important, we need to provide the whole url for the endpoint as a `string`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "60e19b4e-1476-456c-8272-a3175708368e", + "metadata": {}, + "outputs": [], + "source": [ + "url_iss_position = \"https://api.wheretheiss.at/v1/satellites/\"+str(info[0][\"id\"])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "42cb7357-ff48-4f7d-81df-8060c7ed794f", + "metadata": {}, + "outputs": [], + "source": [ + "response = requests.get(url_iss_position)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "340a190c-6010-4134-bfd5-f4acc5484e8b", + "metadata": {}, + "outputs": [], + "source": [ + "response.status_code" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a9ddae28-476f-4133-8a25-4e35c3fac891", + "metadata": {}, + "outputs": [], + "source": [ + "response.json()" + ] + }, + { + "cell_type": "markdown", + "id": "7c21289e-ed73-4fdf-8ff2-8ec53d1bedbb", + "metadata": {}, + "source": [ + "Every time we call the endpoint, the information changes since it gives us the current position of the satellite. Let's check that with a loop to gather information at different times." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c02e4902-90b6-4f31-8326-2dcee2478aa8", + "metadata": {}, + "outputs": [], + "source": [ + "import time\n", + "\n", + "positions = []\n", + "\n", + "for i in range(10):\n", + " response = requests.get(url_iss_position)\n", + " data = response.json()\n", + " \n", + " positions.append(data)\n", + " time.sleep(0.5)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3801506d-82c7-4c1b-b4ab-81fce6a73b36", + "metadata": {}, + "outputs": [], + "source": [ + "type(positions) # We created a list with responses" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5f3da8cc-7ad5-4313-8bbd-71e733359747", + "metadata": {}, + "outputs": [], + "source": [ + "type(positions[0]) # Is a list of dictionaries" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8d24d4c8-860c-4a6f-a5ec-db44ed9d4c65", + "metadata": {}, + "outputs": [], + "source": [ + "positions[0] # So we can access each dictionary by its position in the list" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b588249b-ef73-4cd0-8c3f-4a7a46c9156f", + "metadata": {}, + "outputs": [], + "source": [ + "positions[9]" + ] + }, + { + "cell_type": "markdown", + "id": "57c55b8a-1b23-4e9e-8f7b-d24b7a1b375e", + "metadata": {}, + "source": [ + "Let's extract 10 latitudes from the 10 dictionaries in the list, with 2 decimals." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3371418e-6637-4857-913f-01fd244bd687", + "metadata": {}, + "outputs": [], + "source": [ + "latitudes = [round(p[\"latitude\"], 2) for p in positions]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "256a7492-b9de-416b-9cad-3b4155f767e4", + "metadata": {}, + "outputs": [], + "source": [ + "latitudes" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7b841d34-6a6f-4f31-9764-1d5ac4d2f35f", + "metadata": {}, + "outputs": [], + "source": [ + "import pandas as pd\n", + "pd.DataFrame(positions) # We can create a dataframe from the API response to work with it better" + ] + }, + { + "cell_type": "markdown", + "id": "204abde8-009b-4217-8295-51fbde187742", + "metadata": { + "tags": [] + }, + "source": [ + "\n", + "##### 2. **Query Parameters**\n", + "\n", + "These are added to the end of the URL after a question mark (`?`) and are often used to filter or sort the response. \n", + "\n", + "For example, in the ISS documentation, it mentions that there is a parameter `units` that can take values `miles` or `kilometers`. So in the URL `https://api.wheretheiss.at/v1/satellites/25544?units=miles`, we added `units=miles` which is a query parameter that shows the data in \"miles\". \n", + "\n", + "In general, we add the parameters like this `?param1=value1¶m2=value2...` at the end of the URL." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b252f4b2-f739-4d78-8a14-7933b946c309", + "metadata": {}, + "outputs": [], + "source": [ + "url_iss_position" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b8bfc174-f81f-41cc-92f4-eec321e1961a", + "metadata": {}, + "outputs": [], + "source": [ + "# We saw in the documentation that we can add a parameter units to use miles or kilometers\n", + "\n", + "url_iss_position2 = 'https://api.wheretheiss.at/v1/satellites/25544?units=miles'" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "47ead56f-1882-41f4-8850-67a6611fe36d", + "metadata": {}, + "outputs": [], + "source": [ + "response = requests.get(url_iss_position2)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5a032e2c-abab-4c49-97d1-341d74a22c68", + "metadata": {}, + "outputs": [], + "source": [ + "response.status_code" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "395d4fed-6f83-4c96-8cf5-8523ce297e92", + "metadata": {}, + "outputs": [], + "source": [ + "response.json()" + ] + }, + { + "cell_type": "markdown", + "id": "9c50844d-2d2c-4768-8d8b-703a2cb1875c", + "metadata": {}, + "source": [ + "We can also do it by passing to the argument `params` a dictionary with the parameters in the `get` method." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cdcdd6f7-628b-4978-8656-23f72ecd5cac", + "metadata": {}, + "outputs": [], + "source": [ + "parameters = {\"units\": \"miles\"}" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "31d68523-2258-408f-ac24-0836af707cd9", + "metadata": {}, + "outputs": [], + "source": [ + "url_iss_position" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2688ca3e-e9ef-44cf-8e28-7c575f2d61be", + "metadata": {}, + "outputs": [], + "source": [ + "response = requests.get(\n", + " url=url_iss_position,\n", + " params=parameters\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "87e3afb1-2e29-4c62-8162-e8b27756be0c", + "metadata": {}, + "outputs": [], + "source": [ + "response.json()" + ] + }, + { + "cell_type": "markdown", + "id": "6aea75ed-be20-475d-98c1-86f4e9d2b1fd", + "metadata": {}, + "source": [ + "\n", + "##### 3. **Header Parameters**\n", + "\n", + "These are included in the request header and can be used for various purposes, such as authentication (e.g., sending an API key or token), content negotiation (e.g., defining the response format), or custom settings defined by the API. If the API requires authentication, you might include an `Authorization` header with your API key." + ] + }, + { + "cell_type": "markdown", + "id": "78131fa0-a203-4d64-8fc2-939ee0ebf064", + "metadata": {}, + "source": [ + "ISS API mentions that *currently there is no authentication required*. So we'll look at an example with a different API of `authentication` in `header parameters`." + ] + }, + { + "cell_type": "markdown", + "id": "69484a90-65de-44b0-87bb-b591274f27b5", + "metadata": {}, + "source": [ + "#### Request Headers" + ] + }, + { + "cell_type": "markdown", + "id": "a5782c9a-84ad-434e-b4a5-5657f17592f1", + "metadata": {}, + "source": [ + "Request headers are key-value pairs sent in an HTTP request to provide information about the request itself." + ] + }, + { + "cell_type": "markdown", + "id": "70c5dec7-204a-4c3b-a9e4-8c7bbf516b10", + "metadata": {}, + "source": [ + "Some request headers are:\n", + "\n", + "1. **Content-Type**: Specifies the media type of the resource or data. Common examples include \"application/json\" for JSON data, \"text/html\" for HTML content, and \"application/xml\" for XML data.\n", + "\n", + "3. **Authorization**: Contains credentials for authenticating the client with the server, often used with tokens or other forms of authentication.\n", + "\n", + "4. **User-Agent**: Provides information about the client (browser or other client), including its version and operating system.\n", + "\n", + "\n", + "In Python's `requests` library, you can include headers in a request by using the `headers` argument, like this:\n", + "\n", + "```python\n", + "headers = {'Content-Type': 'application/json', 'Authorization': 'Bearer YOUR_TOKEN'}\n", + "response = requests.get(url, headers=headers)\n", + "```\n" + ] + }, + { + "cell_type": "markdown", + "id": "a5311048-ef5e-4b04-9e4b-8ed700ab3076", + "metadata": {}, + "source": [ + "⚠️🚨 ¡Careful with the authentication token! ⚠️🚨 A token is a personal credential for accessing an API, through which your request quota to the API is managed. Therefore, the ideal procedure is to ensure security by storing the token as a variable in an .env file. This way, you can call the environment variable without having to publicly display the token." + ] + }, + { + "cell_type": "markdown", + "id": "b3cf22bc-d4e1-4e51-876e-f4df41b70f05", + "metadata": {}, + "source": [ + "**Example: News API**" + ] + }, + { + "cell_type": "markdown", + "id": "a6b35b6f-4cb4-4b89-af23-3c9c65fb2686", + "metadata": {}, + "source": [ + "The *News API* lets you locate articles and breaking news headlines from news sources and blogs across the web." + ] + }, + { + "cell_type": "markdown", + "id": "58123f1a-b5e4-4b20-a02d-47077a4a3635", + "metadata": {}, + "source": [ + "- Let's look at the [NewsAPI Authentication documentation](https://newsapi.org/docs/authentication). It mentions: \n", + "```\n", + "You can attach your API key to a request in one of three ways:\n", + "\n", + "- Via the apiKey querystring parameter.\n", + "- Via the X-Api-Key HTTP header.\n", + "- Via the Authorization HTTP header. Including Bearer is optional, and be sure not to base 64 encode it like you may have seen in other authentication tutorials.\n", + "We strongly recommend the either of last two so that your API key isn't visible to others in logs or via request sniffing.\n", + "\n", + "If you don't append your API key correctly, or your API key is invalid, you will receive a 401 - Unauthorized HTTP error.\n", + "```\n", + "\n", + "- Let's look at the documentation for the endpoints to see what **parameters** they accept: [NewsAPI endpoints](https://newsapi.org/docs/endpoints). We see we have the endpoint `/v2/top-headlines` – *returns breaking news headlines for countries, categories, and singular publishers. This is perfect for use with news tickers or anywhere you want to use live up-to-date news headlines.*\n", + "If we look at the [documentation of that endpoint](https://newsapi.org/docs/endpoints/top-headlines) we see we have more parameters we can add such as `country` or `category`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e2853b7d-09f0-46e3-83df-c1a1b03f6b61", + "metadata": {}, + "outputs": [], + "source": [ + "url = \"http://newsapi.org/v2/top-headlines\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ab2d561c-97fc-4750-9334-f19ce984b2ae", + "metadata": {}, + "outputs": [], + "source": [ + "response = requests.get(url)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1d44b74a-8282-4f74-bcbb-e030a2d0a26a", + "metadata": {}, + "outputs": [], + "source": [ + "response.status_code" + ] + }, + { + "cell_type": "markdown", + "id": "6c01205a-e732-44bc-9fc0-cb3cefdfc76f", + "metadata": {}, + "source": [ + "Let's try a made up key, using the apiKey querystring parameter. As we read in the documentation, this is not the recommended way." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "24d14d50-c5dc-4c20-ab4b-4f65d5e61a61", + "metadata": {}, + "outputs": [], + "source": [ + "url = f\"http://newsapi.org/v2/top-headlines?country=us&apiKey=manolonodoesnthavekey\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2d106abb-06ac-4432-b4ad-2c51fbd90e28", + "metadata": {}, + "outputs": [], + "source": [ + "url" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "304ff5dd-8d38-4641-9799-8cd7978434d0", + "metadata": {}, + "outputs": [], + "source": [ + "response = requests.get(url)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "bb9dc34b-21de-4d00-8544-fc9ea2c23f70", + "metadata": {}, + "outputs": [], + "source": [ + "response.status_code" + ] + }, + { + "cell_type": "markdown", + "id": "e829611f-a4a8-4b3d-90a2-31c1bb04fb2b", + "metadata": {}, + "source": [ + "**4xx**: client error. This means, it is our error." + ] + }, + { + "cell_type": "markdown", + "id": "71322103-4234-4105-be5d-037cb6107e26", + "metadata": {}, + "source": [ + "Lets do with a correct key. You should get your own API key through their website. Once we've saved our key, we'll send it via the X-Api-Key HTTP header." + ] + }, + { + "cell_type": "markdown", + "id": "a0ae416d-0215-4acd-9479-2ba49dab313b", + "metadata": {}, + "source": [ + "##### Saving the API Key" + ] + }, + { + "cell_type": "markdown", + "id": "2c509a90-72f3-4bba-9b29-bc4316bd3111", + "metadata": {}, + "source": [ + "Storing an API key directly in your code can expose sensitive information, especially if your code is publicly available (e.g., on a public GitHub repository). The best practice for saving and loading an API key in your code involves the following steps:\n", + "\n", + "1. **Storing the API Key**:\n", + " - **Use Environment Variables**: Store your API key in an environment variable on your system. This keeps the key out of your codebase and allows you to change it without altering your code.\n", + "\n", + "\n", + " - **Create a .env File**: If you prefer, you can create a `.env` file (just call it `.env` nothing else before the `.`) in your project directory to store the API key. Inside this file, you would have something like:\n", + " \n", + " ```\n", + " API_KEY=your-api-key-here\n", + " ```\n", + "\n", + " - **Add .env to .gitignore**: If you're using a version control system like Git, make sure to add the `.env` file to your `.gitignore` file. This prevents the `.env` file (and therefore your API key) from being uploaded to any public repositories. We'll talk about Git and .gitignore in more detail soon.\n", + "\n", + "2. **Load the Key in Your Code**: \n", + "\n", + " You can use libraries like `python-dotenv` to load the key into your code. \n", + " You would need to install `python-dotenv` first.\n", + " ```python \n", + " !pip install python-dotenv\n", + "\n", + " from dotenv import load_dotenv\n", + " import os\n", + "\n", + " load_dotenv()\n", + " api_key = os.getenv(\"API_KEY\")\n", + " ```\n", + "\n", + " \n", + " Now, `api_key` contains the value of your API key, and you can use it to authenticate your requests to the API.\n", + " \n" + ] + }, + { + "cell_type": "markdown", + "id": "98b7f771-e230-4223-a5a7-4353ee337934", + "metadata": {}, + "source": [ + "Lets do the second approach. Lets save it in an env file and save it in a variable." + ] + }, + { + "cell_type": "markdown", + "id": "3766f8a4-3e0a-4b18-8b16-d5e0d64c4dbd", + "metadata": {}, + "source": [ + "`Make sure your file is named .env and not .env.txt! Is one of the most common errors. If you need, look at the document properties to make sure it doesn't have .txt at the end even if its not showing the .txt when looking at your folder.`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "12fa7d1f-37c7-40a7-b60f-e4e264a43574", + "metadata": {}, + "outputs": [], + "source": [ + "#!pip install python-dotenv" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "65104c73-28cb-4b84-949b-013d2f0fd1b0", + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "from dotenv import load_dotenv, find_dotenv\n", + "\n", + "load_dotenv()\n", + "\n", + "my_key = os.getenv(\"API_KEY\")" + ] + }, + { + "cell_type": "markdown", + "id": "c2b8daee-8bbe-4dba-b302-4424bff542ee", + "metadata": {}, + "source": [ + "If you get an error, make sure you saved the .env file in the same directory as your jupyter notebook. You can use `os.getcwd()` to do so." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2edcbffe-e4b5-464b-8e5b-9efd3a41ed59", + "metadata": {}, + "outputs": [], + "source": [ + "os.getcwd()" + ] + }, + { + "cell_type": "markdown", + "id": "f1e87da8-be8c-4f58-94aa-354b879a7ed6", + "metadata": {}, + "source": [ + "##### Using the API Key in the Header Parameters" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c2c03dbc-e8b1-4649-9c71-18e61cc61123", + "metadata": {}, + "outputs": [], + "source": [ + "url_top_headlines = \"http://newsapi.org/v2/top-headlines\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "262b26fd-e3a8-4bc3-a7cb-38901f474817", + "metadata": {}, + "outputs": [], + "source": [ + "parameters = {\"country\": \"us\", \"category\": \"science\"}" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d3f8f790-1a58-4655-99bb-729fccd803eb", + "metadata": {}, + "outputs": [], + "source": [ + "headers = {\"X-Api-Key\": my_key}" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "85034bf8-bcc1-4e4e-bcb1-a442b2d65711", + "metadata": {}, + "outputs": [], + "source": [ + "response = requests.get(\n", + " url=url_top_headlines, \n", + " params=parameters, \n", + " headers=headers\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ad8de9aa-736e-480a-8a4d-1de84dbade07", + "metadata": {}, + "outputs": [], + "source": [ + "response.status_code" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "56058048-233a-41f5-a1da-edb8092730de", + "metadata": {}, + "outputs": [], + "source": [ + "data = response.json()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "eb07a572-50ca-41b0-b3b3-ec82d0cd3f5f", + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "data" + ] + }, + { + "cell_type": "markdown", + "id": "b6e5c398-c333-4831-b15c-230c63e7ac9c", + "metadata": {}, + "source": [ + "## APIs examples" + ] + }, + { + "cell_type": "markdown", + "id": "99d3ab93-1f0f-4c3d-9a4d-ebdf7337641b", + "metadata": {}, + "source": [ + "### NewsAPI" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f9720542-bc2e-4bd5-af7d-395f6aa420a0", + "metadata": {}, + "outputs": [], + "source": [ + "type(data) # response.json() returns a dictionary" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "541f1c98-0837-4e79-b3bc-8266486acf2f", + "metadata": {}, + "outputs": [], + "source": [ + "data.keys() # we can get the keys with method keys()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b536335a-008b-41d9-af92-6ff4466da5e9", + "metadata": {}, + "outputs": [], + "source": [ + "data.get(\"status\") # we can get a key's value by using get method" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "eee49271-cd78-42f5-972d-dfe9728ea730", + "metadata": {}, + "outputs": [], + "source": [ + "data[\"totalResults\"] # or by using dictionary notation to access values" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f26b202a-2d95-4e36-87e4-9a52f32a4be9", + "metadata": {}, + "outputs": [], + "source": [ + "# this is the same as above\n", + "data.get(\"totalResults\")" + ] + }, + { + "cell_type": "markdown", + "id": "61e9315e-c94e-41f8-8794-f255eb5220cc", + "metadata": {}, + "source": [ + "`dict[key]` raises error if the key does not exits \n", + "`dict.get(key)` does not raise an error if the key does not exist" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cc234036-e6d5-450e-9183-76e960220a7f", + "metadata": {}, + "outputs": [], + "source": [ + "# lets look at the articles\n", + "articles = data.get(\"articles\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0fac4939-7f00-4fe8-b758-4dc13e2c6b52", + "metadata": {}, + "outputs": [], + "source": [ + "type(articles) # lets see what type it is to know how to access it" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5988fdf2-5d2a-4236-927f-e8abd9c0fe93", + "metadata": {}, + "outputs": [], + "source": [ + "len(articles)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2353c5e3-c200-40b5-9529-a4f531dc4d77", + "metadata": {}, + "outputs": [], + "source": [ + "articles[0] # since articles is a list, we can access its elements by the index" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a98e38bb-4364-4d3a-817d-6993049f9408", + "metadata": {}, + "outputs": [], + "source": [ + "type(articles[0]) # each element inside the list is a dictionary" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "18eb15ba-18e3-49c5-997d-050815814e21", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "[art[\"title\"] for art in articles] # lets get all the titles using list comprehension" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e8422609-6ccf-4ddc-b044-b5a951f49ab6", + "metadata": {}, + "outputs": [], + "source": [ + "df = pd.DataFrame(articles) # lets create a dataframe with the articles" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9069dc15-b0e6-4db9-acba-f1b5c0c37a5b", + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "df.head()" + ] + }, + { + "cell_type": "markdown", + "id": "2dc2bab1-1cdc-4a3f-9956-b5d5eb832103", + "metadata": {}, + "source": [ + "#### 💡 Check for understanding" + ] + }, + { + "cell_type": "markdown", + "id": "3e21fbb4-6e2d-43ef-a61e-38da7eedc5fe", + "metadata": {}, + "source": [ + "1. What data type is the column *source*?\n", + "2. Since the column *source* is not very useful as it is, create a column called *name* that contains only the name inside the column *source*.\n", + "3. How many times each unique name appears in the \"name\" column?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f906ed21-be98-436b-816b-c73f5dd5c066", + "metadata": {}, + "outputs": [], + "source": [ + "# Your answer goes here" + ] + }, + { + "cell_type": "markdown", + "id": "48ca730a-10fe-4a69-ad11-01bf8759a938", + "metadata": {}, + "source": [ + "### Pokemon API" + ] + }, + { + "cell_type": "markdown", + "id": "be8f6d65-359d-408b-aeb4-64379614377f", + "metadata": {}, + "source": [ + "[PokeAPI](https://pokeapi.co/) is a full RESTful API linked to an extensive database detailing everything about the Pokémon main game series.\n", + "\n", + "In the documentation we see that we can get lot's of data. Let's look at the endpoint *pokemon*: ```https://pokeapi.co/api/v2/pokemon/{id or name}/``` " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b59992ec-a9de-4c14-88ef-177df49393a4", + "metadata": {}, + "outputs": [], + "source": [ + "res=requests.get('https://pokeapi.co/api/v2/pokemon/25')\n", + "res.status_code" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b180faa9-b8b2-4ec5-804e-d46941f6132c", + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "data=res.json()\n", + "data" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "faa82a7d-35f8-441c-9c65-c4cf34c8ba99", + "metadata": {}, + "outputs": [], + "source": [ + "type(data)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a4745456-3fa4-4511-a7be-c6eea20db9df", + "metadata": {}, + "outputs": [], + "source": [ + "data.keys()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9fa671b3-9a78-49e8-bf57-0781a1fa6b70", + "metadata": {}, + "outputs": [], + "source": [ + "# Lets look at the following attributes\n", + "print(data[\"name\"])\n", + "print(data[\"weight\"])\n", + "print(data[\"sprites\"])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b4ceecb0-9e17-4cfe-a09e-67ae3026652d", + "metadata": {}, + "outputs": [], + "source": [ + "# Lets get the pokemon image from sprites - front_default.\n", + "# Remember we can read the documentation to understand how to get all the information\n", + "data['sprites']['front_default']" + ] + }, + { + "cell_type": "markdown", + "id": "5bdb9958-a049-42b2-8c5c-475c7b90601d", + "metadata": {}, + "source": [ + "We will define a function, `get_pokemon`, that takes an ID number as input. This function fetches details about a Pokémon from the PokeAPI using the given ID, extracts its name, weight, and front-facing image, and then returns this data in a dictionary format. If the ID is invalid or there's an issue with the request, a ValueError is raised." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "04981f9a-157d-49ae-9e5d-0b382d5ab2fd", + "metadata": {}, + "outputs": [], + "source": [ + "def get_pokemon(id_number):\n", + " res=requests.get('https://pokeapi.co/api/v2/pokemon/{}'.format(id_number))\n", + " \n", + " if(res.status_code==200):\n", + " data = res.json()\n", + " return {\"id\": id_number,'name':data['name'], \"weight\":data[\"weight\"],'image':data['sprites']['front_default']}\n", + " else:\n", + " raise ValueError(\"Cannot get pokemon, id does not exist\")\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "96dbff41-1e63-40ee-b409-aed7cce7e7a7", + "metadata": {}, + "outputs": [], + "source": [ + "get_pokemon(25) # Lets use the function for pokemon id 25" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ebf5de4c-1150-4cab-812a-7b6d9f1e807c", + "metadata": {}, + "outputs": [], + "source": [ + "[get_pokemon(i+1) for i in range(5)] # Lets use list comprehension to gather information for the first 5 pokemons" + ] + }, + { + "cell_type": "markdown", + "id": "5d419697-cb8a-4737-8b0b-3a742615f16a", + "metadata": {}, + "source": [ + "We will now define a function, `print_pokemon`, that takes in a Pokémon dictionary and displays its ID, name, weight, and its image. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "da875adb-e2d1-4e59-8b2e-ce3e00da5f59", + "metadata": {}, + "outputs": [], + "source": [ + "from IPython.display import Image\n", + "\n", + "def print_pokemon(poke):\n", + " print(str(poke[\"id\"])+\" \" +poke[\"name\"]+ \" \"+str(poke[\"weight\"]) + \" kg\")\n", + " display(Image(url=poke['image']))\n", + " " + ] + }, + { + "cell_type": "markdown", + "id": "d284e0ef-4d49-42a3-bc50-c1a859a7e41a", + "metadata": {}, + "source": [ + "After defining this function, we will retrieve and display details for the first 10 Pokémon by iterating through their IDs and using both the `get_pokemon` and `print_pokemon` functions. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9e8a778b-bf09-4351-b9d6-6adc759566e5", + "metadata": {}, + "outputs": [], + "source": [ + "[print_pokemon(get_pokemon(i+1)) for i in range(10)]" + ] + }, + { + "cell_type": "markdown", + "id": "8ee586a7-a55f-4c14-b3cf-55745308fe84", + "metadata": {}, + "source": [ + "#### json_normalize()" + ] + }, + { + "cell_type": "markdown", + "id": "8ca157cb-530d-49de-9286-f072b3e295e4", + "metadata": {}, + "source": [ + "Want the data as a DataFrame instead of a dictionary?" + ] + }, + { + "cell_type": "markdown", + "id": "9672696a-3e67-4af5-af41-10270e1209b1", + "metadata": {}, + "source": [ + "The following line creates a DataFrame from the items of the dictionary data. Each key-value pair will be treated as a row, with two columns: the first for the key and the second for the value." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5455a1bd-a158-4111-9841-750add5b2708", + "metadata": {}, + "outputs": [], + "source": [ + "df = pd.DataFrame(data.items())\n", + "df" + ] + }, + { + "cell_type": "markdown", + "id": "175094a5-2fcd-4956-8fb4-e0d59f701ffe", + "metadata": {}, + "source": [ + "That format is still very hard to work with. Lets use `json_normalize` instead. `json_normalize` is used to normalize semi-structured JSON data into a flat table (DataFrame). It's particularly useful when dealing with nested dictionaries or lists inside JSON." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "def72654-66aa-41cd-900b-d55fae54894d", + "metadata": {}, + "outputs": [], + "source": [ + "df = pd.json_normalize(data)\n", + "df" + ] + }, + { + "cell_type": "markdown", + "id": "a879ab4b-1a2e-42f7-8a4b-4bf2c4e8687c", + "metadata": {}, + "source": [ + "A lot better, even though we still have nested dictionaries or lists inside the dataframe.\n", + "\n", + "Let's see again the difference between creating a dataframe directly or using `json_normalize` by using `data[\"abilities\"]`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "96389cbb-ee4f-4495-ae45-91de38ccc1fa", + "metadata": {}, + "outputs": [], + "source": [ + "pd.DataFrame(data[\"abilities\"])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "04997d49-0438-40e5-b0cf-6244405aa55e", + "metadata": {}, + "outputs": [], + "source": [ + "pd.json_normalize(data[\"abilities\"])" + ] + }, + { + "cell_type": "markdown", + "id": "4680fda5-270d-45e9-ab0f-abebcfe09088", + "metadata": {}, + "source": [ + "### Coincap API" + ] + }, + { + "cell_type": "markdown", + "id": "d43ce6fe-baf5-42da-9e40-d31f2c8e3b27", + "metadata": {}, + "source": [ + "CoinCap is a useful tool for real-time pricing and market activity for over 1,000 cryptocurrencies. Check the documentation at https://docs.coincap.io/" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "bdfda195-9de8-4e2c-8b01-63161db18060", + "metadata": {}, + "outputs": [], + "source": [ + "url = \"http://api.coincap.io/v2/assets\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8cdd990d-cf9b-4bdc-bbd9-cb038685efd5", + "metadata": {}, + "outputs": [], + "source": [ + "response = requests.get(url)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1ea53c95-0868-4440-80ba-13f492e4da6f", + "metadata": {}, + "outputs": [], + "source": [ + "print(response)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8873ba49-0849-40f9-903d-9e49451de9ee", + "metadata": {}, + "outputs": [], + "source": [ + "res = response.json()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "17605693-433c-4581-813a-136209c382e3", + "metadata": {}, + "outputs": [], + "source": [ + "res # it returns a dictionary with all the data" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b56de457-e978-468c-b752-97000ce71baa", + "metadata": {}, + "outputs": [], + "source": [ + "# Again, let's try to have it as a dataframe\n", + "data = pd.DataFrame(res[\"data\"])\n", + "data.head()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2e264ce5-5f90-49f2-8888-78b64808a3ba", + "metadata": {}, + "outputs": [], + "source": [ + "# Lets see if the result would differ in this case using json_normalize\n", + "pd.json_normalize(res[\"data\"])" + ] + }, + { + "cell_type": "markdown", + "id": "52dfaadb-dd5f-4b7d-a832-c101e4d7c80a", + "metadata": {}, + "source": [ + "### Api Jokes\n", + "\n", + "JokeAPI is a REST API that serves uniformly and well formatted jokes.\n", + "It can be used without any API token, membership, registration or payment.\n", + "It supports a variety of filters that can be applied to get just the right jokes you need.\n", + "Check the documentation here: https://jokeapi.dev/ " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e897b2d9-088c-47fd-9d58-fefec4332150", + "metadata": {}, + "outputs": [], + "source": [ + "url_random_joke = \"https://v2.jokeapi.dev/joke/dark?amount=3\"\n", + "request = requests.get(url_random_joke).json()\n", + "request" + ] + }, + { + "cell_type": "markdown", + "id": "bd07c567-1d95-428a-8ac9-9f9909457c49", + "metadata": { + "lang": "en" + }, + "source": [ + "### 💡 Check for understanding\n" + ] + }, + { + "cell_type": "markdown", + "id": "08d0de37-f0e9-4249-a1f2-3e9c091f83da", + "metadata": {}, + "source": [ + "Choose one API from the [Public APIs list](https://github.com/public-apis/public-apis). Attempt to call your selected API, either with or without parameters, and retrieve some valuable information. Document your findings." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "c8f7e8a5-31b7-4444-a80f-8a94dd85e2eb", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/Users/samvesali/Library/Python/3.9/lib/python/site-packages/urllib3/__init__.py:35: NotOpenSSLWarning: urllib3 v2 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'LibreSSL 2.8.3'. See: https://github.com/urllib3/urllib3/issues/3020\n", + " warnings.warn(\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'request': {'type': 'City', 'query': 'Amsterdam, Netherlands', 'language': 'en', 'unit': 'm'}, 'location': {'name': 'Amsterdam', 'country': 'Netherlands', 'region': 'North Holland', 'lat': '52.374', 'lon': '4.890', 'timezone_id': 'Europe/Amsterdam', 'localtime': '2026-04-07 17:33', 'localtime_epoch': 1775583180, 'utc_offset': '2.0'}, 'current': {'observation_time': '03:33 PM', 'temperature': 16, 'weather_code': 113, 'weather_icons': ['https://cdn.worldweatheronline.com/images/wsymbols01_png_64/wsymbol_0001_sunny.png'], 'weather_descriptions': ['Sunny'], 'astro': {'sunrise': '07:01 AM', 'sunset': '08:25 PM', 'moonrise': '02:09 AM', 'moonset': '08:39 AM', 'moon_phase': 'Waning Gibbous', 'moon_illumination': 79}, 'air_quality': {'co': '187.85', 'no2': '7.95', 'o3': '96', 'so2': '1.35', 'pm2_5': '8.95', 'pm10': '13.05', 'us-epa-index': '1', 'gb-defra-index': '1'}, 'wind_speed': 15, 'wind_degree': 87, 'wind_dir': 'E', 'pressure': 1026, 'precip': 0, 'humidity': 36, 'cloudcover': 0, 'feelslike': 16, 'uv_index': 2, 'visibility': 10, 'is_day': 'yes'}}\n" + ] + } + ], + "source": [ + "#I explored the Weatherstack API through its documentation and request format. The current endpoint can be used to retrieve live weather data by sending an access_key and a query parameter.\n", + "#A successful request returns useful information such as temperature, humidity, wind speed, pressure, weather description, UV index, and visibility.\n", + "import requests\n", + "import os\n", + "from dotenv import load_dotenv, find_dotenv\n", + "\n", + "load_dotenv()\n", + "\n", + "my_key = os.getenv(\"API_KEY\")\n", + "\n", + "url = \"https://api.weatherstack.com/current\"\n", + "params = {\n", + " \"access_key\": my_key,\n", + " \"query\": \"Amsterdam\",\n", + " \"units\": \"m\"\n", + "}\n", + "\n", + "response = requests.get(url, params=params)\n", + "data = response.json()\n", + "\n", + "print(data)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f5ee5eb0-0126-49c4-922a-ea5a12330e06", + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "10d38405-f1bd-464f-a319-4a5f29a4cdc2", + "metadata": {}, + "source": [ + "## API Wrappers\n", + "\n", + "A Python wrapper is a Python library or module that provides a more convenient or more \"Pythonic\" interface to another software component, such as a library in another language, a system tool, or a web API. It \"wraps\" the functionality of that component in a way that abstracts away its complexities and makes it easier to use in a Python context. 🙌🏻\n", + "\n", + "One example is the `tweepy` library, which makes obtaining data from Twitter's API relatively straightforward." + ] + }, + { + "cell_type": "markdown", + "id": "796f214d-01cc-46d1-a020-d4bb8d256799", + "metadata": {}, + "source": [ + "## Summary\n", + "- Import the `requests` library.\n", + "- Store the necessary credentials, such as API key or OAuth tokens, if the API requires them.\n", + "- Execute a `request.get` to the desired API endpoint (the API's documentation will provide details on available endpoints).\n", + "- The API returns a JSON response.\n", + "- This JSON can be converted into a dataframe, or you can further explore its elements (typically a list of dictionaries)." + ] + }, + { + "cell_type": "markdown", + "id": "092ff771-4d9a-4d84-98c6-db652fe4ff72", + "metadata": {}, + "source": [ + "## Glossary" + ] + }, + { + "cell_type": "markdown", + "id": "32ca2f95-7c09-4481-8897-0b0533dba4c9", + "metadata": {}, + "source": [ + "* DNS: domain name system.\n", + "* HTTP: is the protocol used to transfer data over the web.\n", + "* API: application programming interface.\n", + "* REST: series of rules, architectural style." + ] + }, + { + "cell_type": "markdown", + "id": "12ba250f-82d7-450b-9038-0f32013dbaf5", + "metadata": {}, + "source": [ + "* `requests`: Python library for interacting with APIs.\n", + "* URL: server name you want to ask information for.\n", + "* endpoint: server service you want to ask information for.\n", + "* parameters: extra parameters to your query.\n", + "* headers: metadata, invisible information." + ] + }, + { + "cell_type": "markdown", + "id": "df2580ce-5fa4-4716-a1f1-3d6c8edd9673", + "metadata": {}, + "source": [ + "## Further materials" + ] + }, + { + "cell_type": "markdown", + "id": "cdae6b8f-bed4-47f8-b9ae-3d8e24e8aba6", + "metadata": {}, + "source": [ + "[5 Simple-To-Use APIs for Beginners](https://dev.to/alanconstantino/5-simple-to-use-apis-for-beginners-2e0n)" + ] + }, + { + "cell_type": "markdown", + "id": "3e747420-7bb6-4ea7-a5d9-e05c90fafd32", + "metadata": {}, + "source": [ + "[RapidAPI](https://rapidapi.com/category/Sports): access thousands of APIs." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.6" + }, + "toc": { + "base_numbering": 1, + "nav_menu": {}, + "number_sections": true, + "sideBar": true, + "skip_h1_title": false, + "title_cell": "Table of Contents", + "title_sidebar": "Contents", + "toc_cell": true, + "toc_position": {}, + "toc_section_display": true, + "toc_window_display": false + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/lab-apis.ipynb b/lab-apis.ipynb index e923554..283e314 100644 --- a/lab-apis.ipynb +++ b/lab-apis.ipynb @@ -558,7 +558,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.9.13" + "version": "3.9.6" } }, "nbformat": 4,