Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,14 @@ jobs:
- name: Checkout repository
uses: actions/checkout@v6

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v6
- name: Set up uv with Python ${{ matrix.python-version }}
uses: astral-sh/setup-uv@v7
with:
python-version: ${{ matrix.python-version }}

activate-environment: true

- name: Install dependencies
run: pip install -e .[test]
run: uv pip install -e '.[test]'

- name: Run tests
run: pytest tests --ignore-glob='tests/test_docs_*.py' -k "not example"
Expand Down
5 changes: 3 additions & 2 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,12 +30,13 @@ jobs:
echo "Release must run against a tag, got '${{ github.ref }}'" >&2
exit 1

- uses: actions/setup-python@v6
- uses: astral-sh/setup-uv@v7
with:
python-version: ${{ matrix.python-version }}
activate-environment: true

- name: Install dependencies
run: pip install -e .[test]
run: uv pip install -e '.[test]'

- name: Run tests
run: pytest --ignore=tests/test_docs_publishing.py -k "not example"
Expand Down
8 changes: 5 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,14 +31,16 @@ The commands below use `python` from that environment.

### Checks that need no API key

These tests cover response parsing, image uploads and extraction with simulated responses, timeouts, exceptions, documentation syntax, and the example runner:
These tests cover response parsing, image uploads and extraction with simulated responses, timeouts, exceptions, the async client, documentation syntax, and the example runner:

```sh
python -m pytest tests/test_output_formats.py tests/test_image_upload.py tests/test_extract.py tests/test_timeout.py tests/test_exceptions.py tests/test_docs_examples.py tests/test_docs_example_runner.py -k 'not live' -q
python -m pytest tests/test_output_formats.py tests/test_image_upload.py tests/test_extract.py tests/test_timeout.py tests/test_exceptions.py tests/test_async_client.py tests/test_docs_examples.py tests/test_docs_example_runner.py -k 'not live' -q
```

The file list limits this command to tests that make no external requests. Applying `-k 'not live'` to the entire suite is not enough to run offline: some SDK integration tests do not have `live` in their names.

The async client tests need Python 3.11 or newer, where the `test` extra installs httpx2. On older versions, pytest skips them.

### SDK integration tests

Set `API_KEY` in your shell or editor's run configuration before running integration tests. Documentation examples also accept `SERPAPI_KEY`, but the shared SDK fixtures require `API_KEY`. If you already have `SERPAPI_KEY` set, copy it to `API_KEY` in the same shell:
Expand Down Expand Up @@ -67,7 +69,7 @@ Both commands make real SerpApi requests. The full suite can use more searches t

The tests discover Python code blocks in the README and all Markdown pages under `docs/`. They run each page as a separate script against SerpApi, including the multiprocessing example's worker processes. The Lens upload examples use the repository's PNG icon as `image.png`. Each page has a five-minute limit, and HTTP errors or JSON responses containing an `error` fail the check even if the example catches the exception.

With `SERPAPI_KEY` or `API_KEY` set in your environment, run:
With `SERPAPI_KEY` or `API_KEY` set in your environment, run the following command with Python 3.11 or newer, because the README and the async client guide use `serpapi.AsyncClient`:

```sh
python -m pytest tests/test_docs_examples.py --require-docs-key -q
Expand Down
44 changes: 43 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,14 @@ To install the `serpapi` package, simply run the following command inside your P
pip install serpapi
```

To use `serpapi.AsyncClient` with `asyncio`, install the `async` extra instead. It requires Python 3.11 or later:

```bash
pip install "serpapi[async]"
```

Without this extra, creating an `AsyncClient` raises an `ImportError` that includes the installation command. The synchronous `Client` works without it.

## Quickstart

[Create a SerpApi account](https://serpapi.com/users/sign_up?plan=free) to get your API key, then store it in an environment variable:
Expand Down Expand Up @@ -82,6 +90,7 @@ A successful HTTP response can still contain an `error` field, such as when a se
## Features

- **Asynchronous searches** for submitting jobs and retrieving completed results from the Search Archive API.
- **asyncio support** with `serpapi.AsyncClient` for running searches at the same time with `await`.
- **Persistent connections and connection pooling** for reusing HTTP connections across searches.
- Search results as a dict-like **SerpResults** object by default, token-efficient **Markdown** with `"output": "md"`, or raw **search-engine HTML** with `"output": "html"`.
- SDK methods for the [Image API](https://serpapi.com/image-api), [Location API](https://serpapi.com/locations-api), [Search Archive API](https://serpapi.com/search-archive-api), and [Account API](https://serpapi.com/account-api).
Expand Down Expand Up @@ -151,7 +160,7 @@ Learn more about [SerpApi Markdown Output](https://serpapi.com/markdown-output).

Searches wait for their results by default. Use `{"async": True}` to submit a search job on SerpApi and retrieve its results later from the Search Archive API.

SerpApi's `async` parameter is separate from Python `asyncio`. The SDK methods are synchronous: each call blocks until its HTTP response arrives and cannot be used with `await`.
SerpApi's `async` parameter is separate from Python `asyncio`. `serpapi.Client` methods are synchronous: each call blocks until its HTTP response arrives. To use `await`, see [Async client](#async-client).

> **Note:** `async` is a reserved keyword in Python 3.7+, so pass it inside the parameters dictionary (`{"async": True}`) rather than as a keyword argument.

Expand All @@ -172,6 +181,39 @@ print(results["search_metadata"]["status"])

If the search is still processing, retrieve it again later. See the [async search guide](https://serpapi-python.readthedocs.io/en/latest/user_guide/async-search-archive.html) for polling with HTTP timeouts and a deadline.

## Async client

`serpapi.AsyncClient` has the same SDK methods and API parameters as `serpapi.Client`, for programs that use `asyncio`, with some [transport differences](https://serpapi-python.readthedocs.io/en/latest/user_guide/async-client.html#differences-from-client). Call its request methods with `await`. It requires Python 3.11 or later and the `async` extra:

```bash
pip install "serpapi[async]"
```

Run several searches at the same time with `asyncio.gather()`:

```python
import asyncio
import os

import serpapi


async def main():
async with serpapi.AsyncClient(api_key=os.environ["SERPAPI_KEY"], timeout=20) as client:
google, bing = await asyncio.gather(
client.search({"engine": "google", "q": "coffee"}),
client.search({"engine": "bing", "q": "coffee"}),
)

print("Google:", len(google.get("organic_results", [])), "results")
print("Bing:", len(bing.get("organic_results", [])), "results")


asyncio.run(main())
```

`asyncio.gather()` returns the results in the order of the searches. `async with` closes the client's connections when the block ends. To fetch more pages, use `await results.next_page()` or `async for page in results.yield_pages()`. See [Migrating from Sync to Async](https://serpapi-python.readthedocs.io/en/latest/user_guide/async-client.html#migrating-from-sync-to-async) for a before-and-after example, or the [async client guide](https://serpapi-python.readthedocs.io/en/latest/user_guide/async-client.html) for limiting concurrent searches.

## Examples

Here are some examples of our most popular APIs. You can find the full list of supported engines and parameters in our [documentation](https://serpapi.com/search-engine-apis).
Expand Down
2 changes: 1 addition & 1 deletion docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
autodoc_default_options = {
"members": True,
"imported-members": True,
"exclude-members": "HTTPClient, from_http_response",
"exclude-members": "HTTPClient, AsyncHTTPClient, from_http_response",
}

html_theme = "sphinx_rtd_theme"
Expand Down
4 changes: 3 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ uv add serpapi

::::

The package requires Python 3.6 or newer.
The package requires Python 3.6 or newer. To send requests with `asyncio` and `await`, install `serpapi[async]` on Python 3.11 or newer and use [`serpapi.AsyncClient`](user_guide/async-client.md).

## First Request

Expand Down Expand Up @@ -95,6 +95,7 @@ Use the same search parameter names as the [SerpApi API documentation](https://s
- [Parameters and Engines](user_guide/parameters-and-engines.md) explains search parameters and how to test them in the [SerpApi Playground](https://serpapi.com/playground).
- For scripts that collect results, see [pagination](user_guide/pagination.md), [timeouts and errors](user_guide/errors-and-timeouts.md), [request options](user_guide/request-options.md), and [account and locations](user_guide/account-and-locations.md).
- For searches you retrieve later, selected response fields, and data retention settings, see [Async Search Archive](user_guide/async-search-archive.md), [JSON Restrictor](user_guide/json-restrictor.md), and [Zero Trace](user_guide/zero-trace.md).
- [Async Client](user_guide/async-client.md) covers `serpapi.AsyncClient`, which runs searches at the same time with `asyncio` and `await`.
- [Examples](examples/index.md) includes searches for web pages, AI answers, local businesses, products, travel, finance, trends, jobs, media, apps, and research.

```{toctree}
Expand Down Expand Up @@ -134,6 +135,7 @@ user_guide/migrating-from-google-search-results
:caption: Advanced Usage

user_guide/async-search-archive
user_guide/async-client
user_guide/threading
user_guide/multiprocessing
user_guide/zero-trace
Expand Down
4 changes: 4 additions & 0 deletions docs/reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,7 @@ Look up the arguments, return values, and exceptions for the functions and class
.. automodule:: serpapi
:member-order: groupwise
```

```{eval-rst}
.. automethod:: serpapi.AsyncClient.aclose
```
Loading
Loading