Update Gemini with new SDK #121

andrewphilipsmith · 2025-04-23T15:49:15Z

Purpose

Gemini have introduced a new Python SDK. This PR is to migrate from the deprecated google-generativeai package to the newer google-genai package.

See references here https://ai.google.dev/api?lang=python
Migration Guide https://ai.google.dev/gemini-api/docs/migrate

Notes / Questions / Todos:

(In no particular order)

vertexai is dependent on the deprecated google-generativeai package. Do we need to upgrade this too?
Update Auth https://ai.google.dev/gemini-api/docs/migrate#authenticate
async - handled differently in new SDK https://ai.google.dev/gemini-api/docs/migrate#async Check our implementation still makes sense.
Replace GenerativeModel with a client instance, created once, earlier in the pipeline (including updating tests as required).
In test_gemini.py test_gemini_obtain_model_inputs(): Have updated the tests to assume that the methods will return a slightly different tuple of values. For now, assume that the most sensible thing for the _obtain_model_inputs to return here is the AsyncClient instance. It may be that returning nothing is the sensible thing to do, in which case, different parts of the test will need to be updated.
For accuracy of the comments in the test test_gemini_query_string, check if there is a difference in the return type of google.genai.client.aio.models.generate_content and google.genai.client.models.generate_content (both return a genai.types.GenerateContentResponse object).
The existing code supports "Per model API keys". This seems to go against the grain of how the Client authentication works - should this be maintained?
- Probably not required, but easy enough to maintain, so I've left it in for now.
SafetySetting (https://googleapis.github.io/python-genai/genai.html#genai.types.SafetySetting), includes a method parameter. We don't appear to be using this, or any equivalent, in the current code. Do we need to add this?
- Not required
Fix test tests/apis/gemini/test_gemini_chat_input.py::test_gemini_query_chat_no_env_var This test passes when executed alone, but fails when executed with all tests. This is probably due to the environment variable being monkeypatched somewhere without being reset / properly scoped.

andrewphilipsmith · 2025-04-23T15:50:30Z

The first commit updates the relevant tests to use the new SDK, without updating the actual implementation. There will be many failing tests for now.

Copilot

Pull Request Overview

This PR migrates the Gemini API integration from the deprecated google-generativeai package to the newer google-genai package by updating client instantiation, safety settings handling, and test expectations.

Updated tests and patches to reflect the new SDK classes and method signatures.
Refactored GeminiAPI to use Client and GenerateContentConfig from google-genai and updated dependency management in pyproject.toml.

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

File	Description
tests/apis/gemini/*.py	Updated test patches and assertions to reflect new SDK changes
src/prompto/apis/gemini/gemini.py	Refactored client instantiation and method interfaces for the new SDK
pyproject.toml	Updated dependency versions and deprecated package removal notices

Copilot · 2025-04-24T18:32:04Z

src/prompto/apis/gemini/gemini.py

-                HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_NONE,
-                HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_NONE,
-            }
+            SafetySetting()


The standalone call to SafetySetting() in the 'none' safety_filter branch is unnecessary and may confuse readers. Please remove this extraneous call.

Suggested change

SafetySetting()

Copilot · 2025-04-24T18:32:04Z

src/prompto/apis/gemini/gemini.py

@@ -455,13 +543,13 @@ async def _query_history(self, prompt_dict: dict, index: int | str) -> dict:
        i.e. multi-turn chat with history.
        """
        if prompt_dict["prompt"][0]["role"] == "system":
-            prompt, model_name, model, safety_settings, generation_config = (
+            prompt, model_name, client, safety_settings, generation_config = (


In _query_history, the tuple unpacking does not match the updated return type from _obtain_model_inputs (Client and GenerateContentConfig). Update the unpacking (e.g. unpack as (prompt, model_name, client, generation_config, _)) and remove or adjust the usage of the safety_settings parameter in the subsequent send_message_async call.

Suggested change

prompt, model_name, client, safety_settings, generation_config = (

prompt, model_name, client, generation_config, _ = (

…prompto into update-gemini-sdk

Copilot

Pull Request Overview

This PR migrates the Gemini integration from the deprecated google-generativeai package to the new google-genai SDK, updating imports, client initialization, safety settings, and related test cases. Key changes include:

Updating all API calls and client interactions to use google-genai (e.g., AsyncClient, GenerateContentConfig) instead of the deprecated package.
Adjusting test mocks and expected responses for the new SDK APIs and safety settings.
Modifying utility functions (e.g., parsing multimedia parts and converting history dictionaries) to accept the new client parameter and structure.

Reviewed Changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

File	Description
tests/apis/gemini/*.py	Updated tests with new client imports and proper mocks for async calls
src/prompto/apis/gemini/gemini_utils.py	Modified utility functions signature to include a client parameter
src/prompto/apis/gemini/gemini.py	Migrated API client usage and safety settings configuration to new SDK
pyproject.toml	Updated dependency from google-generativeai to google-genai

Copilot · 2025-04-29T16:28:27Z

src/prompto/apis/gemini/gemini_utils.py

@@ -27,7 +28,9 @@ def parse_parts_value(part: dict | str, media_folder: str) -> any:
        Multimedia data object
    """
    if isinstance(part, str):
-        return part
+        # return part
+        print(f"Part is a string: {part}")


Remove the debug print statement used when handling string parts, as it may leak unwanted output in production.

Suggested change

print(f"Part is a string: {part}")

Copilot · 2025-04-29T16:28:28Z

src/prompto/apis/gemini/gemini.py

+            # No need to send the generation_config again, as it is no different
+            # from the one used to create the chat
+            last_msg = prompt[-1]
+            print(f"whole prompt: {prompt}")


Remove or replace debug print statements in the _query_history method before production, to ensure cleaner logging and performance.

codecov-commenter · 2025-04-29T16:34:22Z

Codecov Report

Attention: Patch coverage is 92.70833% with 7 lines in your changes missing coverage. Please review.

Project coverage is 67.24%. Comparing base (227ed0d) to head (6c975bc).

Files with missing lines	Patch %	Lines
src/prompto/upload_media.py	56.25%	7 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #121      +/-   ##
==========================================
+ Coverage   65.09%   67.24%   +2.15%     
==========================================
  Files          44       44              
  Lines        2679     2690      +11     
==========================================
+ Hits         1744     1809      +65     
+ Misses        935      881      -54

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot

Pull Request Overview

This PR migrates the Gemini API integration from the deprecated google-generativeai package to the new google-genai package. Key changes include updating API call methods to use AsyncClient and related async methods, revising safety settings to use new types (SafetySetting) and updating tests and dependency definitions.

Reviewed Changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.

File	Description
tests/apis/gemini/*.py	Updated tests to patch new client methods and use updated types, ensuring compatibility with the new SDK.
src/prompto/apis/gemini/*.py	Replaced usage of GenerativeModel with Client instances and updated safety settings and API calls accordingly.
src/prompto/upload_media.py	Updated media upload functions to use new asynchronous SDK methods and adjusted settings creation.
pyproject.toml	Modified dependency versions and extras to reflect changes in packages and SDK versions.

Copilot · 2025-05-06T21:13:37Z

src/prompto/upload_media.py

+    For now, we just create a temporary directory for the data folder.
+    """
+    # TODO: A better solution would be to create an option in the
+    # Settings constructor to not create the directories.
+    # But for now we'll just pass it a temporary directory.
+    import tempfile
+
+    with tempfile.TemporaryDirectory() as temp_dir:
+        data_folder = os.path.join(temp_dir, "data")
+        os.makedirs(data_folder, exist_ok=True)
+        dummy_settings = Settings(data_folder=data_folder)


Creating a Settings instance inside a TemporaryDirectory context may lead to issues since the temporary directory will be deleted after the with block exits. Consider creating a persistent directory or allowing Settings to manage its own directory lifecycle.

Suggested change

For now, we just create a temporary directory for the data folder.

"""

# TODO: A better solution would be to create an option in the

# Settings constructor to not create the directories.

# But for now we'll just pass it a temporary directory.

import tempfile

with tempfile.TemporaryDirectory() as temp_dir:

data_folder = os.path.join(temp_dir, "data")

os.makedirs(data_folder, exist_ok=True)

dummy_settings = Settings(data_folder=data_folder)

A persistent directory is created for the data folder.

"""

# Define a persistent directory for the data folder

data_folder = os.path.expanduser("~/.prompto/data")

os.makedirs(data_folder, exist_ok=True)

# Create the Settings instance with the persistent data folder

dummy_settings = Settings(data_folder=data_folder)

I have not fixed this in this pull request, but created a separate issue to track it: #122

WIP: update tests with new SDK

d9d6c65

More updates to tests with new SDK

5de42f7

andrewphilipsmith marked this pull request as ready for review April 24, 2025 17:19

Update GeminiAPI class string & chat methods with new SDK

283a564

andrewphilipsmith requested a review from Copilot April 24, 2025 18:31

Copilot AI reviewed Apr 24, 2025

View reviewed changes

andrewphilipsmith added 5 commits April 25, 2025 21:04

Update GeminiAPI class string & chat methods with new SDK

ef96645

Merge branch 'update-gemini-sdk' of github.com:alan-turing-institute/…

c39ea9b

…prompto into update-gemini-sdk

Update gemini_media.py

32cf50d

fix bug in '_get_client' method

c80273f

fix bug in '_get_client' method

60ec15b

andrewphilipsmith requested a review from Copilot April 29, 2025 16:27

Copilot AI reviewed Apr 29, 2025

View reviewed changes

andrewphilipsmith added 7 commits April 30, 2025 12:09

linting and copilot review changes

d561e07

WIP: migrate gemini media functions

7a2af56

tweak dependency versions

6e66327

Update pyproject.toml

487716d

WIP: attempt tighter deps version restrictions

9fa2ad1

Sorted dependencies in pyproject.toml

68b5de1

WIP: tidy up and linting

0ef2373

andrewphilipsmith requested a review from Copilot May 6, 2025 21:13

Copilot AI reviewed May 6, 2025

View reviewed changes

Tweak to comment

6c975bc

andrewphilipsmith changed the title ~~WIP: update Gemini with new SDK~~ Update Gemini with new SDK May 15, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Update Gemini with new SDK #121

Update Gemini with new SDK #121

Uh oh!

andrewphilipsmith commented Apr 23, 2025 •

edited

Loading

Uh oh!

andrewphilipsmith commented Apr 23, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 24, 2025

Uh oh!

Copilot AI Apr 24, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 29, 2025

Uh oh!

Copilot AI Apr 29, 2025

Uh oh!

codecov-commenter commented Apr 29, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI May 6, 2025

Uh oh!

andrewphilipsmith May 7, 2025

Uh oh!

Uh oh!

	prompt, model_name, client, safety_settings, generation_config = (
	prompt, model_name, client, generation_config, _ = (

Update Gemini with new SDK #121

Are you sure you want to change the base?

Update Gemini with new SDK #121

Uh oh!

Conversation

andrewphilipsmith commented Apr 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Notes / Questions / Todos:

Uh oh!

andrewphilipsmith commented Apr 23, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Apr 24, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 24, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Apr 29, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 29, 2025

Choose a reason for hiding this comment

Uh oh!

codecov-commenter commented Apr 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI May 6, 2025

Choose a reason for hiding this comment

Uh oh!

andrewphilipsmith May 7, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

andrewphilipsmith commented Apr 23, 2025 •

edited

Loading

codecov-commenter commented Apr 29, 2025 •

edited

Loading