Skip to content

Fix Ollama API compatibility for newer versions#443

Open
Samay10 wants to merge 1 commit intogoogle:mainfrom
Samay10:fix-ollama-api-compatibility
Open

Fix Ollama API compatibility for newer versions#443
Samay10 wants to merge 1 commit intogoogle:mainfrom
Samay10:fix-ollama-api-compatibility

Conversation

@Samay10
Copy link
Copy Markdown

@Samay10 Samay10 commented Apr 14, 2026

Fixes #413

Description

Ollama API changed response structure in newer versions, moving generated content from response to thinking. This update adds backward compatibility by reading response first and falling back to thinking when response is empty.

  • Handle both old (response) and new (thinking) Ollama API response formats
  • Added a regression test for the new thinking response case
  • Updated Ollama response documentation to reflect the API change

How Has This Been Tested?

  • Added a new regression test for Ollama responses returning output in thinking
  • Ran pytest tests/inference_test.py::TestOllamaLanguageModel
  • Verified ./autoformat.sh and pre-commit formatting checks

Checklist

  • My code follows the style guidelines
  • I have self-reviewed my own code
  • I have commented my code where necessary
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests passed locally with my changes

Ollama API changed response structure in newer versions, moving
generated content from 'response' key to 'thinking' key. This fix
adds backward compatibility by checking both keys.

- Handle both old ('response') and new ('thinking') API formats
- Added comprehensive test for new response structure
- Updated documentation to reflect API changes

Fixes compatibility with Ollama versions that use 'thinking' key.
@github-actions
Copy link
Copy Markdown

No linked issues found. Please link an issue in your pull request description or title.

Per our Contributing Guidelines, all PRs must:

  • Reference an issue with one of:
    • Closing keywords: Fixes #123, Closes #123, Resolves #123 (auto-closes on merge in the same repository)
    • Reference keywords: Related to #123, Refs #123, Part of #123, See #123 (links without closing)
  • The linked issue should have 5+ 👍 reactions from unique users (excluding bots and the PR author)
  • Include discussion demonstrating the importance of the change

You can also use cross-repo references like owner/repo#123 or full URLs.

@github-actions github-actions bot added the size/XS Pull request with less than 50 lines changed label Apr 14, 2026
@github-actions
Copy link
Copy Markdown

⚠️ Branch Update Required

Your branch is 1 commits behind main. Please update your branch to ensure CI checks run with the latest code:

git fetch origin main
git merge origin/main
git push

Note: Enable "Allow edits by maintainers" to allow automatic updates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/XS Pull request with less than 50 lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: Returnning an empty string issue

1 participant