Moondream:v2 issue 500 error

[MichielBontenbal]

I get the error message as seen below: ResponseError: POST predict: Post "http://127.0.0.1:61868/completion": EOF (status code: 500)

My settings:

• Python 3.12.2

• MacOS intel 8Gb RAM with limited free disk space (± 15 Gb)

• VS Code with Jupyter Notebook

• Latest version of ollama, latest version of ollama-python package

• ollama is working correctly from the command line (ollama run moondream:v2 describe this image /CV images datasets/solved/elephant_swimming.jpg gives expected result)

• I only get this when running moondream:v2. When I run text models there is no issue.

---

ResponseError Traceback (most recent call last)
Cell In[7], line 5
1 #source: https://github.com/ollama/ollama-python
2 # WARNING: THIS MIGHT TAKE LONGER THAN A MINUTE DEPENDING ON YOUR DEVICE
3 import ollama
----> 5 res = ollama.chat(
6 model="moondream:v2",
7 messages=[
8 {
9 'role': 'user',
10 'content': 'What is strange about this image?:',
11 'images': [image_path]
12 }
13 ]
14 )
16 print(res['message']['content'])

File /opt/anaconda3/lib/python3.12/site-packages/ollama/_client.py:333, in Client.chat(self, model, messages, tools, stream, format, options, keep_alive)
289 def chat(
290 self,
291 model: str = '',
(...)
298 keep_alive: Optional[Union[float, str]] = None,
299 ) -> Union[ChatResponse, Iterator[ChatResponse]]:
300 """
301 Create a chat response using the requested model.
302
(...)
331 Returns ChatResponse if stream is False, otherwise returns a ChatResponse generator.
332 """
--> 333 return self._request(
334 ChatResponse,
335 'POST',
336 '/api/chat',
337 json=ChatRequest(
338 model=model,
339 messages=[message for message in _copy_messages(messages)],
340 tools=[tool for tool in _copy_tools(tools)],
341 stream=stream,
342 format=format,
343 options=options,
344 keep_alive=keep_alive,
345 ).model_dump(exclude_none=True),
346 stream=stream,
347 )

File /opt/anaconda3/lib/python3.12/site-packages/ollama/_client.py:178, in Client._request(self, cls, stream, *args, **kwargs)
174 yield cls(**part)
176 return inner()
--> 178 return cls(**self._request_raw(*args, **kwargs).json())

File /opt/anaconda3/lib/python3.12/site-packages/ollama/_client.py:122, in Client._request_raw(self, *args, **kwargs)
120 return r
121 except httpx.HTTPStatusError as e:
--> 122 raise ResponseError(e.response.text, e.response.status_code) from None
123 except httpx.ConnectError:
124 raise ConnectionError(CONNECTION_ERROR_MESSAGE) from None

ResponseError: POST predict: Post "http://127.0.0.1:61868/completion": EOF (status code: 500)

[ParthSareen]

Are you port forwarding to get port 61868?
Can you also drop the server logs from Ollama, if it's a 500 won't be able to help without that

[jessegross]

This is triggered by ollama/ollama#7441