Connecting with olllama For Locally hosted models

[Thillai22]

Just wanted to start a discussion regarding having the setup for locally hosted models on their desktop through Ollama.
Do you guys have that in the roadmap. I am interested in contributing to this community as well

[chheller]

If your local model is exposed over an openai-like api, you can add a new model to ~/.code_puppy/extra_models.json and then select it in code-puppy like any other model.

For instance, I installed qwen/qwen3-coder-30b in LM Studio, started the server, and added this config

{
  "Local-Qwen-3-coder-30b": {
    "type": "custom_openai",
    "name": "qwen/qwen3-coder-30b",
    "custom_endpoint": {
      "url": "http://localhost:1234/v1"
    },
    "context_length": 65536
  }
}

and everything worked mostly out of the box. I had to tweak my model to get a context window of 65k to fit in memory with the model, and the name property needs to be the name that you see in LM Studio. The type should be custom_openai.

If like me, you run code-puppy and other tools out of a linux image on WSL2, you may need extra steps to get localhost to mirror correctly to a server run from the Windows side. For me, that mean adding

[wsl2]
networkingMode=mirrored

to the .wslconfig in my user directory and restarting wsl2. I had also tried using http://$(hostname).local:1234/v1 but whenever I tried to connect using that, or whatever $(hostname) actually evaluated to, the request simply hung in both cURL and code-puppy.