Add MLX Local Provider Support in Mirascope

[tnadav]

Description

It would be great if Mirascope could include MLX (Apple’s MLX framework) as a local provider. This would allow running models locally on Apple Silicon under the Mirascope abstraction (via an MLX local provider), enabling acceleration through unified memory / Metal / MLX’s efficient backends.

If there is general interest, I don't mind contributing to this fine project.

[willbakst]

Once we have v2 out in beta, we would love a contribution that adds an MLX client!

We should be releasing the beta for testing some time in the next week or two hopefully

[tnadav]

Hi @willbakst
Just checking in — has there been any update on the v2 beta release? I’m eager to start working on the MLX client as soon as there’s a stable enough interface available. Thanks!

[willbakst]

We're making good progress! We still have a few core items to implement (like thinking support), but the client interfaces should be somewhat stable enough if you wanted to start working on the MLX client.

For reference, you can find the current clients module here. This has great reference implementations of various providers (OpenAI Completions, OpenAI Responses, Anthropic, and Google) that should demonstrate how we plan to support various additional providers in the future.

Unfortunately, we don't have the CONTRIBUTING.md guide written for v2 just yet, so if you have any questions please ask away (either here or in the Discord community). Same goes for any questions you may have on the implementation or how you might want to go about implementing support for MLX.

Also, after a little research, it seems like you're talking about implementing a client on top of mlx-llm? One thing to note when working on this would be to start simple and use raise NotImplementedError for any paths not yet supported. This will make it easy to get smaller chunks into review (like simple generation) and merged before working on additional features (like structured generation).

[tnadav]

Thanks so much for the detailed and encouraging response — really appreciated!

Yes, I’m indeed referring to implementing the client on top of mlx-llm. My plan is to start with a basic version to get things working, and then extend it to fully support tool calling and structured output on models like gpt-oss-20b.

To achieve that, I’m considering implementing:

• a variant that natively supports OpenAI Harmony (for models aligned with that interface), and

• another variant that uses huggingface chat templating for other models such as llama qwen etc.

[willbakst]

Sounds like a plan! Before anything else, I would draft the interface first and think through any challenges that may arise given the differences between local models and cloud providers. Also worth researching if existing libraries can handle a lot of the conversion to native types (like Harmony) so the maintenance burden doesn't fall on us unless necessary.

[tnadav]

I've opened a draft PR with a limited, basic, but working implementation