add some documentation on porting models #264
Merged
+731
−42
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
davidkoski
commented
Apr 9, 2025
| @@ -106,19 +105,13 @@ private class MLP: Module, UnaryLayer { | |||
| } | |||
|
|
|||
| private class TransformerBlock: Module { | |||
| let numAttentionHeads: Int | |||
| let hiddenSize: Int | |||
There was a problem hiding this comment.
Just some minor cleanup (unused) to match the documentation
davidkoski
commented
Apr 9, 2025
| private let _ropeTheta: Float? | ||
| public var ropeTheta: Float { _ropeTheta ?? 10_000 } | ||
| private let _ropeTraditional: Bool? | ||
| public var ropeTraditional: Bool { _ropeTraditional ?? false } |
There was a problem hiding this comment.
Use the newer style of default values.
davidkoski
commented
Apr 9, 2025
| @@ -0,0 +1,696 @@ | |||
| # Porting Models | |||
|
|
|||
| How to make new models using `mlx-swift`. | |||
There was a problem hiding this comment.
This is the real change -- some documentation on how to port. This is focused on the LLM side but it would apply to any model really. We can add a section on VLMs as they have a slightly different structure (more complex)
awni
reviewed
Apr 11, 2025
Libraries/MLXLLM/README.md
Outdated
| - [Porting and implementing models](https://swiftpackageindex.com/ml-explore/mlx-swift-examples/main/documentation/mlxlmcommon/porting) | ||
| - [MLXLLMCommon](https://swiftpackageindex.com/ml-explore/mlx-swift-examples/main/documentation/mlxlmcommon) -- common API for LLM and VLM | ||
| - [MLXLLM](https://swiftpackageindex.com/ml-explore/mlx-swift-examples/main/documentation/mlxllm) -- large language model example implementations | ||
| - [MLXVLM](https://swiftpackageindex.com/ml-explore/mlx-swift-examples/main/documentation/mlxvlm) -- visual language model example implementations |
There was a problem hiding this comment.
Nit: visual language model -> vision language model is the more common expansion. Mind updating it here and below?
awni
reviewed
Apr 11, 2025
README.md
Outdated
| @@ -2,6 +2,7 @@ | |||
|
|
|||
| Developers can use these examples in their own programs -- just import the swift package! | |||
|
|
|||
| - [Porting and implementing models](https://swiftpackageindex.com/ml-explore/mlx-swift-examples/main/documentation/mlxlmcommon/porting) | |||
| - [MLXLLMCommon](https://swiftpackageindex.com/ml-explore/mlx-swift-examples/main/documentation/mlxlmcommon) -- common API for LLM and VLM | |||
| - [MLXLLM](https://swiftpackageindex.com/ml-explore/mlx-swift-examples/main/documentation/mlxllm) -- large language model example implementations | |||
| - [MLXVLM](https://swiftpackageindex.com/ml-explore/mlx-swift-examples/main/documentation/mlxvlm) -- visual language model example implementations | |||
awni
reviewed
Apr 11, 2025
Libraries/MLXVLM/README.md
Outdated
| - [Porting and implementing models](https://swiftpackageindex.com/ml-explore/mlx-swift-examples/main/documentation/mlxlmcommon/porting) | ||
| - [MLXLLMCommon](https://swiftpackageindex.com/ml-explore/mlx-swift-examples/main/documentation/mlxlmcommon) -- common API for LLM and VLM | ||
| - [MLXLLM](https://swiftpackageindex.com/ml-explore/mlx-swift-examples/main/documentation/mlxllm) -- large language model example implementations | ||
| - [MLXVLM](https://swiftpackageindex.com/ml-explore/mlx-swift-examples/main/documentation/mlxvlm) -- visual language model example implementations |
awni
reviewed
Apr 11, 2025
Libraries/MLXLMCommon/README.md
Outdated
| - [Porting and implementing models](https://swiftpackageindex.com/ml-explore/mlx-swift-examples/main/documentation/mlxlmcommon/porting) | ||
| - [MLXLLMCommon](https://swiftpackageindex.com/ml-explore/mlx-swift-examples/main/documentation/mlxlmcommon) -- common API for LLM and VLM | ||
| - [MLXLLM](https://swiftpackageindex.com/ml-explore/mlx-swift-examples/main/documentation/mlxllm) -- large language model example implementations | ||
| - [MLXVLM](https://swiftpackageindex.com/ml-explore/mlx-swift-examples/main/documentation/mlxvlm) -- visual language model example implementations |
awni
reviewed
Apr 11, 2025
| ## Porting Models from MLX (Python) | ||
|
|
||
| Let's consider a concrete example, | ||
| [Gemma.py](https://github.com/ml-explore/mlx-lm/blob/main/mlx_lm/models/gemma.py). For |
awni
reviewed
Apr 11, 2025
awni
reviewed
Apr 11, 2025
awni
reviewed
Apr 11, 2025
Comment on lines
37
to
39
| - MLXFast | ||
| - specialized and optimized functions for e.g. `rmsNorm` | ||
| - Note: this becomes part of `MLX` in current releases |
There was a problem hiding this comment.
Since it's part of MLX should we remove this?
There was a problem hiding this comment.
We haven't updated the dependency on mlx-swift yet, but I think we should soon -- and then this can be deleted.
awni
reviewed
Apr 11, 2025
Co-authored-by: Awni Hannun <[email protected]>
Co-authored-by: Awni Hannun <[email protected]>
Co-authored-by: Awni Hannun <[email protected]>
|
@awni thanks for reading through and the feedback -- I will get this addressed first thing next week! |
awni
approved these changes
Apr 11, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add some notes and tips on how to port models to mlx-swift