Skip to content

Latest commit

 

History

History
32 lines (20 loc) · 896 Bytes

README.md

File metadata and controls

32 lines (20 loc) · 896 Bytes

Llama

This is a port of several models from:

using the Hugging Face swift transformers package to provide tokenization:

https://github.com/huggingface/swift-transformers

The Models.swift provides minor overrides and customization -- if you require overrides for the tokenizer or prompt customizations they can be added there.

This is set up to load models from Hugging Face, e.g. https://huggingface.co/mlx-community

The following models have been tried:

  • mlx-community/Mistral-7B-v0.1-hf-4bit-mlx
  • mlx-community/CodeLlama-13b-Instruct-hf-4bit-MLX
  • mlx-community/phi-2-hf-4bit-mlx
  • mlx-community/quantized-gemma-2b-it

Currently supported model types are:

  • Llama / Mistral
  • Gemma
  • Phi

See Configuration.swift for more info.

See llm-tool