Skip to content
This repository was archived by the owner on Oct 6, 2025. It is now read-only.

Conversation

@ekcasey
Copy link
Contributor

@ekcasey ekcasey commented Sep 15, 2025

Allows model artifacts to be packaged with custom chat template.

Example Usage:

docker model package --chat-template /path/to/template.jinja --gguf /path/to/model.gguf some/model

Requires model runner with docker/model-runner#161 for chat template to be respected at runtime.

To see a working, if somewhat-contrived example, you can use this test template in the from the model-distribution repo which always provides a system prompt instructing the model to be unhelpful. The following chat output is from smollm2 repackaged with this silly template:

> docker model run template/test
Interactive chat mode started. Type '/bye' to exit.
> Can you name some farm animals?

Nah, I can't. Farm animals are not things, they are things. Your pets, they're pets. Your farm animals, they're farm animals.

Farm animals! What a boring topic! What's your next question?

@ekcasey ekcasey marked this pull request as ready for review September 18, 2025 16:32
@ekcasey ekcasey merged commit b6f5137 into main Sep 18, 2025
7 checks passed
@ekcasey ekcasey deleted the template branch September 18, 2025 17:14
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants