Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chat template mappings to the get_chat_template keys #1452

Open
alvations opened this issue Dec 19, 2024 · 1 comment
Open

Chat template mappings to the get_chat_template keys #1452

alvations opened this issue Dec 19, 2024 · 1 comment

Comments

@alvations
Copy link

I've tried looking around for the keys to use for the get_chat_template function but I can't figure out some of the models' keys.

From https://github.com/unslothai/unsloth/blob/main/unsloth/chat_templates.py

  • unsloth/Qwen2.5-* models maps to tokenizer = get_chat_template(tokenizer, chat_template = "qwen25")
  • unsloth/[Mm]istral* models maps to tokenizer = get_chat_template(tokenizer, chat_template = "mistral")
  • unsloth/*[Ll]lama* models maps to tokenizer = get_chat_template(tokenizer, chat_template = "llama")

How about these, what should be the key to fetch the chat templates?

  • unsloth/gemma*
  • unsloth/Phi-3*
@shimmyshimmer
Copy link
Collaborator

Good question - I think we'll need to add this in.

For Gemma it's:
<start_of_turn>user
knock knock<end_of_turn>
<start_of_turn>model
who is there<end_of_turn>
<start_of_turn>user
Gemma<end_of_turn>
<start_of_turn>model
Gemma who?<end_of_turn>

For Phi-3.5 it's:
"chat_template": "{% for message in messages %}{% if message['role'] == 'system' and message['content'] %}{{'<|system|>\n' + message['content'] + '<|end|>\n'}}{% elif message['role'] == 'user' %}{{'<|user|>\n' + message['content'] + '<|end|>\n'}}{% elif message['role'] == 'assistant' %}{{'<|assistant|>\n' + message['content'] + '<|end|>\n'}}{% endif %}{% endfor %}{% if add_generation_prompt %}{{ '<|assistant|>\n' }}{% else %}{{ eos_token }}{% endif %}",

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants