Skip to content

Add Gemma#87

Closed
yisi-wang-slalom wants to merge 5 commits intoaws-neuron:mainfrom
yisi-wang-slalom:add_gemma
Closed

Add Gemma#87
yisi-wang-slalom wants to merge 5 commits intoaws-neuron:mainfrom
yisi-wang-slalom:add_gemma

Conversation

@yisi-wang-slalom
Copy link
Copy Markdown

@yisi-wang-slalom yisi-wang-slalom commented May 23, 2024

Description of changes:

We are excited to help adding support for Gemma model architectured by Google.

A few highlights, mainly referencing: huggingface/transformers#29402

  • Using gelu_new activation function, we've validated it against the PyTorch implementation of gelu_pytorch_tanh to ensure same result, and recommend implementing gelu_pytorch_tanh when possible.
  • Add Layernorm (w+1) of RMS Layernorm for gemma
  • Include additional normalization in embedding (multiplies the embeddings by sqrt(hidden_dim)) for gemma

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@aws-maens aws-maens requested a review from mmcclean-aws July 9, 2024 20:29
@aws-guangyhu
Copy link
Copy Markdown

As of release 2.26.0, support has ended for the Transformers NeuronX library. If you are still using Transformers NeuronX, Neuron recommends that you migrate to the NxD Inference library, which provides a PyTorch-based inference library. Refer to the migration guide to learn how to migrate your Transformers NeuronX workloads to NxD Inference. PyTorch inference Deep Learning Container (DLC) will no longer include the transformers-neuronx package as well and Neuron no longer provides the transformers_neuronx virtual environment in both single and multi-framework DLAMIs. For more details, see Announcing end of support for Transformers NeuronX library starting in Neuron 2.26 release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants