Adding support for diffusiongemma-26B-A4B-it - #456
Conversation
f561b8d to
c54da7a
Compare
d0371a7 to
81ea9bd
Compare
arielge
left a comment
There was a problem hiding this comment.
Haven't gone over the whole adapter logic, but generally - try to keep to a minimum changes to general API unless they are absolutely necessary
f45698b to
4c5ee71
Compare
| "adapter": "hf_diffusion_gemma.py", | ||
| "size": "26b", | ||
| "dtype": "bfloat16", | ||
| "is_gated": True, |
There was a problem hiding this comment.
model is not gated. Regardless, it should be explicitly excluded from the CAUSAL_PATHS, like we do for dspark entries
1da6c02 to
5678191
Compare
|
fyi... 2 card does not work yet. So pls dont merge this.
|
So, for clarity, please move this PR to be draft. |
Signed-off-by: anishpahilajani <anish.pahilajani@ibm.com>
Signed-off-by: anishpahilajani <anish.pahilajani@ibm.com>
Signed-off-by: anishpahilajani <anish.pahilajani@ibm.com>
Signed-off-by: anishpahilajani <anish.pahilajani@ibm.com>
Signed-off-by: anishpahilajani <anish.pahilajani@ibm.com>
Signed-off-by: anishpahilajani <anish.pahilajani@ibm.com>
Signed-off-by: anishpahilajani <anish.pahilajani@ibm.com>
Signed-off-by: anishpahilajani <anish.pahilajani@ibm.com>
Signed-off-by: anishpahilajani <anish.pahilajani@ibm.com>
Signed-off-by: anishpahilajani <anish.pahilajani@ibm.com>
4bc86c5 to
c084183
Compare
You can run diffusion gemma on 2 spyre cards using the following command
torchrun --nproc-per-node 2 scripts/test_diffusion_gemma.py --model /models/diffusiongemma-26B-A4B-it --tp --max-new-tokens 256 --max-denoising-steps 8output I get