Add Quanto4,2, HQQ4,2 KV cache quantization support to Transformers loader#6768
Open
dinerburger wants to merge 3 commits intooobabooga:devfrom
Open
Add Quanto4,2, HQQ4,2 KV cache quantization support to Transformers loader#6768dinerburger wants to merge 3 commits intooobabooga:devfrom
dinerburger wants to merge 3 commits intooobabooga:devfrom