How can I leverage additional VRAM headroom?

[JamesDConley]

I've been testing on an RTX Pro 6000 Blackwell and see with the standard Deepseek-V3-Chat.yaml only 16372MiB / 97887MiB is being used.

Can I offload more layers/compute to cuda to speed things up further, and if so what would you recommend offloading first?

Thanks!

[fernandaspets]

Sam question I have 7 gpus and trying to use all available vram with Kimi K2 but the optimized .yaml file seems not to use more than first gpu?