VRAM requirement?

[Cubey42]

I'm using a 4090 with 24gb of VRAM, and after a couple hours of setting up, I'm finding I can't seem to run it (I had success with Trellis.2 In the past) does it add more memory requirements? I had to switch to flash_attn and build the correct natten, and was just wondering if its simply too large to use locally?

[Saganaki22]

I'm using a 4090 with 24gb of VRAM, and after a couple hours of setting up, I'm finding I can't seem to run it (I had success with Trellis.2 In the past) does it add more memory requirements? I had to switch to flash_attn and build the correct natten, and was just wondering if its simply too large to use locally?

depending on settings 18-36 gb vram

[Cubey42]

Okay after a bit of vibecoding, I was able to find parameters and multiple offloading strategies to keep most of the default settings of the repo but also needing
pipeline.low_vram = True
and
return_latent=False,
otherwise seems to work as expected.

[LDYang694]

Hi, thanks for your feedback!

We have merged the related PR for low-VRAM support. For running Pixal3D with limited VRAM, please refer to the latest README and follow the updated low-VRAM usage instructions.

Feel free to try the latest version and let us know if you still encounter any issues.

[droplotus]

Can we use multiple gpus?
Got 1 with 12vram and another with 16 vram.

[able2608]

Dropped the peak VRAM use (for 1024x1024) down to around 11GB with patches on the DinoV3. Verified to work on an RTX3060 Laptop with 6GB of VRAM (+system memory fallback) and 16GB of RAM (and a whole lot of swap, though seems to mostly been used for model parameter storage) with 1024x1024 resolution. Will open PR if there are any requests after cleaning it up a bit (thanks Claude), and alongside DiT layer offloading implemented with accelerate.

The main catch seems to be that as the resolution grows DinoV3 uses up a lot more memory, and (according to Claude) it seems that model parameter is not the case, but the intermediate features were. Claude made it work with patched feature extraction, but still on 1536x1536 resolution 6+8 GB (VRAM + max system RAM fallback) it OOMs on HR SLat sampling with crazy low patch size. The DiTs on the other hand seems to not use up that much memory in the first place, and while using layer offloading can pretty much zero out VRAM usage, I don't think it is that useful in the first place as DinoV3 is pretty much the one that sets the memory requirements.

In short, for low (low) VRAM users 1024x1024 will work as long as you patch DinoV3 to do patched feature extraction, and expect around 11GB of VRAM+system RAM fallback during inference.