Question regarding the asymmetrical input/output resolutions (224 vs 256) in RAE

[T-S-Liang]

Dear Authors,

Thank you for your impressive work!

I have a question regarding the implementation details of the RAE mentioned in the "Practical Notes" section. You mentioned that:

The input images are interpolated to 224 × 224 with an encoder patch size pe =14.

The decoder uses a patch size pd = 16 to reconstruct 256 × 256 images.

Both configurations result in 256 tokens (16×16 grid).

My questions are:

1. Motivation for Asymmetry: What was the primary motivation for keeping the token count consistent (256) while allowing the spatial resolution to change between input (224) and output (256)? Is this primarily to accommodate the fixed pe =14 constraint of the pre-trained DINOv2 weights?

2. Did you experiment with a symmetric setup (e.g., both input and output at 224 × 224 or 256 × 256)? Given that newer variants (like DINOv3) or different backbones support a patch size of 16, which would allow for a symmetric 256 × 256 pipeline, did you observe any significant impact on the quality of learned representations or reconstruction fidelity when using the asymmetrical 224-to-256 approach?

Thank you for your time and for sharing your insights!

[bytetriper]

Hi,

1. It's simply about keeping the token amount (256) the same for fair comparisons with other DiT models and baselines. And yes, it would be better if dinov2 has a pe=16 version

2. In scale-rae, all images operates on 224x224, hope that provide some references. In RAE we didn't try using 224x224 becuase calculating FID on 224 resolution is not a standard practice and is hard to compare with 256 resolution FIDs. Also it's non-trivial to apply DINO on 256 images because 256 can't be divided by 14. We tried DINOv3 and it brings small (but existing) difference to both generation and reconstruction. But yes using 224-to-256 would definitely hurts reconstruction.

[T-S-Liang]

Thank you for the clarification, this is very helpful.

I was also wondering whether you have any plans to release the DINOv3-based RAE checkpoint or the related implementation?

Thanks again for your time!