Unclear coordinate convention in KHR_gaussian_splatting

[hybridherbst]

The KHR_gaussian_splatting spec (https://github.com/KhronosGroup/glTF/tree/main/extensions/2.0/Khronos/KHR_gaussian_splatting) has this note about coordinate systems that I personally find rather unclear:

For training software and exporters, it is recommended that the Gaussians are trained within the glTF coordinate system when targeting glTF. Otherwise, when converting pre-trained data from other coordinate systems into the glTF coordinate system, the positions, quaternions and spherical harmonics will have to be rotated on load.

(emphasis on the confusing "rotated on load" by me)

I was looking into what coordinate system the splat data is in, and was expecting to find a note that defines that as the glTF coordinate system. I first thought that this is implicit as this is a glTF extension specification, but that note "rotated on load" now has me confused. Does that mean it's totally valid if someone embeds arbitrarily oriented data in their glTF, and then has to tell the consuming app what the correct orientation for load is? Or is that then defined per extra extension (e.g. #2531)?

cc @weegeekps @lexaknyazev

[javagl]

@weegeekps To confirm, but I have the strong suspicion that this statement is a leftover from the state where "THE" Gaussian Splatting glTF extension was still based on SPZ (i.e. from the state before the actual KHR_gaussian_splatting base extension was extracted from that).

One important point is: In KHR_gaussian_splatting, there simply is no way to tell the client what the coordinate system is. So it has to be the glTF coordinate system.

(There once had been a paragraph in the spec that talked about the coordinate system conversion that is required for SPZ - namely, "RUB->LUF" - but it looks like this paragraph was also removed, and the required conversion is only "implicitly" given via https://github.com/nianticlabs/spz/tree/v2.0.0#coordinate-system )

[lexaknyazev]

The recommendation is that the entire pipeline should use the glTF coordinate system. However if the source data uses a different coordinate system, it would have to be converted when exporting to glTF. The "on load" part refers to the conversion process, not the final glTF asset.

[hybridherbst]

Thanks for the clarification! I think it would be good if the "on load" part was adjusted to make the intent more clear; I don't think I'm the only one who thinks of "on load" as the point in time where the glTF is loaded into a runtime.