Thanks for open sourcing this super powerful differentiable rendering framework. Actually I have used it in MANY of my project. However, I still don't fully understand some core principles and ideas of nvidiffrast.
(1) How does NVDiffrast make the rasterization differentiable?
As far as I know, traditional rasterization is NOT differentiable because of the abrupt triangle boundary change (XY for color blending, Z for depth testing), and mipmap level calculation etc. According to Section3.6(Analytic Antialiasing for Visibility Gradients) of the paper, the anti-alias op propagates the gradient from pixels to triangles with silhouette edges. In that case, if we don't do anti-alias, there will be no gradients? And what about occluded triangles? If we are optimizing the depth order of two(or more objects) , and a triangle is occluded by another, no gradient would be propagated to the occluded triangle, what made the optimization possible?
(2) Could you further explain the following claim about visibility gradients premise in Section2 of the paper? Which part of nvdiffrast is built upon this premise? And what does that mean to the capability boundary of nvdiffrast?
The reparameterization technique of Loubet et al. [2019] used in Mitsuba 2 [Nimier-David et al. 2019] produces
correct visibility gradients only when occluders and occludees can be inferred from four samples. Although more complex occlusion
scenarios are not handled correctly, Loubet et al. introduce a bandwidth parameter that can be adjusted to reduce the errors at the cost
of increased noise. It is argued that handling the most common, simple cases is sufficient for practical purposes, and our approach for
visiblity gradients is founded on the same premise.
Appreciate a lot if you could further explain these two! Thank you very much :)
Thanks for open sourcing this super powerful differentiable rendering framework. Actually I have used it in MANY of my project. However, I still don't fully understand some core principles and ideas of nvidiffrast.
(1) How does NVDiffrast make the rasterization differentiable?
As far as I know, traditional rasterization is NOT differentiable because of the abrupt triangle boundary change (XY for color blending, Z for depth testing), and mipmap level calculation etc. According to Section3.6(Analytic Antialiasing for Visibility Gradients) of the paper, the anti-alias op propagates the gradient from pixels to triangles with silhouette edges. In that case, if we don't do anti-alias, there will be no gradients? And what about occluded triangles? If we are optimizing the depth order of two(or more objects) , and a triangle is occluded by another, no gradient would be propagated to the occluded triangle, what made the optimization possible?
(2) Could you further explain the following claim about visibility gradients premise in Section2 of the paper? Which part of nvdiffrast is built upon this premise? And what does that mean to the capability boundary of nvdiffrast?
Appreciate a lot if you could further explain these two! Thank you very much :)