Draft : Neural Harmonics Texture integration#235
Open
moennen wants to merge 9 commits into
Open
Conversation
…ecialization `HitParticleT<HasCanonical>` splits the k-buffer hit record into an SH base and an NHT derived that adds the `canonicalIntersection` float3. The renderer aliases pick the right one from `Params::PerRayParticleFeatures`, so the SH path pays no storage for a field it never reads. `densityHit` routes its out-arg through `canonicalIntersectionSlot` (struct field or stack scratch). The forward feature ternary becomes `if constexpr` so the SH specialization does not name-lookup a missing member. Made-with: Cursor
Updates NHT encoding, tetrahedral interpolation, alpha handling, initialization, and refinement behavior so benchmark runs are closer to the GSplat reference while preserving SH defaults.
moennen
force-pushed
the
nicolasm/nht-integration-2-1
branch
from
2270775 to
f850909
Compare
Keeps the production NHT backward path unconditional so diagnostic-only broken modes cannot affect training builds.
Aligns final evaluation with EMA semantics, keeps resumed training on live decoder weights, and mirrors forward hit bounds in the 3DGUT no-k-buffer backward path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds Neural Harmonic Textures (NHT) as a new per-particle feature model for both 3DGRT (OptiX/Slang) and 3DGUT (CUDA/Slang), alongside the existing SH radiance path.
Generalizes the rendering pipeline from fixed 3-channel radiance to arbitrary-dim particle/ray features, with an optional MLP feature decoder (threedgrut/model/feature_decoder.py, features.py).
Adds optional FP16 storage for per-particle features (render.particle_feature_half) and for the integrated per-ray feature output (render.feature_output_half); gradients stay FP32.
New app configs: {nerf_synthetic,colmap}_{3dgrt,3dgut}_mcmc_nht.yaml.