diff --git a/.gitignore b/.gitignore index 054e565..083628c 100644 --- a/.gitignore +++ b/.gitignore @@ -24,4 +24,5 @@ dist-ssr *.sw? /.vite -*/scenes \ No newline at end of file +*/scenes +/scenes \ No newline at end of file diff --git a/README.md b/README.md index edffdaf..e2e1153 100644 --- a/README.md +++ b/README.md @@ -2,25 +2,135 @@ **University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 5** -* (TODO) YOUR NAME HERE -* Tested on: (TODO) **Google Chrome 222.2** on - Windows 22, i7-2222 @ 2.22GHz 22GB, GTX 222 222MB (Moore 2222 Lab) +* Jacqueline (Jackie) Li + * [LinkedIn](https://www.linkedin.com/in/jackie-lii/), [personal website](https://sites.google.com/seas.upenn.edu/jacquelineli/home), [Instagram](https://www.instagram.com/sagescherrytree/), etc. +* Tested on: : Chrome/141.0.7390.67, : Windows NT 10.0.19045.6332, 11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz, NVIDIA GeForce RTX 3060 Laptop GPU (6 GB) ### Live Demo -[![](img/thumb.png)](http://TODO.github.io/Project4-WebGPU-Forward-Plus-and-Clustered-Deferred) +[Demo Link](https://sagescherrytree.github.io/Project5-WebGPU-Gaussian-Splat-Viewer-2025/) +[![](images/HUGEHUGEHUGEBIKE.png)](https://sagescherrytree.github.io/Project5-WebGPU-Gaussian-Splat-Viewer-2025/) ### Demo Video/GIF -[![](img/video.mp4)](TODO) +| ![](images/demo_bicycle.gif) | ![](images/demo_bonsai.gif) | +|:--:|:--:| +| Bicycle | Bonsai | -### (TODO: Your README) +## Gaussian Splats Overview -*DO NOT* leave the README to the last minute! It is a crucial part of the -project, and we will not be able to grade you without a good README. +Gaussian Splats is a concept first introduced in the paper [3D Gaussian Splatting for Real Time Rendering](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/) by Kerbl et al at SIGGRAPH 2023. It is a concept which significantly optimised the neural radiance fields methodology, by representing the scene in 3D Gaussians that each carry properties of position, colour, size, and depth to recreate the scene while reducing unnecessary computations and thus optimising the runtime. -This assignment has a considerable amount of performance analysis compared -to implementation work. Complete the implementation early to leave time! +The portion that I focused on mainly for this project is not the actual training of the neural rendering representation, but the algorithm to render the resulting point clouds as Gaussian splats. For that, I set up a rendering pipeline on WebGPU that first passes the 3D Gaussian information to a preprocess compute shader to calculate each splat in an initial indirect render pass, then the created splat data will be passed to a vertex and fragment shader to be visualised. + +### Preprocess Step + +To render the Gaussian splats, I first pass the point cloud points to a preprocess shader in an indirect render pass. The points are transformed to NDC, then I use the rotation and scale information from the gaussian data to calculate the 2D covariance (following [this reference](https://github.com/kwea123/gaussian_splatting_notes)). Colour is calculated using [spherical harmonics](https://beatthezombie.github.io/sh_post_1/). Depth information is calculated prior to the preprocess compute step by running radix sort on the splat indices, but is passed to the compute shader to update to the vertex and fragment visualisation shader. Finally, all of this information is stored in the splats buffer, which then gets passed in the actual render pass to vertex and fragment wgsl shader, that uses the information to visualise the splats. + +Below is a breakdown of the different steps of the preprocess step, rendered as simple quads on the visualisation shader. These test scenes utilitse the bicycle.ply scene. + +| ![](images/quads_output.png) | +|:--:| +| Quads over points, plain visualisation (no colour, no depth) | + +| ![](images/radiusOP.png) | +|:--:| +| Quads over depth visualisation | + +| ![](images/shCoeffCols.png) | +|:--:| +| SH Coefficient visualisation | + +### Visualisation Step + +After the information from the preprocess step is calculated and repassed into the splats buffer, it gets passed through the render pass to a vertex shader, where all of the splat information is unpacked and used to draw the quads and its position in the vertex shader. The colour, size, and opacity get passed to the fragment shader, where the position gets converted to NDC again, and an alpha value is calculated to draw each quad with its correct transparency. + +| ![](images/HUGEBIKE.png) | +|:--:| +| Render output with no alpha | + +| ![](images/HUGEHUGEHUGEBIKE.png) | +|:--:| +| Render output with alpha included | + +## Performance Anaylsis + +### Point Cloud v. Gaussian Splats Render + +#### Number of Points v. Runtime Pointclouds and Runtime Gaussian + +| Scene | # of Points | Pointclouds | Gaussian Splat | +|-------------|----------------|-------------|----------------| +| bicycle.ply | 1063091 | 144 | 60 | +| bonsai.ply | 272956 | 140 | 144 | + +| ![](images/pointClouds_v_splats.png) | +|:--:| + +#### Gaussian Multiplier v. Runtime per Scene + +| Multiplier | bicycle.ply | bonsai.ply | +|------------|-------------|------------| +| 0.0 | 54 | 144 | +| 1.0 | 60 | 144 | +| 1.5 | 17 | 84 | + +| ![](images/gaussianMult_v_FPS.png) | +|:--:| + +From comparing the point clouds method with the Gaussian splat method, I can see a distinct difference in runtime for the bicycle.ply scene than for the bonsai.ply scene. I hypothesise that this is due to the bonsai scene having less points to splat, therefore running each of the covariance computes for each point would be less efficient than simply rendering the point cloud directly. For scenes with larger amounts of points do we truly see Gaussian splats method having more of an advantage, as exhibited by the bicycle.ply scene with 1063091 points. This scene sees a significant decrease in its runtime compared to its pointcloud counterpart, which is due to the fact that Gaussian Splats rely on large 2D splats that cover multiple pixels in size. Therefore, while a fragment shader for point clouds might be doing less with calculations, its total work scales linearly, meaning the more points it has the more cumulative work it does. Gaussian splats are therefore smarter in the sense that because its work covers multiple pixels, the rendering calculations only matter for only the currently visible portions, so fragment bound instead of per pixel. + +### Workgroup size and Performance + +#### Workgroup size v. Runtime per Scene + +| Workgroup Size | Runtime (FPS) | +|----------------|---------------| +| 128 | 100 | +| 256 | 58 | + +Since on this device I was only able to test two sizes of workgroups, I will make my observations regarding workgroup size based off of these two samples shown in the chart above. From observation, we can see that the size 128 workgroup runs slower than the size 256 workgroup. Since workgroups are defined as the number of concurrent commands being processed, for larger workgroups that typically means more threads running in parallel, utilising the most of the parallel computing power of the GPU. Too small workgroup size may stagger the runtime more and potentially cause issues as the latency in waiting for commands to complete calculation may cause instructions to execute before the previous command is completed with calculation, while too large workgroup size causes memory storage issues, as each thread fights for space to store its calculations. For my program, it seems that a workgroup size of 256 seems to optimise the FPS runtime the most, while still utilising the most of the GPU to the best it can. + +### View Frustrum Culling and Performance + +| View Frustrum Culling | Runtime (FPS) | +|-----------------------|---------------| +| Off | 20 | +| On | 17 | + +View frustrum culling is essentially a simple method to ensure that one only spends the necessary time computing the render calculations for the splats or points that are currently on the screen. For instance, taking bicycle.ply, if the entire scene contained 1000000 points but we only see 300000 at the moment, then it would indeed be more efficient to render only the 300000 points that one currently sees, instead of the entire 1000000 points still, even though it would be useless because we cannot even see it with the camera. For my Gaussian splat program using bicycle.ply, with view frustrum culling on, the runtime is 17 FPS, which is less than the 20 FPS for when view frustrum culling is off. I would surmise that the results would be even more prominent for a Gaussian scene with background points, as only rendering the currently visible splats in that case would reduce the runtime calculations by quite a significant amount. + +### Number of Gaussians and Performance + +In general, the number of Gaussians scale with runtime; the more Gaussians one loads into the scene, usually the higher the runtime because that would mean more total work and compute for each splat. Furthermore, without any optimisations in consideration, larger numbers of splats may lead to memory overload and storage issues... for instance, my machine can only load the basic scenes and has trouble with visualising the medium and high count ones due to the hardware limitation. View frustrum culling does help in a sense with optimisation because at least it ensures that only on screen gaussians will be calculated on, but there are certainly other ways to optimise, say, screen space culling by not rendering splats smaller than one pixel and the sort by depth tile method that was proposed in the paper for covariance and conic calculations linked in the overview section. + +## Bloopers + +Here are a collection of bloopers that I used when I was implementing. + +| ![](images/varianceByDepth.png) | +|:--:| +| Odd depth mapping | + +This visual bug occurred when I initially implemented the depth sorting, in which I may have used the wrong method to visualise the depth. I thought the result to be interesting, so I took a screenshot. + +| ![](images/altColQuads.png) | ![](images/monsterEnergy.png) | +|:--:|:--:| +| Colour visualisation 1 | Colour visualisation 2 | + +These above images were some debugging that I did for testing methods of passing the colour to the fragment shader, which was from before I changed my packaging methodology for the Splat struct. I visualised position and passed opacity in the output to test to see if at least those were being calculated relatively correctly, to narrow down the issue. + +| ![](images/bikeMoreBlurry.png) | ![](images/thinkMeQuadColTooDark2.png) | +|:--:|:--:| +| Quad size debug | Quad colour debug | + +These two results are from when I was testing out the quad size calculations... I do not recall what precisely I fixed to obtain either result, but for a while my quad colour was too dark, resulting in the right image. Finally, the image on the left is a visualisation method to test each quad's scale size. + +| ![](images/HUGEHUGEBIKE.png) | +|:--:| +| Rotation matrix off | + +This final bug, for which I think to be the most visually interesting, is a rotation bug in which the sign of one of the elements of the covariance vector was flipped. Although wrong, this result reminds me of a colour pencil sketch, which gives me interesting thoughts regarding stylising a Gaussian splat. ### Credits diff --git a/images/BONSAI.png b/images/BONSAI.png new file mode 100644 index 0000000..8f4a860 Binary files /dev/null and b/images/BONSAI.png differ diff --git a/images/HUGEBIKE.png b/images/HUGEBIKE.png new file mode 100644 index 0000000..42e3665 Binary files /dev/null and b/images/HUGEBIKE.png differ diff --git a/images/HUGEHUGEBIKE.png b/images/HUGEHUGEBIKE.png new file mode 100644 index 0000000..249317d Binary files /dev/null and b/images/HUGEHUGEBIKE.png differ diff --git a/images/HUGEHUGEHUGEBIKE.png b/images/HUGEHUGEHUGEBIKE.png new file mode 100644 index 0000000..4ba4196 Binary files /dev/null and b/images/HUGEHUGEHUGEBIKE.png differ diff --git a/images/altColQuads.png b/images/altColQuads.png new file mode 100644 index 0000000..7228ab9 Binary files /dev/null and b/images/altColQuads.png differ diff --git a/images/bicycle_0Mult.png b/images/bicycle_0Mult.png new file mode 100644 index 0000000..bc37c7e Binary files /dev/null and b/images/bicycle_0Mult.png differ diff --git a/images/bicycle_1.5Mult.png b/images/bicycle_1.5Mult.png new file mode 100644 index 0000000..55f71c4 Binary files /dev/null and b/images/bicycle_1.5Mult.png differ diff --git a/images/bikeMoreBlurry.png b/images/bikeMoreBlurry.png new file mode 100644 index 0000000..864d66c Binary files /dev/null and b/images/bikeMoreBlurry.png differ diff --git a/images/demo_bicycle.gif b/images/demo_bicycle.gif new file mode 100644 index 0000000..0f03ba0 Binary files /dev/null and b/images/demo_bicycle.gif differ diff --git a/images/demo_bicycle_mult.gif b/images/demo_bicycle_mult.gif new file mode 100644 index 0000000..be1b207 Binary files /dev/null and b/images/demo_bicycle_mult.gif differ diff --git a/images/demo_bonsai.gif b/images/demo_bonsai.gif new file mode 100644 index 0000000..2e62e11 Binary files /dev/null and b/images/demo_bonsai.gif differ diff --git a/images/demo_bonsai_mult.gif b/images/demo_bonsai_mult.gif new file mode 100644 index 0000000..fffccdc Binary files /dev/null and b/images/demo_bonsai_mult.gif differ diff --git a/images/flippedQuads.png b/images/flippedQuads.png new file mode 100644 index 0000000..a457b81 Binary files /dev/null and b/images/flippedQuads.png differ diff --git a/images/gaussianMult_v_FPS.png b/images/gaussianMult_v_FPS.png new file mode 100644 index 0000000..139d708 Binary files /dev/null and b/images/gaussianMult_v_FPS.png differ diff --git a/images/monsterEnergy.png b/images/monsterEnergy.png new file mode 100644 index 0000000..046f367 Binary files /dev/null and b/images/monsterEnergy.png differ diff --git a/images/pointClouds_bicycle.png b/images/pointClouds_bicycle.png new file mode 100644 index 0000000..52b17c9 Binary files /dev/null and b/images/pointClouds_bicycle.png differ diff --git a/images/pointClouds_bonsai.png b/images/pointClouds_bonsai.png new file mode 100644 index 0000000..1cb9f7b Binary files /dev/null and b/images/pointClouds_bonsai.png differ diff --git a/images/pointClouds_v_splats.png b/images/pointClouds_v_splats.png new file mode 100644 index 0000000..c3e98d9 Binary files /dev/null and b/images/pointClouds_v_splats.png differ diff --git a/images/quads_output.png b/images/quads_output.png new file mode 100644 index 0000000..123e932 Binary files /dev/null and b/images/quads_output.png differ diff --git a/images/radiusOP.png b/images/radiusOP.png new file mode 100644 index 0000000..788fb35 Binary files /dev/null and b/images/radiusOP.png differ diff --git a/images/radiusZoom.png b/images/radiusZoom.png new file mode 100644 index 0000000..04c4d2a Binary files /dev/null and b/images/radiusZoom.png differ diff --git a/images/shCoeffCols.png b/images/shCoeffCols.png new file mode 100644 index 0000000..fb402dc Binary files /dev/null and b/images/shCoeffCols.png differ diff --git a/images/thinkMeQuadColTooDark.png b/images/thinkMeQuadColTooDark.png new file mode 100644 index 0000000..194c846 Binary files /dev/null and b/images/thinkMeQuadColTooDark.png differ diff --git a/images/thinkMeQuadColTooDark2.png b/images/thinkMeQuadColTooDark2.png new file mode 100644 index 0000000..c77302f Binary files /dev/null and b/images/thinkMeQuadColTooDark2.png differ diff --git a/images/varianceByDepth.png b/images/varianceByDepth.png new file mode 100644 index 0000000..56e798b Binary files /dev/null and b/images/varianceByDepth.png differ diff --git a/package-lock.json b/package-lock.json index 04843bd..d0721e2 100644 --- a/package-lock.json +++ b/package-lock.json @@ -12,6 +12,7 @@ "@loaders.gl/ply": "^4.2.2", "@petamoriken/float16": "^3.8.7", "tweakpane": "^3.1.8", + "tweakpane-plugin-file-import": "^0.2.0", "wgpu-matrix": "^3.2.0" }, "devDependencies": { @@ -19,360 +20,450 @@ "@webgpu/types": "^0.1.31", "tweakpane-plugin-file-import": "^0.2.0", "typescript": "^5.0.4", - "vite": "^4.3.1", + "vite": "^7.1.11", "vite-raw-plugin": "^1.0.1" } }, + "node_modules/@esbuild/aix-ppc64": { + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.25.11.tgz", + "integrity": "sha512-Xt1dOL13m8u0WE8iplx9Ibbm+hFAO0GsU2P34UNoDGvZYkY8ifSiy6Zuc1lYxfG7svWE2fzqCUmFp5HCn51gJg==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "aix" + ], + "engines": { + "node": ">=18" + } + }, "node_modules/@esbuild/android-arm": { - "version": "0.18.20", - "resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.18.20.tgz", - "integrity": "sha512-fyi7TDI/ijKKNZTUJAQqiG5T7YjJXgnzkURqmGj13C6dCqckZBLdl4h7bkhHt/t0WP+zO9/zwroDvANaOqO5Sw==", + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.25.11.tgz", + "integrity": "sha512-uoa7dU+Dt3HYsethkJ1k6Z9YdcHjTrSb5NUy66ZfZaSV8hEYGD5ZHbEMXnqLFlbBflLsl89Zke7CAdDJ4JI+Gg==", "cpu": [ "arm" ], "dev": true, + "license": "MIT", "optional": true, "os": [ "android" ], "engines": { - "node": ">=12" + "node": ">=18" } }, "node_modules/@esbuild/android-arm64": { - "version": "0.18.20", - "resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.18.20.tgz", - "integrity": "sha512-Nz4rJcchGDtENV0eMKUNa6L12zz2zBDXuhj/Vjh18zGqB44Bi7MBMSXjgunJgjRhCmKOjnPuZp4Mb6OKqtMHLQ==", + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.25.11.tgz", + "integrity": "sha512-9slpyFBc4FPPz48+f6jyiXOx/Y4v34TUeDDXJpZqAWQn/08lKGeD8aDp9TMn9jDz2CiEuHwfhRmGBvpnd/PWIQ==", "cpu": [ "arm64" ], "dev": true, + "license": "MIT", "optional": true, "os": [ "android" ], "engines": { - "node": ">=12" + "node": ">=18" } }, "node_modules/@esbuild/android-x64": { - "version": "0.18.20", - "resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.18.20.tgz", - "integrity": "sha512-8GDdlePJA8D6zlZYJV/jnrRAi6rOiNaCC/JclcXpB+KIuvfBN4owLtgzY2bsxnx666XjJx2kDPUmnTtR8qKQUg==", + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.25.11.tgz", + "integrity": "sha512-Sgiab4xBjPU1QoPEIqS3Xx+R2lezu0LKIEcYe6pftr56PqPygbB7+szVnzoShbx64MUupqoE0KyRlN7gezbl8g==", "cpu": [ "x64" ], "dev": true, + "license": "MIT", "optional": true, "os": [ "android" ], "engines": { - "node": ">=12" + "node": ">=18" } }, "node_modules/@esbuild/darwin-arm64": { - "version": "0.18.20", - "resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.18.20.tgz", - "integrity": "sha512-bxRHW5kHU38zS2lPTPOyuyTm+S+eobPUnTNkdJEfAddYgEcll4xkT8DB9d2008DtTbl7uJag2HuE5NZAZgnNEA==", + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.25.11.tgz", + "integrity": "sha512-VekY0PBCukppoQrycFxUqkCojnTQhdec0vevUL/EDOCnXd9LKWqD/bHwMPzigIJXPhC59Vd1WFIL57SKs2mg4w==", "cpu": [ "arm64" ], "dev": true, + "license": "MIT", "optional": true, "os": [ "darwin" ], "engines": { - "node": ">=12" + "node": ">=18" } }, "node_modules/@esbuild/darwin-x64": { - "version": "0.18.20", - "resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.18.20.tgz", - "integrity": "sha512-pc5gxlMDxzm513qPGbCbDukOdsGtKhfxD1zJKXjCCcU7ju50O7MeAZ8c4krSJcOIJGFR+qx21yMMVYwiQvyTyQ==", + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.25.11.tgz", + "integrity": "sha512-+hfp3yfBalNEpTGp9loYgbknjR695HkqtY3d3/JjSRUyPg/xd6q+mQqIb5qdywnDxRZykIHs3axEqU6l1+oWEQ==", "cpu": [ "x64" ], "dev": true, + "license": "MIT", "optional": true, "os": [ "darwin" ], "engines": { - "node": ">=12" + "node": ">=18" } }, "node_modules/@esbuild/freebsd-arm64": { - "version": "0.18.20", - "resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.18.20.tgz", - "integrity": "sha512-yqDQHy4QHevpMAaxhhIwYPMv1NECwOvIpGCZkECn8w2WFHXjEwrBn3CeNIYsibZ/iZEUemj++M26W3cNR5h+Tw==", + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.25.11.tgz", + "integrity": "sha512-CmKjrnayyTJF2eVuO//uSjl/K3KsMIeYeyN7FyDBjsR3lnSJHaXlVoAK8DZa7lXWChbuOk7NjAc7ygAwrnPBhA==", "cpu": [ "arm64" ], "dev": true, + "license": "MIT", "optional": true, "os": [ "freebsd" ], "engines": { - "node": ">=12" + "node": ">=18" } }, "node_modules/@esbuild/freebsd-x64": { - "version": "0.18.20", - "resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.18.20.tgz", - "integrity": "sha512-tgWRPPuQsd3RmBZwarGVHZQvtzfEBOreNuxEMKFcd5DaDn2PbBxfwLcj4+aenoh7ctXcbXmOQIn8HI6mCSw5MQ==", + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.25.11.tgz", + "integrity": "sha512-Dyq+5oscTJvMaYPvW3x3FLpi2+gSZTCE/1ffdwuM6G1ARang/mb3jvjxs0mw6n3Lsw84ocfo9CrNMqc5lTfGOw==", "cpu": [ "x64" ], "dev": true, + "license": "MIT", "optional": true, "os": [ "freebsd" ], "engines": { - "node": ">=12" + "node": ">=18" } }, "node_modules/@esbuild/linux-arm": { - "version": "0.18.20", - "resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.18.20.tgz", - "integrity": "sha512-/5bHkMWnq1EgKr1V+Ybz3s1hWXok7mDFUMQ4cG10AfW3wL02PSZi5kFpYKrptDsgb2WAJIvRcDm+qIvXf/apvg==", + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.25.11.tgz", + "integrity": "sha512-TBMv6B4kCfrGJ8cUPo7vd6NECZH/8hPpBHHlYI3qzoYFvWu2AdTvZNuU/7hsbKWqu/COU7NIK12dHAAqBLLXgw==", "cpu": [ "arm" ], "dev": true, + "license": "MIT", "optional": true, "os": [ "linux" ], "engines": { - "node": ">=12" + "node": ">=18" } }, "node_modules/@esbuild/linux-arm64": { - "version": "0.18.20", - "resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.18.20.tgz", - "integrity": "sha512-2YbscF+UL7SQAVIpnWvYwM+3LskyDmPhe31pE7/aoTMFKKzIc9lLbyGUpmmb8a8AixOL61sQ/mFh3jEjHYFvdA==", + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.25.11.tgz", + "integrity": "sha512-Qr8AzcplUhGvdyUF08A1kHU3Vr2O88xxP0Tm8GcdVOUm25XYcMPp2YqSVHbLuXzYQMf9Bh/iKx7YPqECs6ffLA==", "cpu": [ "arm64" ], "dev": true, + "license": "MIT", "optional": true, "os": [ "linux" ], "engines": { - "node": ">=12" + "node": ">=18" } }, "node_modules/@esbuild/linux-ia32": { - "version": "0.18.20", - "resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.18.20.tgz", - "integrity": "sha512-P4etWwq6IsReT0E1KHU40bOnzMHoH73aXp96Fs8TIT6z9Hu8G6+0SHSw9i2isWrD2nbx2qo5yUqACgdfVGx7TA==", + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.25.11.tgz", + "integrity": "sha512-TmnJg8BMGPehs5JKrCLqyWTVAvielc615jbkOirATQvWWB1NMXY77oLMzsUjRLa0+ngecEmDGqt5jiDC6bfvOw==", "cpu": [ "ia32" ], "dev": true, + "license": "MIT", "optional": true, "os": [ "linux" ], "engines": { - "node": ">=12" + "node": ">=18" } }, "node_modules/@esbuild/linux-loong64": { - "version": "0.18.20", - "resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.18.20.tgz", - "integrity": "sha512-nXW8nqBTrOpDLPgPY9uV+/1DjxoQ7DoB2N8eocyq8I9XuqJ7BiAMDMf9n1xZM9TgW0J8zrquIb/A7s3BJv7rjg==", + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.25.11.tgz", + "integrity": "sha512-DIGXL2+gvDaXlaq8xruNXUJdT5tF+SBbJQKbWy/0J7OhU8gOHOzKmGIlfTTl6nHaCOoipxQbuJi7O++ldrxgMw==", "cpu": [ "loong64" ], "dev": true, + "license": "MIT", "optional": true, "os": [ "linux" ], "engines": { - "node": ">=12" + "node": ">=18" } }, "node_modules/@esbuild/linux-mips64el": { - "version": "0.18.20", - "resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.18.20.tgz", - "integrity": "sha512-d5NeaXZcHp8PzYy5VnXV3VSd2D328Zb+9dEq5HE6bw6+N86JVPExrA6O68OPwobntbNJ0pzCpUFZTo3w0GyetQ==", + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.25.11.tgz", + "integrity": "sha512-Osx1nALUJu4pU43o9OyjSCXokFkFbyzjXb6VhGIJZQ5JZi8ylCQ9/LFagolPsHtgw6himDSyb5ETSfmp4rpiKQ==", "cpu": [ "mips64el" ], "dev": true, + "license": "MIT", "optional": true, "os": [ "linux" ], "engines": { - "node": ">=12" + "node": ">=18" } }, "node_modules/@esbuild/linux-ppc64": { - "version": "0.18.20", - "resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.18.20.tgz", - "integrity": "sha512-WHPyeScRNcmANnLQkq6AfyXRFr5D6N2sKgkFo2FqguP44Nw2eyDlbTdZwd9GYk98DZG9QItIiTlFLHJHjxP3FA==", + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.25.11.tgz", + "integrity": "sha512-nbLFgsQQEsBa8XSgSTSlrnBSrpoWh7ioFDUmwo158gIm5NNP+17IYmNWzaIzWmgCxq56vfr34xGkOcZ7jX6CPw==", "cpu": [ "ppc64" ], "dev": true, + "license": "MIT", "optional": true, "os": [ "linux" ], "engines": { - "node": ">=12" + "node": ">=18" } }, "node_modules/@esbuild/linux-riscv64": { - "version": "0.18.20", - "resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.18.20.tgz", - "integrity": "sha512-WSxo6h5ecI5XH34KC7w5veNnKkju3zBRLEQNY7mv5mtBmrP/MjNBCAlsM2u5hDBlS3NGcTQpoBvRzqBcRtpq1A==", + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.25.11.tgz", + "integrity": "sha512-HfyAmqZi9uBAbgKYP1yGuI7tSREXwIb438q0nqvlpxAOs3XnZ8RsisRfmVsgV486NdjD7Mw2UrFSw51lzUk1ww==", "cpu": [ "riscv64" ], "dev": true, + "license": "MIT", "optional": true, "os": [ "linux" ], "engines": { - "node": ">=12" + "node": ">=18" } }, "node_modules/@esbuild/linux-s390x": { - "version": "0.18.20", - "resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.18.20.tgz", - "integrity": "sha512-+8231GMs3mAEth6Ja1iK0a1sQ3ohfcpzpRLH8uuc5/KVDFneH6jtAJLFGafpzpMRO6DzJ6AvXKze9LfFMrIHVQ==", + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.25.11.tgz", + "integrity": "sha512-HjLqVgSSYnVXRisyfmzsH6mXqyvj0SA7pG5g+9W7ESgwA70AXYNpfKBqh1KbTxmQVaYxpzA/SvlB9oclGPbApw==", "cpu": [ "s390x" ], "dev": true, + "license": "MIT", "optional": true, "os": [ "linux" ], "engines": { - "node": ">=12" + "node": ">=18" } }, "node_modules/@esbuild/linux-x64": { - "version": "0.18.20", - "resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.18.20.tgz", - "integrity": "sha512-UYqiqemphJcNsFEskc73jQ7B9jgwjWrSayxawS6UVFZGWrAAtkzjxSqnoclCXxWtfwLdzU+vTpcNYhpn43uP1w==", + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.25.11.tgz", + "integrity": "sha512-HSFAT4+WYjIhrHxKBwGmOOSpphjYkcswF449j6EjsjbinTZbp8PJtjsVK1XFJStdzXdy/jaddAep2FGY+wyFAQ==", "cpu": [ "x64" ], "dev": true, + "license": "MIT", "optional": true, "os": [ "linux" ], "engines": { - "node": ">=12" + "node": ">=18" + } + }, + "node_modules/@esbuild/netbsd-arm64": { + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.25.11.tgz", + "integrity": "sha512-hr9Oxj1Fa4r04dNpWr3P8QKVVsjQhqrMSUzZzf+LZcYjZNqhA3IAfPQdEh1FLVUJSiu6sgAwp3OmwBfbFgG2Xg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=18" } }, "node_modules/@esbuild/netbsd-x64": { - "version": "0.18.20", - "resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.18.20.tgz", - "integrity": "sha512-iO1c++VP6xUBUmltHZoMtCUdPlnPGdBom6IrO4gyKPFFVBKioIImVooR5I83nTew5UOYrk3gIJhbZh8X44y06A==", + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.25.11.tgz", + "integrity": "sha512-u7tKA+qbzBydyj0vgpu+5h5AeudxOAGncb8N6C9Kh1N4n7wU1Xw1JDApsRjpShRpXRQlJLb9wY28ELpwdPcZ7A==", "cpu": [ "x64" ], "dev": true, + "license": "MIT", "optional": true, "os": [ "netbsd" ], "engines": { - "node": ">=12" + "node": ">=18" + } + }, + "node_modules/@esbuild/openbsd-arm64": { + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.25.11.tgz", + "integrity": "sha512-Qq6YHhayieor3DxFOoYM1q0q1uMFYb7cSpLD2qzDSvK1NAvqFi8Xgivv0cFC6J+hWVw2teCYltyy9/m/14ryHg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=18" } }, "node_modules/@esbuild/openbsd-x64": { - "version": "0.18.20", - "resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.18.20.tgz", - "integrity": "sha512-e5e4YSsuQfX4cxcygw/UCPIEP6wbIL+se3sxPdCiMbFLBWu0eiZOJ7WoD+ptCLrmjZBK1Wk7I6D/I3NglUGOxg==", + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.25.11.tgz", + "integrity": "sha512-CN+7c++kkbrckTOz5hrehxWN7uIhFFlmS/hqziSFVWpAzpWrQoAG4chH+nN3Be+Kzv/uuo7zhX716x3Sn2Jduw==", "cpu": [ "x64" ], "dev": true, + "license": "MIT", "optional": true, "os": [ "openbsd" ], "engines": { - "node": ">=12" + "node": ">=18" + } + }, + "node_modules/@esbuild/openharmony-arm64": { + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.25.11.tgz", + "integrity": "sha512-rOREuNIQgaiR+9QuNkbkxubbp8MSO9rONmwP5nKncnWJ9v5jQ4JxFnLu4zDSRPf3x4u+2VN4pM4RdyIzDty/wQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openharmony" + ], + "engines": { + "node": ">=18" } }, "node_modules/@esbuild/sunos-x64": { - "version": "0.18.20", - "resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.18.20.tgz", - "integrity": "sha512-kDbFRFp0YpTQVVrqUd5FTYmWo45zGaXe0X8E1G/LKFC0v8x0vWrhOWSLITcCn63lmZIxfOMXtCfti/RxN/0wnQ==", + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.25.11.tgz", + "integrity": "sha512-nq2xdYaWxyg9DcIyXkZhcYulC6pQ2FuCgem3LI92IwMgIZ69KHeY8T4Y88pcwoLIjbed8n36CyKoYRDygNSGhA==", "cpu": [ "x64" ], "dev": true, + "license": "MIT", "optional": true, "os": [ "sunos" ], "engines": { - "node": ">=12" + "node": ">=18" } }, "node_modules/@esbuild/win32-arm64": { - "version": "0.18.20", - "resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.18.20.tgz", - "integrity": "sha512-ddYFR6ItYgoaq4v4JmQQaAI5s7npztfV4Ag6NrhiaW0RrnOXqBkgwZLofVTlq1daVTQNhtI5oieTvkRPfZrePg==", + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.25.11.tgz", + "integrity": "sha512-3XxECOWJq1qMZ3MN8srCJ/QfoLpL+VaxD/WfNRm1O3B4+AZ/BnLVgFbUV3eiRYDMXetciH16dwPbbHqwe1uU0Q==", "cpu": [ "arm64" ], "dev": true, + "license": "MIT", "optional": true, "os": [ "win32" ], "engines": { - "node": ">=12" + "node": ">=18" } }, "node_modules/@esbuild/win32-ia32": { - "version": "0.18.20", - "resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.18.20.tgz", - "integrity": "sha512-Wv7QBi3ID/rROT08SABTS7eV4hX26sVduqDOTe1MvGMjNd3EjOz4b7zeexIR62GTIEKrfJXKL9LFxTYgkyeu7g==", + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.25.11.tgz", + "integrity": "sha512-3ukss6gb9XZ8TlRyJlgLn17ecsK4NSQTmdIXRASVsiS2sQ6zPPZklNJT5GR5tE/MUarymmy8kCEf5xPCNCqVOA==", "cpu": [ "ia32" ], "dev": true, + "license": "MIT", "optional": true, "os": [ "win32" ], "engines": { - "node": ">=12" + "node": ">=18" } }, "node_modules/@esbuild/win32-x64": { - "version": "0.18.20", - "resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.18.20.tgz", - "integrity": "sha512-kTdfRcSiDfQca/y9QIkng02avJ+NCaQvrMejlsB3RRv5sE9rRoeBPISaZpKxHELzRxZyLvNts1P27W3wV+8geQ==", + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.25.11.tgz", + "integrity": "sha512-D7Hpz6A2L4hzsRpPaCYkQnGOotdUpDzSGRIv9I+1ITdHROSFUWW95ZPZWQmGka1Fg7W3zFJowyn9WGwMJ0+KPA==", "cpu": [ "x64" ], "dev": true, + "license": "MIT", "optional": true, "os": [ "win32" ], "engines": { - "node": ">=12" + "node": ">=18" } }, "node_modules/@loaders.gl/core": { @@ -453,12 +544,327 @@ "resolved": "https://registry.npmjs.org/@probe.gl/stats/-/stats-4.0.9.tgz", "integrity": "sha512-Q9Xt/sJUQaMsbjRKjOscv2t7wXIymTrOEJ4a3da4FTCn7bkKvcdxdyFAQySCrtPxE+YZ5I5lXpWPgv9BwmpE1g==" }, + "node_modules/@rollup/rollup-android-arm-eabi": { + "version": "4.52.5", + "resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm-eabi/-/rollup-android-arm-eabi-4.52.5.tgz", + "integrity": "sha512-8c1vW4ocv3UOMp9K+gToY5zL2XiiVw3k7f1ksf4yO1FlDFQ1C2u72iACFnSOceJFsWskc2WZNqeRhFRPzv+wtQ==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ] + }, + "node_modules/@rollup/rollup-android-arm64": { + "version": "4.52.5", + "resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm64/-/rollup-android-arm64-4.52.5.tgz", + "integrity": "sha512-mQGfsIEFcu21mvqkEKKu2dYmtuSZOBMmAl5CFlPGLY94Vlcm+zWApK7F/eocsNzp8tKmbeBP8yXyAbx0XHsFNA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ] + }, + "node_modules/@rollup/rollup-darwin-arm64": { + "version": "4.52.5", + "resolved": "https://registry.npmjs.org/@rollup/rollup-darwin-arm64/-/rollup-darwin-arm64-4.52.5.tgz", + "integrity": "sha512-takF3CR71mCAGA+v794QUZ0b6ZSrgJkArC+gUiG6LB6TQty9T0Mqh3m2ImRBOxS2IeYBo4lKWIieSvnEk2OQWA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ] + }, + "node_modules/@rollup/rollup-darwin-x64": { + "version": "4.52.5", + "resolved": "https://registry.npmjs.org/@rollup/rollup-darwin-x64/-/rollup-darwin-x64-4.52.5.tgz", + "integrity": "sha512-W901Pla8Ya95WpxDn//VF9K9u2JbocwV/v75TE0YIHNTbhqUTv9w4VuQ9MaWlNOkkEfFwkdNhXgcLqPSmHy0fA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ] + }, + "node_modules/@rollup/rollup-freebsd-arm64": { + "version": "4.52.5", + "resolved": "https://registry.npmjs.org/@rollup/rollup-freebsd-arm64/-/rollup-freebsd-arm64-4.52.5.tgz", + "integrity": "sha512-QofO7i7JycsYOWxe0GFqhLmF6l1TqBswJMvICnRUjqCx8b47MTo46W8AoeQwiokAx3zVryVnxtBMcGcnX12LvA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ] + }, + "node_modules/@rollup/rollup-freebsd-x64": { + "version": "4.52.5", + "resolved": "https://registry.npmjs.org/@rollup/rollup-freebsd-x64/-/rollup-freebsd-x64-4.52.5.tgz", + "integrity": "sha512-jr21b/99ew8ujZubPo9skbrItHEIE50WdV86cdSoRkKtmWa+DDr6fu2c/xyRT0F/WazZpam6kk7IHBerSL7LDQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ] + }, + "node_modules/@rollup/rollup-linux-arm-gnueabihf": { + "version": "4.52.5", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm-gnueabihf/-/rollup-linux-arm-gnueabihf-4.52.5.tgz", + "integrity": "sha512-PsNAbcyv9CcecAUagQefwX8fQn9LQ4nZkpDboBOttmyffnInRy8R8dSg6hxxl2Re5QhHBf6FYIDhIj5v982ATQ==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-arm-musleabihf": { + "version": "4.52.5", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm-musleabihf/-/rollup-linux-arm-musleabihf-4.52.5.tgz", + "integrity": "sha512-Fw4tysRutyQc/wwkmcyoqFtJhh0u31K+Q6jYjeicsGJJ7bbEq8LwPWV/w0cnzOqR2m694/Af6hpFayLJZkG2VQ==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-arm64-gnu": { + "version": "4.52.5", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm64-gnu/-/rollup-linux-arm64-gnu-4.52.5.tgz", + "integrity": "sha512-a+3wVnAYdQClOTlyapKmyI6BLPAFYs0JM8HRpgYZQO02rMR09ZcV9LbQB+NL6sljzG38869YqThrRnfPMCDtZg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-arm64-musl": { + "version": "4.52.5", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm64-musl/-/rollup-linux-arm64-musl-4.52.5.tgz", + "integrity": "sha512-AvttBOMwO9Pcuuf7m9PkC1PUIKsfaAJ4AYhy944qeTJgQOqJYJ9oVl2nYgY7Rk0mkbsuOpCAYSs6wLYB2Xiw0Q==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-loong64-gnu": { + "version": "4.52.5", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-loong64-gnu/-/rollup-linux-loong64-gnu-4.52.5.tgz", + "integrity": "sha512-DkDk8pmXQV2wVrF6oq5tONK6UHLz/XcEVow4JTTerdeV1uqPeHxwcg7aFsfnSm9L+OO8WJsWotKM2JJPMWrQtA==", + "cpu": [ + "loong64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-ppc64-gnu": { + "version": "4.52.5", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-ppc64-gnu/-/rollup-linux-ppc64-gnu-4.52.5.tgz", + "integrity": "sha512-W/b9ZN/U9+hPQVvlGwjzi+Wy4xdoH2I8EjaCkMvzpI7wJUs8sWJ03Rq96jRnHkSrcHTpQe8h5Tg3ZzUPGauvAw==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-riscv64-gnu": { + "version": "4.52.5", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-riscv64-gnu/-/rollup-linux-riscv64-gnu-4.52.5.tgz", + "integrity": "sha512-sjQLr9BW7R/ZiXnQiWPkErNfLMkkWIoCz7YMn27HldKsADEKa5WYdobaa1hmN6slu9oWQbB6/jFpJ+P2IkVrmw==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-riscv64-musl": { + "version": "4.52.5", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-riscv64-musl/-/rollup-linux-riscv64-musl-4.52.5.tgz", + "integrity": "sha512-hq3jU/kGyjXWTvAh2awn8oHroCbrPm8JqM7RUpKjalIRWWXE01CQOf/tUNWNHjmbMHg/hmNCwc/Pz3k1T/j/Lg==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-s390x-gnu": { + "version": "4.52.5", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-s390x-gnu/-/rollup-linux-s390x-gnu-4.52.5.tgz", + "integrity": "sha512-gn8kHOrku8D4NGHMK1Y7NA7INQTRdVOntt1OCYypZPRt6skGbddska44K8iocdpxHTMMNui5oH4elPH4QOLrFQ==", + "cpu": [ + "s390x" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-x64-gnu": { + "version": "4.52.5", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-x64-gnu/-/rollup-linux-x64-gnu-4.52.5.tgz", + "integrity": "sha512-hXGLYpdhiNElzN770+H2nlx+jRog8TyynpTVzdlc6bndktjKWyZyiCsuDAlpd+j+W+WNqfcyAWz9HxxIGfZm1Q==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-x64-musl": { + "version": "4.52.5", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-x64-musl/-/rollup-linux-x64-musl-4.52.5.tgz", + "integrity": "sha512-arCGIcuNKjBoKAXD+y7XomR9gY6Mw7HnFBv5Rw7wQRvwYLR7gBAgV7Mb2QTyjXfTveBNFAtPt46/36vV9STLNg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-openharmony-arm64": { + "version": "4.52.5", + "resolved": "https://registry.npmjs.org/@rollup/rollup-openharmony-arm64/-/rollup-openharmony-arm64-4.52.5.tgz", + "integrity": "sha512-QoFqB6+/9Rly/RiPjaomPLmR/13cgkIGfA40LHly9zcH1S0bN2HVFYk3a1eAyHQyjs3ZJYlXvIGtcCs5tko9Cw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openharmony" + ] + }, + "node_modules/@rollup/rollup-win32-arm64-msvc": { + "version": "4.52.5", + "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-arm64-msvc/-/rollup-win32-arm64-msvc-4.52.5.tgz", + "integrity": "sha512-w0cDWVR6MlTstla1cIfOGyl8+qb93FlAVutcor14Gf5Md5ap5ySfQ7R9S/NjNaMLSFdUnKGEasmVnu3lCMqB7w==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, + "node_modules/@rollup/rollup-win32-ia32-msvc": { + "version": "4.52.5", + "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-ia32-msvc/-/rollup-win32-ia32-msvc-4.52.5.tgz", + "integrity": "sha512-Aufdpzp7DpOTULJCuvzqcItSGDH73pF3ko/f+ckJhxQyHtp67rHw3HMNxoIdDMUITJESNE6a8uh4Lo4SLouOUg==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, + "node_modules/@rollup/rollup-win32-x64-gnu": { + "version": "4.52.5", + "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-x64-gnu/-/rollup-win32-x64-gnu-4.52.5.tgz", + "integrity": "sha512-UGBUGPFp1vkj6p8wCRraqNhqwX/4kNQPS57BCFc8wYh0g94iVIW33wJtQAx3G7vrjjNtRaxiMUylM0ktp/TRSQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, + "node_modules/@rollup/rollup-win32-x64-msvc": { + "version": "4.52.5", + "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-x64-msvc/-/rollup-win32-x64-msvc-4.52.5.tgz", + "integrity": "sha512-TAcgQh2sSkykPRWLrdyy2AiceMckNf5loITqXxFI5VuQjS5tSuw3WlwdN8qv8vzjLAUTvYaH/mVjSFpbkFbpTg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, "node_modules/@tweakpane/core": { "version": "1.1.9", "resolved": "https://registry.npmjs.org/@tweakpane/core/-/core-1.1.9.tgz", "integrity": "sha512-9tq+KAhaqPiOgsFyLPAz1IMXkVfhRqxGzAgy1ps3As6o3W7XjnU7sev6OlD/Z+Pzw8uZVMukkSHf2e0uCU6u0A==", "dev": true }, + "node_modules/@types/estree": { + "version": "1.0.8", + "resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.8.tgz", + "integrity": "sha512-dWHzHa2WqEXI/O1E9OjrocMTKJl2mSrEolh1Iomrv6U+JuNwaHXsXx9bLu5gG7BUWFIN0skIQJQ/L1rIex4X6w==", + "dev": true, + "license": "MIT" + }, "node_modules/@types/geojson": { "version": "7946.0.14", "resolved": "https://registry.npmjs.org/@types/geojson/-/geojson-7946.0.14.tgz", @@ -471,40 +877,63 @@ "dev": true }, "node_modules/esbuild": { - "version": "0.18.20", - "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.18.20.tgz", - "integrity": "sha512-ceqxoedUrcayh7Y7ZX6NdbbDzGROiyVBgC4PriJThBKSVPWnnFHZAkfI1lJT8QFkOwH4qOS2SJkS4wvpGl8BpA==", + "version": "0.25.11", + "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.25.11.tgz", + "integrity": "sha512-KohQwyzrKTQmhXDW1PjCv3Tyspn9n5GcY2RTDqeORIdIJY8yKIF7sTSopFmn/wpMPW4rdPXI0UE5LJLuq3bx0Q==", "dev": true, "hasInstallScript": true, + "license": "MIT", "bin": { "esbuild": "bin/esbuild" }, "engines": { - "node": ">=12" + "node": ">=18" }, "optionalDependencies": { - "@esbuild/android-arm": "0.18.20", - "@esbuild/android-arm64": "0.18.20", - "@esbuild/android-x64": "0.18.20", - "@esbuild/darwin-arm64": "0.18.20", - "@esbuild/darwin-x64": "0.18.20", - "@esbuild/freebsd-arm64": "0.18.20", - "@esbuild/freebsd-x64": "0.18.20", - "@esbuild/linux-arm": "0.18.20", - "@esbuild/linux-arm64": "0.18.20", - "@esbuild/linux-ia32": "0.18.20", - "@esbuild/linux-loong64": "0.18.20", - "@esbuild/linux-mips64el": "0.18.20", - "@esbuild/linux-ppc64": "0.18.20", - "@esbuild/linux-riscv64": "0.18.20", - "@esbuild/linux-s390x": "0.18.20", - "@esbuild/linux-x64": "0.18.20", - "@esbuild/netbsd-x64": "0.18.20", - "@esbuild/openbsd-x64": "0.18.20", - "@esbuild/sunos-x64": "0.18.20", - "@esbuild/win32-arm64": "0.18.20", - "@esbuild/win32-ia32": "0.18.20", - "@esbuild/win32-x64": "0.18.20" + "@esbuild/aix-ppc64": "0.25.11", + "@esbuild/android-arm": "0.25.11", + "@esbuild/android-arm64": "0.25.11", + "@esbuild/android-x64": "0.25.11", + "@esbuild/darwin-arm64": "0.25.11", + "@esbuild/darwin-x64": "0.25.11", + "@esbuild/freebsd-arm64": "0.25.11", + "@esbuild/freebsd-x64": "0.25.11", + "@esbuild/linux-arm": "0.25.11", + "@esbuild/linux-arm64": "0.25.11", + "@esbuild/linux-ia32": "0.25.11", + "@esbuild/linux-loong64": "0.25.11", + "@esbuild/linux-mips64el": "0.25.11", + "@esbuild/linux-ppc64": "0.25.11", + "@esbuild/linux-riscv64": "0.25.11", + "@esbuild/linux-s390x": "0.25.11", + "@esbuild/linux-x64": "0.25.11", + "@esbuild/netbsd-arm64": "0.25.11", + "@esbuild/netbsd-x64": "0.25.11", + "@esbuild/openbsd-arm64": "0.25.11", + "@esbuild/openbsd-x64": "0.25.11", + "@esbuild/openharmony-arm64": "0.25.11", + "@esbuild/sunos-x64": "0.25.11", + "@esbuild/win32-arm64": "0.25.11", + "@esbuild/win32-ia32": "0.25.11", + "@esbuild/win32-x64": "0.25.11" + } + }, + "node_modules/fdir": { + "version": "6.5.0", + "resolved": "https://registry.npmjs.org/fdir/-/fdir-6.5.0.tgz", + "integrity": "sha512-tIbYtZbucOs0BRGqPJkshJUYdL+SDH7dVM8gjy+ERp3WAUjLEFJE+02kanyHtwjWOnwrKYBiwAmM0p4kLJAnXg==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=12.0.0" + }, + "peerDependencies": { + "picomatch": "^3 || ^4" + }, + "peerDependenciesMeta": { + "picomatch": { + "optional": true + } } }, "node_modules/fsevents": { @@ -513,6 +942,7 @@ "integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==", "dev": true, "hasInstallScript": true, + "license": "MIT", "optional": true, "os": [ "darwin" @@ -522,9 +952,9 @@ } }, "node_modules/nanoid": { - "version": "3.3.7", - "resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.7.tgz", - "integrity": "sha512-eSRppjcPIatRIMC1U6UngP8XFcz8MQWGQdt1MTBQ7NaAmvXDfvNxbvWV3x2y6CdEUciCSsDHDQZbhYaB8QEo2g==", + "version": "3.3.11", + "resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.11.tgz", + "integrity": "sha512-N8SpfPUnUp1bK+PMYW8qSWdl9U+wwNWI4QKxOYDy9JAro3WMX7p2OeVRF9v+347pnakNevPmiHhNmZ2HbFA76w==", "dev": true, "funding": [ { @@ -532,6 +962,7 @@ "url": "https://github.com/sponsors/ai" } ], + "license": "MIT", "bin": { "nanoid": "bin/nanoid.cjs" }, @@ -540,15 +971,29 @@ } }, "node_modules/picocolors": { - "version": "1.0.1", - "resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.0.1.tgz", - "integrity": "sha512-anP1Z8qwhkbmu7MFP5iTt+wQKXgwzf7zTyGlcdzabySa9vd0Xt392U0rVmz9poOaBj0uHJKyyo9/upk0HrEQew==", - "dev": true + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.1.1.tgz", + "integrity": "sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA==", + "dev": true, + "license": "ISC" + }, + "node_modules/picomatch": { + "version": "4.0.3", + "resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.3.tgz", + "integrity": "sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=12" + }, + "funding": { + "url": "https://github.com/sponsors/jonschlinkert" + } }, "node_modules/postcss": { - "version": "8.4.41", - "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.4.41.tgz", - "integrity": "sha512-TesUflQ0WKZqAvg52PWL6kHgLKP6xB6heTOdoYM0Wt2UHyxNa4K25EZZMgKns3BH1RLVbZCREPpLY0rhnNoHVQ==", + "version": "8.5.6", + "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.6.tgz", + "integrity": "sha512-3Ybi1tAuwAP9s0r1UQ2J4n5Y0G05bJkpUIO0/bI9MhwmD70S5aTWbXGBwxHrelT+XM1k6dM0pk+SwNkpTRN7Pg==", "dev": true, "funding": [ { @@ -564,40 +1009,85 @@ "url": "https://github.com/sponsors/ai" } ], + "license": "MIT", "dependencies": { - "nanoid": "^3.3.7", - "picocolors": "^1.0.1", - "source-map-js": "^1.2.0" + "nanoid": "^3.3.11", + "picocolors": "^1.1.1", + "source-map-js": "^1.2.1" }, "engines": { "node": "^10 || ^12 || >=14" } }, "node_modules/rollup": { - "version": "3.29.5", - "resolved": "https://registry.npmjs.org/rollup/-/rollup-3.29.5.tgz", - "integrity": "sha512-GVsDdsbJzzy4S/v3dqWPJ7EfvZJfCHiDqe80IyrF59LYuP+e6U1LJoUqeuqRbwAWoMNoXivMNeNAOf5E22VA1w==", + "version": "4.52.5", + "resolved": "https://registry.npmjs.org/rollup/-/rollup-4.52.5.tgz", + "integrity": "sha512-3GuObel8h7Kqdjt0gxkEzaifHTqLVW56Y/bjN7PSQtkKr0w3V/QYSdt6QWYtd7A1xUtYQigtdUfgj1RvWVtorw==", "dev": true, + "license": "MIT", + "dependencies": { + "@types/estree": "1.0.8" + }, "bin": { "rollup": "dist/bin/rollup" }, "engines": { - "node": ">=14.18.0", + "node": ">=18.0.0", "npm": ">=8.0.0" }, "optionalDependencies": { + "@rollup/rollup-android-arm-eabi": "4.52.5", + "@rollup/rollup-android-arm64": "4.52.5", + "@rollup/rollup-darwin-arm64": "4.52.5", + "@rollup/rollup-darwin-x64": "4.52.5", + "@rollup/rollup-freebsd-arm64": "4.52.5", + "@rollup/rollup-freebsd-x64": "4.52.5", + "@rollup/rollup-linux-arm-gnueabihf": "4.52.5", + "@rollup/rollup-linux-arm-musleabihf": "4.52.5", + "@rollup/rollup-linux-arm64-gnu": "4.52.5", + "@rollup/rollup-linux-arm64-musl": "4.52.5", + "@rollup/rollup-linux-loong64-gnu": "4.52.5", + "@rollup/rollup-linux-ppc64-gnu": "4.52.5", + "@rollup/rollup-linux-riscv64-gnu": "4.52.5", + "@rollup/rollup-linux-riscv64-musl": "4.52.5", + "@rollup/rollup-linux-s390x-gnu": "4.52.5", + "@rollup/rollup-linux-x64-gnu": "4.52.5", + "@rollup/rollup-linux-x64-musl": "4.52.5", + "@rollup/rollup-openharmony-arm64": "4.52.5", + "@rollup/rollup-win32-arm64-msvc": "4.52.5", + "@rollup/rollup-win32-ia32-msvc": "4.52.5", + "@rollup/rollup-win32-x64-gnu": "4.52.5", + "@rollup/rollup-win32-x64-msvc": "4.52.5", "fsevents": "~2.3.2" } }, "node_modules/source-map-js": { - "version": "1.2.0", - "resolved": "https://registry.npmjs.org/source-map-js/-/source-map-js-1.2.0.tgz", - "integrity": "sha512-itJW8lvSA0TXEphiRoawsCksnlf8SyvmFzIhltqAHluXd88pkCd+cXJVHTDwdCr0IzwptSm035IHQktUu1QUMg==", + "version": "1.2.1", + "resolved": "https://registry.npmjs.org/source-map-js/-/source-map-js-1.2.1.tgz", + "integrity": "sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA==", "dev": true, + "license": "BSD-3-Clause", "engines": { "node": ">=0.10.0" } }, + "node_modules/tinyglobby": { + "version": "0.2.15", + "resolved": "https://registry.npmjs.org/tinyglobby/-/tinyglobby-0.2.15.tgz", + "integrity": "sha512-j2Zq4NyQYG5XMST4cbs02Ak8iJUdxRM0XI5QyxXuZOzKOINmWurp3smXu3y5wDcJrptwpSjgXHzIQxR0omXljQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "fdir": "^6.5.0", + "picomatch": "^4.0.3" + }, + "engines": { + "node": ">=12.0.0" + }, + "funding": { + "url": "https://github.com/sponsors/SuperchupuDev" + } + }, "node_modules/tweakpane": { "version": "3.1.10", "resolved": "https://registry.npmjs.org/tweakpane/-/tweakpane-3.1.10.tgz", @@ -629,40 +1119,51 @@ } }, "node_modules/vite": { - "version": "4.5.5", - "resolved": "https://registry.npmjs.org/vite/-/vite-4.5.5.tgz", - "integrity": "sha512-ifW3Lb2sMdX+WU91s3R0FyQlAyLxOzCSCP37ujw0+r5POeHPwe6udWVIElKQq8gk3t7b8rkmvqC6IHBpCff4GQ==", + "version": "7.1.11", + "resolved": "https://registry.npmjs.org/vite/-/vite-7.1.11.tgz", + "integrity": "sha512-uzcxnSDVjAopEUjljkWh8EIrg6tlzrjFUfMcR1EVsRDGwf/ccef0qQPRyOrROwhrTDaApueq+ja+KLPlzR/zdg==", "dev": true, + "license": "MIT", "dependencies": { - "esbuild": "^0.18.10", - "postcss": "^8.4.27", - "rollup": "^3.27.1" + "esbuild": "^0.25.0", + "fdir": "^6.5.0", + "picomatch": "^4.0.3", + "postcss": "^8.5.6", + "rollup": "^4.43.0", + "tinyglobby": "^0.2.15" }, "bin": { "vite": "bin/vite.js" }, "engines": { - "node": "^14.18.0 || >=16.0.0" + "node": "^20.19.0 || >=22.12.0" }, "funding": { "url": "https://github.com/vitejs/vite?sponsor=1" }, "optionalDependencies": { - "fsevents": "~2.3.2" + "fsevents": "~2.3.3" }, "peerDependencies": { - "@types/node": ">= 14", - "less": "*", + "@types/node": "^20.19.0 || >=22.12.0", + "jiti": ">=1.21.0", + "less": "^4.0.0", "lightningcss": "^1.21.0", - "sass": "*", - "stylus": "*", - "sugarss": "*", - "terser": "^5.4.0" + "sass": "^1.70.0", + "sass-embedded": "^1.70.0", + "stylus": ">=0.54.8", + "sugarss": "^5.0.0", + "terser": "^5.16.0", + "tsx": "^4.8.1", + "yaml": "^2.4.2" }, "peerDependenciesMeta": { "@types/node": { "optional": true }, + "jiti": { + "optional": true + }, "less": { "optional": true }, @@ -672,6 +1173,9 @@ "sass": { "optional": true }, + "sass-embedded": { + "optional": true + }, "stylus": { "optional": true }, @@ -680,6 +1184,12 @@ }, "terser": { "optional": true + }, + "tsx": { + "optional": true + }, + "yaml": { + "optional": true } } }, diff --git a/package.json b/package.json index 8a751f0..71f052b 100644 --- a/package.json +++ b/package.json @@ -9,10 +9,10 @@ }, "devDependencies": { "@tweakpane/core": "^1.1.7", - "tweakpane-plugin-file-import": "^0.2.0", "@webgpu/types": "^0.1.31", + "tweakpane-plugin-file-import": "^0.2.0", "typescript": "^5.0.4", - "vite": "^4.3.1", + "vite": "^7.1.11", "vite-raw-plugin": "^1.0.1" }, "dependencies": { diff --git a/src/renderers/gaussian-renderer.ts b/src/renderers/gaussian-renderer.ts index 1684523..982392c 100644 --- a/src/renderers/gaussian-renderer.ts +++ b/src/renderers/gaussian-renderer.ts @@ -1,11 +1,11 @@ import { PointCloud } from '../utils/load'; import preprocessWGSL from '../shaders/preprocess.wgsl'; import renderWGSL from '../shaders/gaussian.wgsl'; -import { get_sorter,c_histogram_block_rows,C } from '../sort/sort'; +import { get_sorter, c_histogram_block_rows, C } from '../sort/sort'; import { Renderer } from './renderer'; export interface GaussianRenderer extends Renderer { - + render_settings_buffer: GPUBuffer } // Utility to create GPU buffers @@ -29,13 +29,50 @@ export default function get_renderer( ): GaussianRenderer { const sorter = get_sorter(pc.num_points, device); - + // =============================================== // Initialize GPU Buffers // =============================================== const nulling_data = new Uint32Array([0]); + // Create null buffer. + const null_buffer = createBuffer( + device, + 'null_buffer', + 4, + GPUBufferUsage.COPY_SRC | GPUBufferUsage.COPY_DST, nulling_data + ); + + // Create indirect buffer. + const indirect_buffer = createBuffer( + device, + 'indirect_buffer', + 4 * 4, + GPUBufferUsage.COPY_DST | GPUBufferUsage.INDIRECT, + new Uint32Array([6, pc.num_points, 0, 0]) + ); + + // Create splat buffer. + const splatBufferSize = pc.num_points * 64; + + const splat_buffer = createBuffer( + device, + 'splat_buffer', + splatBufferSize, + GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_SRC | GPUBufferUsage.COPY_DST + ) + + // Render settings buffer? + const renderSettings_size = new Float32Array([1.0, pc.sh_deg, 0.0, 0.0]); + const render_settings_buffer = createBuffer( + device, + 'render_settings', + renderSettings_size.byteLength, + GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST, + renderSettings_size + ) + // =============================================== // Create Compute Pipeline and Bind Groups // =============================================== @@ -52,9 +89,22 @@ export default function get_renderer( }, }); + // Holds buffers for splat, camera+gaussian, etc. + const compute_bind_group = device.createBindGroup({ + label: 'Compute Bind Group', + layout: preprocess_pipeline.getBindGroupLayout(0), + entries: [ + { binding: 0, resource: { buffer: camera_buffer } }, + { binding: 1, resource: { buffer: pc.gaussian_3d_buffer } }, + { binding: 2, resource: { buffer: splat_buffer } }, + { binding: 3, resource: { buffer: render_settings_buffer } }, + { binding: 4, resource: { buffer: pc.sh_buffer } }, + ], + }); + const sort_bind_group = device.createBindGroup({ label: 'sort', - layout: preprocess_pipeline.getBindGroupLayout(2), + layout: preprocess_pipeline.getBindGroupLayout(1), entries: [ { binding: 0, resource: { buffer: sorter.sort_info_buffer } }, { binding: 1, resource: { buffer: sorter.ping_pong[0].sort_depths_buffer } }, @@ -63,24 +113,134 @@ export default function get_renderer( ], }); - // =============================================== // Create Render Pipeline and Bind Groups // =============================================== - + const render_pipeline = device.createRenderPipeline({ + label: "render pipeline", + layout: "auto", + vertex: { + module: device.createShaderModule({ + label: "vert shader", + code: renderWGSL + }), + entryPoint: "vs_main", + buffers: [] + }, + fragment: { + module: device.createShaderModule({ + label: "frag shader", + code: renderWGSL + }), + targets: [{ + format: presentation_format, + blend: { + color: { + srcFactor: "one", + dstFactor: "one-minus-src-alpha", + operation: "add" + }, + alpha: { + srcFactor: "one", + dstFactor: "one-minus-src-alpha", + operation: "add" + } + }, + }], + entryPoint: "fs_main" + } + }); + + // create the render pipeline bind group for all other resources + const render_pipeline_bind_group = device.createBindGroup({ + label: 'render_pipeline_bind_group', + layout: render_pipeline.getBindGroupLayout(0), + entries: [ + + // declare a new entry for the splat data buffer + { binding: 0, resource: { buffer: splat_buffer, } }, + { binding: 1, resource: { buffer: sorter.ping_pong[0].sort_indices_buffer } }, + { binding: 2, resource: { buffer: camera_buffer } }, + ], + }); // =============================================== // Command Encoder Functions // =============================================== - + // Compute pass first for preprocessing (?). + const compute_pass = (encoder: GPUCommandEncoder) => { + const preprocess_compute_pass = encoder.beginComputePass(); + + // Bind preprocess. + preprocess_compute_pass.setPipeline(preprocess_pipeline); + + // Set bind groups. + preprocess_compute_pass.setBindGroup(0, compute_bind_group); + preprocess_compute_pass.setBindGroup(1, sort_bind_group); + + // Dispatch work groups. + preprocess_compute_pass.dispatchWorkgroups(Math.ceil(pc.num_points / C.histogram_wg_size)); + + // End preprocess pass. + preprocess_compute_pass.end(); + }; + + const render_pass = (encoder: GPUCommandEncoder, texture_view: GPUTextureView) => { + const gaussian_render_pass = encoder.beginRenderPass({ + label: 'gaussian render pass', + colorAttachments: [ + { + view: texture_view, + loadOp: "clear", + storeOp: "store", + clearValue: [ + 0.0, 0.0, 0.0, 1.0 + ] + }] + }); + // Somewhere here set pipeline and draw indirect buffer. + gaussian_render_pass.setPipeline(render_pipeline); + gaussian_render_pass.setBindGroup(0, render_pipeline_bind_group); + gaussian_render_pass.drawIndirect(indirect_buffer, 0); + gaussian_render_pass.end(); + }; + + // Render pass next. // =============================================== // Return Render Object // =============================================== return { frame: (encoder: GPUCommandEncoder, texture_view: GPUTextureView) => { + encoder.copyBufferToBuffer( + null_buffer, 0, + sorter.sort_info_buffer, 0, + 4 + ); + encoder.copyBufferToBuffer( + null_buffer, 0, + sorter.sort_dispatch_indirect_buffer, 0, + 4 + ); + + // Preprocess first. + compute_pass(encoder); + sorter.sort(encoder); + + encoder.copyBufferToBuffer( + sorter.sort_info_buffer, + 0, + indirect_buffer, + 4, + 4 + ); + + // New render pass. + // We return the render pass. + render_pass(encoder, texture_view); }, camera_buffer, + render_settings_buffer }; } diff --git a/src/renderers/renderer.ts b/src/renderers/renderer.ts index ffdf9ba..7581b0e 100644 --- a/src/renderers/renderer.ts +++ b/src/renderers/renderer.ts @@ -3,7 +3,7 @@ import { Pane } from 'tweakpane'; import * as TweakpaneFileImportPlugin from 'tweakpane-plugin-file-import'; import { default as get_renderer_gaussian, GaussianRenderer } from './gaussian-renderer'; import { default as get_renderer_pointcloud } from './point-cloud-renderer'; -import { Camera, load_camera_presets} from '../camera/camera'; +import { Camera, load_camera_presets } from '../camera/camera'; import { CameraControl } from '../camera/camera-control'; import { time, timeReturn } from '../utils/simple-console'; @@ -17,14 +17,14 @@ export default async function init( context: GPUCanvasContext, device: GPUDevice ) { - let ply_file_loaded = false; - let cam_file_loaded = false; + let ply_file_loaded = false; + let cam_file_loaded = false; let renderers: { pointcloud?: Renderer, gaussian?: Renderer } = {}; - let gaussian_renderer: GaussianRenderer | undefined; - let pointcloud_renderer: Renderer | undefined; - let renderer: Renderer | undefined; + let gaussian_renderer: GaussianRenderer | undefined; + let pointcloud_renderer: Renderer | undefined; + let renderer: Renderer | undefined; let cameras; - + const camera = new Camera(canvas, device); const control = new CameraControl(camera); @@ -35,14 +35,14 @@ export default async function init( camera.on_update_canvas(); }); observer.observe(canvas); - + const presentation_format = navigator.gpu.getPreferredCanvasFormat(); context.configure({ device, format: presentation_format, alphaMode: 'opaque', }); - + // Tweakpane: easily adding tweak control for parameters. const params = { @@ -60,7 +60,7 @@ export default async function init( pane.registerPlugin(TweakpaneFileImportPlugin); { pane.addMonitor(params, 'fps', { - readonly:true + readonly: true }); } { @@ -80,22 +80,22 @@ export default async function init( filetypes: ['.ply'], invalidFiletypeMessage: "We can't accept those filetypes!" }) - .on('change', async (file) => { - const uploadedFile = file.value; - if (uploadedFile) { - const pc = await load(uploadedFile, device); - pointcloud_renderer = get_renderer_pointcloud(pc, device, presentation_format, camera.uniform_buffer); - gaussian_renderer = get_renderer_gaussian(pc, device, presentation_format, camera.uniform_buffer); - renderers = { - pointcloud: pointcloud_renderer, - gaussian: gaussian_renderer, - }; - renderer = renderers[params.renderer]; - ply_file_loaded = true; - }else{ - ply_file_loaded = false; - } - }); + .on('change', async (file) => { + const uploadedFile = file.value; + if (uploadedFile) { + const pc = await load(uploadedFile, device); + pointcloud_renderer = get_renderer_pointcloud(pc, device, presentation_format, camera.uniform_buffer); + gaussian_renderer = get_renderer_gaussian(pc, device, presentation_format, camera.uniform_buffer); + renderers = { + pointcloud: pointcloud_renderer, + gaussian: gaussian_renderer, + }; + renderer = renderers[params.renderer]; + ply_file_loaded = true; + } else { + ply_file_loaded = false; + } + }); } { pane.addInput(params, 'cam_file', { @@ -104,29 +104,38 @@ export default async function init( filetypes: ['.json'], invalidFiletypeMessage: "We can't accept those filetypes!" }) - .on('change', async (file) => { - const uploadedFile = file.value; - if (uploadedFile) { - cameras=await load_camera_presets(file.value); - camera.set_preset(cameras[0]); - cam_file_loaded = true; - }else{ - cam_file_loaded = false; - } - }); + .on('change', async (file) => { + const uploadedFile = file.value; + if (uploadedFile) { + cameras = await load_camera_presets(file.value); + camera.set_preset(cameras[0]); + cam_file_loaded = true; + } else { + cam_file_loaded = false; + } + }); } { pane.addInput( params, 'gaussian_multiplier', - {min: 0, max: 1.5} + { min: 0, max: 1.5 } ).on('change', (e) => { //TODO: Bind constants to the gaussian renderer. + if (gaussian_renderer) { + // Get render settings mult value. + device.queue.writeBuffer( + gaussian_renderer.render_settings_buffer, + 0, new Float32Array([ + params.gaussian_multiplier, + ]) + ); + } }); } document.addEventListener('keydown', (event) => { - switch(event.key) { + switch (event.key) { case '0': case '1': case '2': @@ -146,7 +155,7 @@ export default async function init( function frame() { if (ply_file_loaded && cam_file_loaded) { - params.fps=1.0/timeReturn()*1000.0; + params.fps = 1.0 / timeReturn() * 1000.0; time(); const encoder = device.createCommandEncoder(); const texture_view = context.getCurrentTexture().createView(); diff --git a/src/shaders/gaussian.wgsl b/src/shaders/gaussian.wgsl index 759226d..b5d5453 100644 --- a/src/shaders/gaussian.wgsl +++ b/src/shaders/gaussian.wgsl @@ -1,22 +1,109 @@ struct VertexOutput { @builtin(position) position: vec4, //TODO: information passed from vertex shader to fragment shader + @location(0) v_color: vec3, + @location(1) v_conic_opacity: vec4, + @location(2) v_center: vec2 +}; + +// Camera uniform struct defines. +struct CameraUniforms { + view: mat4x4, + view_inv: mat4x4, + proj: mat4x4, + proj_inv: mat4x4, + viewport: vec2, + focal: vec2 }; struct Splat { //TODO: information defined in preprocess compute shader + // Same as one in preprocess. + pos_ndc: u32, + size: u32, + color: vec3, + conic_opacity0: u32, + conic_opacity1: u32 }; +// Read in splats from storage buffer that was set up in preprocess. +@group(0) @binding(0) +var splats: array; +@group(0) @binding(1) +var sort_indices: array; +@group(0) @binding(2) +var camera: CameraUniforms; + +// Quad offsets. +const QUAD_OFFSETS = array, 6>( + vec2(-1.0, -1.0), + vec2( 1.0, -1.0), + vec2(-1.0, 1.0), + vec2(-1.0, 1.0), + vec2( 1.0, -1.0), + vec2( 1.0, 1.0), +); + @vertex fn vs_main( + @builtin(instance_index) instance_index: u32, + @builtin(vertex_index) vertex_index: u32 ) -> VertexOutput { //TODO: reconstruct 2D quad based on information from splat, pass + + // Obtain sorted index. + let sortedIndex = sort_indices[instance_index]; + // Read in current splat from sortedIndex. + let currSplat = splats[sortedIndex]; + + let posXY = unpack2x16float(currSplat.pos_ndc); + let size = unpack2x16float(currSplat.size); + + let conicXY = unpack2x16float(currSplat.conic_opacity0); + let conicZOpacity = unpack2x16float(currSplat.conic_opacity1); + + let depth_variance = conicZOpacity.x; + let opacity = conicZOpacity.y; + + var pos = vec4(posXY, 0.0, 1.0); + var col = currSplat.color; + + let offset = QUAD_OFFSETS[vertex_index] * size; + + let finalPos = vec4(pos.xy + offset, pos.z, pos.w); + + // Pass out VertexOutput params for frag shader. + let conicOpacity = vec4(conicXY.xy, conicZOpacity.xy); + var out: VertexOutput; - out.position = vec4(1. ,1. , 0., 1.); + out.position = finalPos; + out.v_color = col; + out.v_conic_opacity = conicOpacity; + out.v_center = vec2(posXY); return out; } @fragment fn fs_main(in: VertexOutput) -> @location(0) vec4 { - return vec4(1.); + // Calculate ndc position again. + var posNdc = (in.position.xy / camera.viewport) * 2.0 - 1.0; + posNdc.y = -posNdc.y; + + // Offset in ndc. + let offset = posNdc.xy - in.v_center.xy; + let offset_px = offset * camera.viewport * 0.5; + let A = in.v_conic_opacity.x; + let B = in.v_conic_opacity.y; + let C = in.v_conic_opacity.z; + + var power = A * offset_px.x * offset_px.x + C * offset_px.y * offset_px.y + 2.0 * B * offset_px.x * offset_px.y; + power *= -0.5; + + if (power > 0.0) { + return vec4(0.0); + } + + let alpha = clamp(in.v_conic_opacity.w * exp(power), 0.0, 1.0); + + return vec4(in.v_color * alpha, alpha); } \ No newline at end of file diff --git a/src/shaders/point_cloud.wgsl b/src/shaders/point_cloud.wgsl index 01dded1..617171e 100644 --- a/src/shaders/point_cloud.wgsl +++ b/src/shaders/point_cloud.wgsl @@ -35,7 +35,7 @@ fn vs_main( let pos = vec4(a.x, a.y, b.x, 1.); // TODO: MVP calculations - out.position = pos; + out.position = camera.proj * camera.view * pos; return out; } diff --git a/src/shaders/preprocess.wgsl b/src/shaders/preprocess.wgsl index bbc63f5..29206a6 100644 --- a/src/shaders/preprocess.wgsl +++ b/src/shaders/preprocess.wgsl @@ -57,22 +57,47 @@ struct Gaussian { struct Splat { //TODO: store information for 2D splat rendering + pos_ndc: u32, + size: u32, + color: vec3, + conic_opacity0: u32, + conic_opacity1: u32 }; //TODO: bind your data here -@group(2) @binding(0) + +@group(0) @binding(0) +var camera: CameraUniforms; +// Storage buffer for gaussians. +@group(0) @binding(1) +var gaussians: array; +@group(0) @binding(2) +var splatBuffer: array; +@group(0) @binding(3) +var settings: RenderSettings; +@group(0) @binding(4) +var sh_coeffs: array; + +@group(1) @binding(0) var sort_infos: SortInfos; -@group(2) @binding(1) +@group(1) @binding(1) var sort_depths : array; -@group(2) @binding(2) +@group(1) @binding(2) var sort_indices : array; -@group(2) @binding(3) +@group(1) @binding(3) var sort_dispatch: DispatchIndirect; /// reads the ith sh coef from the storage buffer fn sh_coef(splat_idx: u32, c_idx: u32) -> vec3 { //TODO: access your binded sh_coeff, see load.ts for how it is stored - return vec3(0.0); + let base_index = splat_idx * 24u + (c_idx / 2u) * 3u + (c_idx % 2u); + let color01 = unpack2x16float(sh_coeffs[base_index + 0u]); + let color23 = unpack2x16float(sh_coeffs[base_index + 1u]); + if (c_idx % 2u == 0u) { + return vec3(color01.x, color01.y, color23.x); + } else { + return vec3(color01.y, color23.x, color23.y); + } } // spherical harmonics evaluation with Condon–Shortley phase @@ -112,7 +137,191 @@ fn computeColorFromSH(dir: vec3, v_idx: u32, sh_deg: u32) -> vec3 { fn preprocess(@builtin(global_invocation_id) gid: vec3, @builtin(num_workgroups) wgs: vec3) { let idx = gid.x; //TODO: set up pipeline as described in instruction + // Length check for gaussians buffer. + if(idx >= arrayLength(&gaussians)){ + return; + } + + // Unpack Gaussian data. + let currGaussian = gaussians[idx]; + + let xy = unpack2x16float(currGaussian.pos_opacity[0]); + let z_opacity = unpack2x16float(currGaussian.pos_opacity[1]); + let position = vec3(xy, z_opacity.x); + let alpha = f32(z_opacity.y); + + // Project position to NDC. + let viewPos = (camera.view * vec4(position, 1.0f)).xyz; + let clipPos = camera.proj * camera.view * vec4(position, 1.0); + let posNdc = clipPos.xyz/clipPos.w; + + // Check if outside bounds, if yes, then cull. + if (any(posNdc.xy < vec2(-1.2)) || any(posNdc.xy > vec2(1.2)) || posNdc.z < 0.0 || posNdc.z > 1.0) { + return; + } + + // Compute 3D covariance. + + // Unpack rotation. + let rotationWX = unpack2x16float(currGaussian.rot[0]); + let rotationYZ = unpack2x16float(currGaussian.rot[1]); + + let rotation = vec4f( + rotationWX.x, + rotationWX.y, + rotationYZ.x, + rotationYZ.y + ); + + // Unpack scale. + let scaleXY = unpack2x16float(currGaussian.scale[0]); + let scaleZW = unpack2x16float(currGaussian.scale[1]); + + let scale = exp(vec3( + scaleXY.x, + scaleXY.y, + scaleZW.x + )); + + let r = rotation.x; + let x = rotation.y; + let y = rotation.z; + let z = rotation.w; + + // Compute R matrix. + let R = mat3x3( + 1.0 - 2.0 * (y * y + z * z), 2.0 * (x * y - r * z), 2.0 * (x * z + r * y), + 2.0 * (x * y + r * z), 1.0 - 2.0 * (x * x + z * z), 2.0 * (y * z - r * x), + 2.0 * (x * z - r * y), 2.0 * (y * z + r * x), 1.0 - 2.0 * (x * x + y * y) + ); + + let scaleSettings = settings.gaussian_scaling; + + // Construct S matrix. + let S = mat3x3( + vec3(scale.x * scaleSettings, 0.0, 0.0), + vec3(0.0, scale.y * scaleSettings, 0.0), + vec3(0.0, 0.0, scale.z * scaleSettings) + ); + + // Compute M matrix. + let M = S * R; + + // Compute 3D covariance matrix. + let covar_3D_M = transpose(M) * M; + + let covar_3D = array( + covar_3D_M[0][0], + covar_3D_M[0][1], + covar_3D_M[0][2], + covar_3D_M[1][1], + covar_3D_M[1][2], + covar_3D_M[2][2] + ); + + // Compute t vector. + var t = (camera.view * vec4(position, 1.0)).xyz; + let limx = 0.65 * camera.viewport.x / camera.focal.x; + let limy = 0.65 * camera.viewport.y / camera.focal.y; + let txtz = t.x / t.z; + let tytz = t.y / t.z; + t.x = min(limx, max(-limx, txtz)) * t.z; + t.y = min(limy, max(-limy, tytz)) * t.z; + + // Compute Jacobian. + let J = mat3x3( + camera.focal.x / t.z, 0.0f, -(camera.focal.x * t.x) / (t.z * t.z), + 0.0f, camera.focal.y / t.z, -(camera.focal.y * t.y) / (t.z * t.z), + 0.0f, 0.0f, 0.0f + ); + + // Compute W matrix. + let W = transpose(mat3x3( + camera.view[0].xyz, + camera.view[1].xyz, + camera.view[2].xyz + )); + + // Comute T matrix. + let T = W * J; + + // Get V matrix. + let V = mat3x3( + vec3(covar_3D[0], covar_3D[1], covar_3D[2]), + vec3(covar_3D[1], covar_3D[3], covar_3D[4]), + vec3(covar_3D[2], covar_3D[4], covar_3D[5]) + ); + + // Calculate 2D covariance matrix. + var covar_2D_M = transpose(T) * transpose(V) * T; + covar_2D_M[0][0] += 0.3f; + covar_2D_M[1][1] += 0.3f; + + // Compute 2D covariance. + let covar_2D = vec3( + covar_2D_M[0][0], + covar_2D_M[0][1], + covar_2D_M[1][1] + ); + + // Calculate determinant. + var determinant = covar_2D.x * covar_2D.z - covar_2D.y * covar_2D.y; + if (determinant == 0.0) { + return; + } + + // Calculate radius. + let mid = (covar_2D.x + covar_2D.z) * 0.5f; + let lambda1 = mid + sqrt(max(0.1f, mid * mid - determinant)); + let lambda2 = mid - sqrt(max(0.1f, mid * mid - determinant)); + let radius = ceil(3.0f * sqrt(max(lambda1, lambda2))); + + // Calculate size (please work...). + let size = vec2(2.0 * radius, 2.0 * radius) / camera.viewport; + + let testingSH = sh_coeffs[0]; + let sortIdx = atomicAdd(&sort_infos.keys_size, 1u); + + let sortDepth = sort_depths[0]; + let sortIndices = sort_indices[0]; + let sortDispatch = atomicAdd(&sort_dispatch.dispatch_x, 0u); + + // Pack stuff into new splat struct, to render in gaussian.wgsl. + let packedPosNdc = pack2x16float(posNdc.xy); + let packedSize = pack2x16float(size); + + // Compute conic. + let conic = vec3( + covar_2D.z / determinant, + covar_2D.y / determinant, + covar_2D.x / determinant, + ); + + let opacity_f = 1.0 / (1.0 + exp(-alpha)); + + let cam_pos = camera.view_inv[3].xyz; + + let splatCol = computeColorFromSH(normalize(cam_pos - position), idx, u32(settings.sh_deg)); + + splatBuffer[sortIdx].pos_ndc = packedPosNdc; + splatBuffer[sortIdx].size = packedSize; + + // let debug_color = vec3(abs(posNdc.x), abs(posNdc.y), posNdc.z); + // splatBuffer[sortIdx].color = vec3(radius / 10.0, 0.0, 0.0); + splatBuffer[sortIdx].color = splatCol; + + let packedConicXY: u32 = pack2x16float(conic.xy); + let packedConicZOpacity: u32 = pack2x16float(vec2(conic.z, opacity_f)); + + splatBuffer[sortIdx].conic_opacity0 = packedConicXY; + splatBuffer[sortIdx].conic_opacity1 = packedConicZOpacity; + + sort_indices[sortIdx] = sortIdx; + sort_depths[sortIdx]= bitcast(100.0 - viewPos.z); let keys_per_dispatch = workgroupSize * sortKeyPerThread; // increment DispatchIndirect.dispatchx each time you reach limit for one dispatch of keys + if (sortIdx % keys_per_dispatch == 0){ + atomicAdd(&sort_dispatch.dispatch_x, 1); + } } \ No newline at end of file