Project 5: Lu Men

.github/workflows/static.yml

@@ -33,7 +33,7 @@ jobs:
       - name: Set up Node
         uses: actions/setup-node@v4
         with:
-          node-version: 20
+          node-version: 22
           cache: 'npm'
       - name: Install dependencies
         run: npm ci

.gitignore

@@ -24,4 +24,4 @@ dist-ssr
 *.sw?
 /.vite
 
-*/scenes
+scenes

README.md

@@ -1,26 +1,155 @@
-# Project5-WebGPU-Gaussian-Splat-Viewer
+# WebGPU Gaussian Splatting
+3D Gaussian splatting renderer and viewer in WebGPU.
 
-**University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 5**
+**Author:** Lu M.
+- [LinkedIn](https://www.linkedin.com/in/lu-m-673425323/)
+- [Personal Site](lu-m-dev.github.io)
 
-* (TODO) YOUR NAME HERE
-* Tested on: (TODO) **Google Chrome 222.2** on
-  Windows 22, i7-2222 @ 2.22GHz 22GB, GTX 222 222MB (Moore 2222 Lab)
+**Tested System:**
+ - Windows 11 Home
+ - AMD Ryzen 7 5800HS @ 3.20GHz, 16GB RAM
+ - NVIDIA GeForce RTX 3060 Laptop GPU 6GB (Compute Capability 8.6)
+ - Brave 1.83.118 (Official Build) (64-bit), Chromium: 141.0.7390.108
 
-### Live Demo
+## Abstract
 
-[![](img/thumb.png)](http://TODO.github.io/Project4-WebGPU-Forward-Plus-and-Clustered-Deferred)
+Gaussian splatting is a point-based rendering technique where each 3D point is rasterized as a screen-space Gaussian instead of a single pixel. Each splat carries a position, covariance, and color. When splats overlap, their contributions are composited to reconstruct a smooth, continuous appearance.
 
-### Demo Video/GIF
+## Methods
 
-[![](img/video.mp4)](TODO)
+This implementation follows three main stages: preprocess, sort, and render. Each stage is WebGPU-parallelized.
 
-### (TODO: Your README)
+### Preprocess
 
-*DO NOT* leave the README to the last minute! It is a crucial part of the
-project, and we will not be able to grade you without a good README.
+- Input a 3D point-cloud loaded from a PLY.
+- Transform points into view, clip and NDC coordinates.
+- Compute per-point splat parameters: positions, 3D covariance, 2D covariance.
 
-This assignment has a considerable amount of performance analysis compared
-to implementation work. Complete the implementation early to leave time!
+### Sort
+
+- Purpose: sorting points back-to-front is required for correct alpha blending and improved memory coherence.
+- Implementation: a GPU radix sort.
+
+### Render
+
+Two renderers are provided:
+- Point-cloud renderer: renders raw points with a simple size and per-point color. This uses a minimal vertex/fragment pipeline.
+- Gaussian splat renderer: for each splat, a screen-space quad is rasterized and shaded with a precomputed Gaussian weight using the splat covariance and color. The fragment shader evaluates the color and opacity.
+
+### Tile-Based Depth Sorting
+Improve rendering performance on large point clouds by organizing Gaussian splats into screen-space tiles and sorting per-tile, reducing memory bandwidth and improving cache locality.
+
+Implementation: a compute-based pipeline that:
+1. For each Gaussian, determines which tiles it overlaps using its 2D projection and splat radius.
+2. Generates key-value pairs `(tile_id | depth, gaussian_id)` for each Gaussian-tile intersection using atomic operations to avoid write conflicts.
+3. Sorts these pairs globally using radix sort.
+4. Identifies start/end ranges for each tile via a second compute pass.
+5. During rendering, tiles can be rasterized in any order with per-tile depth ordering.
+
+Trade-offs: This approach reduces per-frame fragment shader overdraw in theory but adds preprocessing overhead. For scenes with significant depth complexity and overlapping splats per tile, this can improve performance. For scenes with sparse or well-distributed splats, the overhead may outweigh the benefits.
+
+## Visual comparison
+
+### Bonsai (272,965 points)
+
+<table>
+  <tr>
+    <th>Point Cloud Renderer</th>
+    <th>Gaussian Splatting</th>
+  </tr>
+  <tr>
+    <td><img src="./images/bonsai_pointcloud.png" alt="bonsai pointcloud" style="max-width:100%"></td>
+    <td><img src="./images/bonsai_gaussian.png" alt="bonsai gaussian" style="max-width:100%"></td>
+  </tr>
+  <tr>
+    <td style="text-align:center">272,965 points — point renderer</td>
+    <td style="text-align:center">272,965 points — gaussian splatting</td>
+  </tr>
+</table>
+
+Observation: for the bonsai dataset (272,965 points) both renderers saturate the display refresh on the test laptop and hit the monitor's 144 Hz cap in the author's measurements.
+
+### Bicycle (1,063,091 points)
+
+<table>
+  <tr>
+    <th>Point Cloud Renderer</th>
+    <th>Gaussian Splatting</th>
+  </tr>
+  <tr>
+    <td><img src="./images/bicycle_pointcloud.png" alt="bicycle pointcloud" style="max-width:100%"></td>
+    <td><img src="./images/bicycle_gaussian.png" alt="bicycle gaussian" style="max-width:100%"></td>
+  </tr>
+  <tr>
+    <td style="text-align:center">1,063,091 points — point renderer</td>
+    <td style="text-align:center">1,063,091 points — gaussian splatting</td>
+  </tr>
+</table>
+
+Measured performance (personal laptop):
+
+- Bonsai (272,965 points): both renderers hit the display cap at 144 Hz.
+- Bicycle (1,063,091 points): point-cloud renderer ≈ 100 Hz; Gaussian splatting renderer ≈ 50 Hz; Gaussian splatting with tile-based depth sorting ≈ 40 Hz.
+
+Discussion: the Gaussian splatting renderer rasterizes a screen-space quad per point and evaluates a Gaussian per-fragment. For dense clouds (the bicycle set) this produces significantly more fragment shader work and overdraw compared to rendering simple points. For the bonsai dataset the total fragment workload is low enough that both appear similarly fast.
+
+The tile-based depth sorting variant demonstrates no visual difference in rendered quality compared to the base Gaussian splatting renderer. However, it runs approximately 20% slower (40 FPS vs 50 FPS on bicycle) despite the theoretical benefits of improved memory coherence and reduced overdraw. This performance regression occurs because:
+1. **Preprocessing overhead dominates**: the tile-based approach adds significant compute cost in key generation, atomic allocations, sorting the larger tile-pair dataset, and range identification. These stages are necessary every frame and account for substantial GPU time.
+2. **Tile-sorting benefits don't materialize at this scale**: while tile-based sorting theoretically improves cache locality, the actual savings in fragment shader evaluation are minimal. The 1M point cloud is already sparse enough per tile that depth coherence doesn't yield meaningful performance gains on modern GPUs with efficient caching.
+3. **Atomic operation contention**: multiple Gaussians writing to the same tile allocator creates atomic contention, serializing work that could otherwise be parallelized.
+4. **Small tile sizes**: to fit within WebGPU buffer limits (max_tile_pairs ≈ 16M), tile sizes must be modest, spreading splats across many tiles and reducing the per-tile depth-sorting benefit.
+
+Note: these are preliminary numbers from a single machine. More comprehensive benchmarking is required (different GPUs, tile-based profiling, memory bandwidth counters, and varying splat sizes) to draw robust conclusions.
+
+## Bloopers
+### Incorrect depth and sort buffer
+<div style="text-align:center">
+<img src="./images/blooper.gif" alt="bonsai blooper" style="max-width:70%">
+</div>
+I implemented incorrect depth calculation and improper sort buffer usage. This resulted in the output displaying Gaussian quads refreshing with black square artifacts that obscured the view.
+
+### Mistaken opacity
+<table>
+  <tr>
+    <th>Bonsai</th>
+    <th>Bicycle</th>
+  </tr>
+  <tr>
+    <td><img src="./images/blooper_bonsai.png" alt="bonsai blooper" style="max-width:100%"></td>
+    <td><img src="./images/blooper_bicycle.png" alt="bicycle blooper" style="max-width:100%"></td>
+  </tr>
+</table>
+I implemented wrong alpha calculation in the fragment shader, mistakenly assigning an arbitrary value of `1.0f` to all fragments. This gave the air around the object an opacity coherent with the object's color.
+
+### Tile-based sorting buffer overflow
+<div style="text-align:center">
+<img src="./images/blooper_tile.gif" alt="tile sorting blooper" style="max-width:70%">
+</div>
+During tile-based depth sorting, I failed to properly allocate space for tile-pair entries using atomic operations, causing buffer overwrites. This resulted in splats rendering as solid colored discs rather than smoothly blended Gaussians, due to corrupted splat data and invalid depth values in the tile-sorted output.
+
+## Build instructions
+
+1) Download [Node.js](https://nodejs.org/en/download)
+
+2) Clone repository
+    ```cmd
+    git clone https://github.com/lu-m-dev/WebGPU-gaussian-splatting.git
+    ```
+
+3) Install dependencies
+    ```cmd
+    cd WebGPU-gaussian-splatting
+    npm install
+    ```
+
+3) Launch dev server
+    ```cmd
+    npm run dev
+    ```
+    Optional build to static `dist/`
+    ```cmd
+    npm run build
+    ```
 
 ### Credits
 

index.html

@@ -3,7 +3,7 @@
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <title>WebGPU 3D Gaussian Splat Viewer</title>
+    <title>WebGPU-gaussian-splatting</title>
   </head>
   <body>
     <div class="console">