Project 5: Thomas Shaw

.github/workflows/code-quality.yml

@@ -0,0 +1,26 @@
+name: Run code quality tooling
+
+on:
+  pull_request:
+    branches: [main]
+  workflow_call:
+
+jobs:
+  static-analysis:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+      - name: Set up Node
+        uses: actions/setup-node@v4
+        with:
+          node-version: 22
+          cache: 'npm'
+      - name: Install dependencies
+        run: npm ci
+      - name: Typecheck
+        run: npm run typecheck
+      - name: Lint
+        run: npm run lint
+      - name: Check formatting
+        run: npm run format:check

.github/workflows/static.yml → .github/workflows/deploy.yml

@@ -21,8 +21,12 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
+  check-code-quality:
+    uses: ./.github/workflows/code-quality.yml
+
   # Single deploy job since we're just deploying
   deploy:
+    needs: [check-code-quality]
     environment:
       name: github-pages
       url: ${{ steps.deployment.outputs.page_url }}
@@ -33,13 +37,14 @@ jobs:
       - name: Set up Node
         uses: actions/setup-node@v4
         with:
-          node-version: 20
+          node-version: 22
           cache: 'npm'
       - name: Install dependencies
         run: npm ci
       - name: Build
         env:
-          GITHUB_ACTIONS_BASE: ${{ github.event.repository.name }}
+          # Base URL to be passed into Vite
+          GITHUB_ACTIONS_BASE: /${{ github.event.repository.name }}
         run: npm run build
       - name: Setup Pages
         uses: actions/configure-pages@v4
@@ -50,4 +55,4 @@ jobs:
           path: './dist'
       - name: Deploy to GitHub Pages
         id: deployment
-        uses: actions/deploy-pages@v4
+        uses: actions/deploy-pages@v4

.prettierrc.json

@@ -0,0 +1,9 @@
+{
+  "$schema": "https://json.schemastore.org/prettierrc",
+  "printWidth": 100,
+  "singleQuote": true,
+  "endOfLine": "auto",
+  "semi": true,
+  "tabWidth": 2,
+  "trailingComma": "all"
+}

INSTRUCTION.md

@@ -1,5 +1,4 @@
-WebGPU WebGPU Gaussian Splat Viewer Instructions
-==========================================================
+# WebGPU WebGPU Gaussian Splat Viewer Instructions
 
 **This is due Tuesday, October 28th at 11:59 PM.**
 
@@ -9,15 +8,16 @@ In this project, you will implement the 3D Gaussian Splat Viewer. You are given
 
 - `src/` contains all the TypeScript and WGSL code for this project. This contains several subdirectories:
   - `renderers/` defines the different renderers in which you will implement gaussian renderer.
-  - `shaders/` contains the WGSL files that are interpreted as shader programs at runtime. 
+  - `shaders/` contains the WGSL files that are interpreted as shader programs at runtime.
   - `camera/` includes camera controls, camera file loading.
-  - `utils/` includes PLY file loading, utility functions for debugging. 
+  - `utils/` includes PLY file loading, utility functions for debugging.
 
 <b>Scene Files: </b> download scene files from [drive](https://drive.google.com/drive/folders/1Fz0QhyDU12JTsl2e7umGi5iy_V9drrIW?usp=sharing).
 
 ## Running the code
 
 Follow these steps to install and view the project:
+
 - Clone this repository
 - Download and install [Node.js](https://nodejs.org/en/)
 - Run `npm install` in the root directory of this project to download and install dependencies
@@ -45,6 +45,7 @@ Follow these steps to install and view the project:
 ### GitHub Pages setup
 
 Since this project uses WebGPU, it is easy to deploy it on the web for anyone to see. To set this up, do the following:
+
 - Go to your repository's settings
 - Go to the "Pages" tab
 - Under "Build and Deployment", set "Source" to "GitHub Actions"
@@ -58,6 +59,7 @@ Once you've done those steps, any new commit to the `main` branch should automat
 **Ask on Ed Discussion for any clarifications.**
 
 In this project, you are given code for:
+
 - ply scene loading
 - camera json file loading
 - radix sort compute shader
@@ -69,39 +71,42 @@ For editing the project, you will want to use [Visual Studio Code](https://code.
 WebGPU errors will appear in your browser's developer console (Ctrl + Shift + J for Chrome on Windows). Unlike some other graphics APIs, WebGPU error messages are often very helpful, especially if you've labeled your various pipeline components with meaningful names. Be sure to check the console whenever something isn't working correctly.
 
 ### Part 1: Understanding 3D Gaussian Point Cloud & Add MVP calculation (10pts)
-- Read over the [3D Gaussian Splatting Paper](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/) to have a basic understanding. 
-- Then read over `point_cloud` renderer, add MVP calculation to the vertex shader. After that, you can see yellow point cloud rendered to screen. 
+
+- Read over the [3D Gaussian Splatting Paper](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/) to have a basic understanding.
+- Then read over `point_cloud` renderer, add MVP calculation to the vertex shader. After that, you can see yellow point cloud rendered to screen.
 
 ### Part 2: Gaussian Renderer (80pts total)(50pts(preprocess)+30pts(renderer))
 
-#### Gaussian Renderer Pipeline Overview: 
-  - Loading 3D gaussian data into GPU (this part is done for you, see `PointCloud` in load.ts)
-  - Preprocess 3D gaussian data
-    - Implement view frustum culling to remove non-visible Gaussians (make bounding box to be slightly larger to keep the edge gaussians)
-    - Compute 3D covariance based on rotation and scale, also user inputted gaussian multipler. (see [post](https://github.com/kwea123/gaussian_splatting_notes) on 1.1 section)
-    - Compute 2D conic, maximum radius, and <b>maximum quad size in NDC</b> (see [post](https://github.com/kwea123/gaussian_splatting_notes) on 1.1 section)
-    - Using spherical harmonics coeffiecients to evaluate the color of the gaussian from particular view direction (evaluation function is provided, see [post](https://beatthezombie.github.io/sh_post_1/)  ). 
-    - Store essential 2D gaussian data for later rasteriation pipeline
-    - Add key_size, indices, and depth to sorter. 
-  - Sort Gaussians based on depth
-  - Render the 2D splat on quad utlizing indirect draw call (instance count from process step) in sorted order.
-    - vertex shader: reconstruct 2D quad vertices (NDC) from splat data, send conic and color information to fragment shader
-    - fragment shader: using conic matrix [see "Centered matrix equation"](https://en.wikipedia.org/wiki/Matrix_representation_of_conic_sections) and [CUDA implementation](https://github.com/graphdeco-inria/diff-gaussian-rasterization/blob/main/cuda_rasterizer/forward.cu#L330) to determine whether point is inside splat. The opacity should decade exponentially as it distant from center.
+#### Gaussian Renderer Pipeline Overview:
+
+- Loading 3D gaussian data into GPU (this part is done for you, see `PointCloud` in load.ts)
+- Preprocess 3D gaussian data
+  - Implement view frustum culling to remove non-visible Gaussians (make bounding box to be slightly larger to keep the edge gaussians)
+  - Compute 3D covariance based on rotation and scale, also user inputted gaussian multipler. (see [post](https://github.com/kwea123/gaussian_splatting_notes) on 1.1 section)
+  - Compute 2D conic, maximum radius, and <b>maximum quad size in NDC</b> (see [post](https://github.com/kwea123/gaussian_splatting_notes) on 1.1 section)
+  - Using spherical harmonics coeffiecients to evaluate the color of the gaussian from particular view direction (evaluation function is provided, see [post](https://beatthezombie.github.io/sh_post_1/) ).
+  - Store essential 2D gaussian data for later rasteriation pipeline
+  - Add key_size, indices, and depth to sorter.
+- Sort Gaussians based on depth
+- Render the 2D splat on quad utlizing indirect draw call (instance count from process step) in sorted order.
+  - vertex shader: reconstruct 2D quad vertices (NDC) from splat data, send conic and color information to fragment shader
+  - fragment shader: using conic matrix [see "Centered matrix equation"](https://en.wikipedia.org/wiki/Matrix_representation_of_conic_sections) and [CUDA implementation](https://github.com/graphdeco-inria/diff-gaussian-rasterization/blob/main/cuda_rasterizer/forward.cu#L330) to determine whether point is inside splat. The opacity should decade exponentially as it distant from center.
 
 #### Implementation Hints:
+
 - scene file information:
   - `opacity`: use [sigmoid](https://github.com/graphdeco-inria/diff-gaussian-rasterization/blob/59f5f77e3ddbac3ed9db93ec2cfe99ed6c5d121d/cuda_rasterizer/auxiliary.h#L134) function to bring back maximum opacity
   - `scale` (log space): use exponential function to bring back actual scale
 - useful shader functions:
-  - `unpack2x16float`: all gaussian data is packed in f16, need to unpacked it in shader. 
+  - `unpack2x16float`: all gaussian data is packed in f16, need to unpacked it in shader.
   - `pack2x16float`: pack you 2D gaussian data in f16 format
   - `atomicAdd` : store indices in sorting buffer using thread-safe updates in compute shaders
   - `arrayLength`: remember to check thread idx is within the numbers of gaussians
 - Setting up pipeline:
-  - `device.queue.writeBuffer`: Remember to clean sort infos each frame.  
+  - `device.queue.writeBuffer`: Remember to clean sort infos each frame.
   - `encoder.copyBufferToBuffer`: GPU buffer transfer data to other GPU buffer
-  - `blend`: using similar blending function as rendering semi-transparent texture to screen. 
-  - `depth`: similarly to semi-transparent texture, we should render gaussians back to front. 
+  - `blend`: using similar blending function as rendering semi-transparent texture to screen.
+  - `depth`: similarly to semi-transparent texture, we should render gaussians back to front.
 
 #### <b>Implementation Step Recommendation:</b>
 
@@ -112,19 +117,19 @@ WebGPU errors will appear in your browser's developer console (Ctrl + Shift + J
 Before working on the preprocessing compute pipeline, you should first set up the indirect rendering pipeline that replicates the point cloud rendering you did in part 1. The recommended workflow is as follows:
 
 1. Set up the indirect draw buffer and the render pipeline, and render pc.num_points number of quads (with smaller size such as 0.01 x 0.01) on the screen.
-2. Finish the compute pipeline by binding the correct buffers. In the preprocess shader, transform the Gaussian points from world space to NDC space and stored data in Splat (similar to point cloud renderer). In the render pipeline, read from this Splat buffer to determine the position for drawing the quad. 
+2. Finish the compute pipeline by binding the correct buffers. In the preprocess shader, transform the Gaussian points from world space to NDC space and stored data in Splat (similar to point cloud renderer). In the render pipeline, read from this Splat buffer to determine the position for drawing the quad.
 3. Now, implement a simple view-frustum culling (bounding box of 1.2x screen size is recommended). Instead of drawing pc.num_points quads through the hard-coded indirect buffer, use `atomicAdd` on `sort_infos.keys_size` to determine how many quads to draw. After the preprocess shader, use `sort_infos.keys_size` to update the indirect draw buffer's instance count. Now the total number of quads to render is determined by the preprocess shader.
 4. The final step for setup is to bind the render settings as a buffer, as declared in the shader. The Gaussian scaling member variable should be adjustable on the host side using a scroll bar (see in `renderer.ts`).
 
-To test whether the buffer is working, use the Gaussian scaling member to influence the quad size, allowing you to adjust the size of the quads with the scroll bar. Also, be careful that complier might optimize away shader binding buffer if you do not actually use it in shader functions. 
+To test whether the buffer is working, use the Gaussian scaling member to influence the quad size, allowing you to adjust the size of the quads with the scroll bar. Also, be careful that complier might optimize away shader binding buffer if you do not actually use it in shader functions.
 
 <b>Preprocess Compute Setup Suggestions: </b>
 
 Now, implement the computation of the 3D covariance, 2D covariance, and finally the radius, as detailed in the post links provided above. After computing the radius, use it to determine the quad size. You can load `bicycle_30000.cleaned.ply` (default camera) and compare it with the provided image below:
 
 ![quad_white](./images/quad_white.png)
 
-After computing the radius, you can implement sorting by specifying `sort_depths` and `sort_indices` for each visible Gaussian. Also, remember to use `atomicAdd` on `sort_dispatch.dispatch_x` every time the visible Gaussian count exceeds the total number of sorting threads you will dispatch. To visualize the sorting result, assign each quad a color determined by its size, for example, `(width, height, 0.0, 1.0)` as the RGBA. 
+After computing the radius, you can implement sorting by specifying `sort_depths` and `sort_indices` for each visible Gaussian. Also, remember to use `atomicAdd` on `sort_dispatch.dispatch_x` every time the visible Gaussian count exceeds the total number of sorting threads you will dispatch. To visualize the sorting result, assign each quad a color determined by its size, for example, `(width, height, 0.0, 1.0)` as the RGBA.
 
 ![quad_green](./images/quad_green.png)
 
@@ -140,72 +145,84 @@ To determine whether a point is inside a splat, you need to compute the offset f
 
 ![quad_pixeldiff](./images/quad_pixeldiff.png)
 
-### Part 3: Extra Credit: 
+### Part 3: Extra Credit:
 
 #### Optimization: tile-based depth sorting (20pts)
 
-Follow the [paper](https://github.com/kwea123/gaussian_splatting_notes) implementation using tile-based depth sorting. Then composite the final image using compute shader. 
+Follow the [paper](https://github.com/kwea123/gaussian_splatting_notes) implementation using tile-based depth sorting. Then composite the final image using compute shader.
 
 ![Gaussian with Tile](./images/sorting2.webp)
 ![Gaussian with Tile](./images/sorting1.webp)
 
 #### Optimization: half-precision floating point calculation (10pts)
 
-See the [WebGPU supported f16 function](https://webgpufundamentals.org/webgpu/lessons/webgpu-wgsl-function-reference.html), implement your compressed f16 compute shader for preprocess step. 
+See the [WebGPU supported f16 function](https://webgpufundamentals.org/webgpu/lessons/webgpu-wgsl-function-reference.html), implement your compressed f16 compute shader for preprocess step.
 
 ## Performance Analysis (10pts)
 
 #### Answer these questions:
+
 - Compare your results from point-cloud and gaussian renderer, what are the differences?
 - For gaussian renderer, how does changing the workgroup-size affect performance? Why do you think this is?
-- Does view-frustum culling give performance improvement? Why do you think this is? 
-- Does number of guassians affect performance?  Why do you think this is? 
+- Does view-frustum culling give performance improvement? Why do you think this is?
+- Does number of guassians affect performance? Why do you think this is?
 
 ## Base Code Walkthrough
 
 In general, you can search for comments containing "TODO" to see the most important/useful parts of the base code.
 
 #### `renderers/`
+
 Defines the different renderers for Gaussian splats and point clouds.
+
 - **`gaussian-renderer.ts`**: Implements the renderer for Gaussian splats.
 - **`point-cloud-renderer.ts`**: Implements the renderer for point clouds.
 - **`renderer.ts`**: Abstract base render app for common renderer functionality, as well as GUI.
 
 #### `shaders/`
+
 Contains WGSL shader programs that run on the GPU for rendering.
+
 - **`gaussian.wgsl`**: Vertex and fragment shaders for Gaussian splat rendering.
 - **`point_cloud.wgsl`**: Shaders for rendering point cloud data.
 - **`preprocess.wgsl`**: Preprocesses Gaussian data for rendering.
 
 #### `camera/`
+
 Manages camera controls and view/projection transformations.
+
 - **`camera.ts`**: Handles the camera's view and projection matrices.
 - **`camera-control.ts`**: Implements user input handling for camera movement.
 
 #### `utils/`
+
 Includes PLY file loading and utility functions for debugging and GPU management.
+
 - **`load.ts`**: Loads Gaussian PLY data, and sets up WebGPU buffers.
 - **`plyreader.ts`**: Parses PLY files to extract point cloud data.
 - **`simple-console.ts`**: Provides basic logging and debugging utilities.
 - **`util.ts`**: Provides utility functions for assertions, error handling, and alignment operations.
 
 #### `sort/`
+
 Handles sorting operations for rendering transparency in Gaussian splats.
+
 - **`sort.ts`**: Interfaces with the radix sort WGSL shader to sort Gaussian splats by depth.
 - **`radix_sort.wgsl`**: Implements parallel radix sort on the GPU.
 
 #### `main.ts`
+
 The entry point of the project, responsible for initializing WebGPU.
 
 ## README
 
 Replace the contents of `README.md` with the following:
+
 - A brief description of your project and the specific features you implemented
 - At least one screenshot of your project running
 - A 30+ second video/gif of your project running showing all features (even though your demo can be seen online, it may not run on all computers, while a video will work everywhere)
 - A link to your project's live website (see [GitHub Pages setup](#github-pages-setup))
 
-
 ## Submit
 
 Open a GitHub pull request so that we can see that you have finished. The title should be "Project 4: YOUR NAME". The template of the comment section of your pull request is attached below, you can do some copy and paste:

README.md

@@ -2,25 +2,29 @@
 
 **University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 5**
 
-* (TODO) YOUR NAME HERE
-* Tested on: (TODO) **Google Chrome 222.2** on
-  Windows 22, i7-2222 @ 2.22GHz 22GB, GTX 222 222MB (Moore 2222 Lab)
+- Thomas Shaw
+- Tested on: **Google Chrome 139.0** on
+  Windows 11, Ryzen 7 5700x @ 4.67GHz, 32GB, RTX 2070 8GB
 
-### Live Demo
-
-[![](img/thumb.png)](http://TODO.github.io/Project4-WebGPU-Forward-Plus-and-Clustered-Deferred)
+### [Live Demo](https://printer83mph.github.io/CIS5650-Project5-WebGPU-Gaussian-Splat-Viewer/)
 
 ### Demo Video/GIF
 
-[![](img/video.mp4)](TODO)
+<img src="docs/video/bonsai.webp" width="512px" />
+
+### Analysis
+
+- Comparing our point cloud renderer to our gaussian splat renderer, we find a fairly large degredation in performance. This makes sense — our point cloud renderer only has to draw points, while the gaussian pipeline must run depth sorting, covariance computation, and overdraw many pixels from each overlapping splat quad.
 
-### (TODO: Your README)
+- Updating the workgroup size leads to strange behavior. This may be due to my implementation, but it seems that any workgroup size outside of 256 will lead to depth sorting artifacts and reduced performance. It may be that smaller workgroups greatly increase the required depth of the parallel radix sort.
+  - 64: 30-40fps, rendering artifacts
+  - 128: 50-100fps, rendering artifacts
+  - 256: 80-120fps
+  - \> 256: invalid size for webgpu
 
-*DO NOT* leave the README to the last minute! It is a crucial part of the
-project, and we will not be able to grade you without a good README.
+- View-frustum culling gives a small performance boost only when looking away from some of the scene. It seems that a large part of the performance cost is simply drawing all of the quads.
 
-This assignment has a considerable amount of performance analysis compared
-to implementation work. Complete the implementation early to leave time!
+- With a larger number of gaussians, scene load time increases dramatically, and performance degrades quite a bit. However, of course, it is only when facing the model and thus drawing these quads that the performance hit is seen.
 
 ### Credits