Project 5B: Rony Edde

README.md

@@ -3,27 +3,85 @@ WebGL Deferred Shading
 
 **University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 5**
 
-* (TODO) YOUR NAME HERE
-* Tested on: (TODO) **Google Chrome 222.2** on
-  Windows 22, i7-2222 @ 2.22GHz 22GB, GTX 222 222MB (Moore 2222 Lab)
+* (TODO) Rony Edde
+* Tested on: **Google Chrome 222.2** on
+  Windows 10, i7-6700k @ 4.00GHz 64GB, GTX 980M 8GB (Personal Laptop)
 
 ### Live Online
-
-[![](img/thumb.png)](http://TODO.github.io/Project5B-WebGL-Deferred-Shading)
+[Live Demo](https://reddeupenn.github.io/Project5-WebGL-Deferred-Shading-with-glTF/)
 
 ### Demo Video/GIF
+![fov1](./img/layers.gif)
 
-[![](img/video.png)](TODO)
-
-### (TODO: Your README)
-
-*DO NOT* leave the README to the last minute! It is a crucial part of the
-project, and we will not be able to grade you without a good README.
-
-This assignment has a considerable amount of performance analysis compared
-to implementation work. Complete the implementation early to leave time!
-
+### Description
+This is a WebGL deferred renderer.
+  * Passes are rendered to the frame buffer, then composited in a deferred shader.
+  The deferred shaders shade and light the resulting fragments by their attributes.
+  Normals, normal maps, positions, texture and depth are precomputed and the deferred
+  pass computes the lighting and shading of the final image.
+
+  * Pass layout.
+    * Z-depth.
+      * This is the depth pass that computes the distance to camera.  This is useful
+      for multiple post effects such as depth of field.
+      ![zdepth](./img/depth.png)
+    * Position.
+      * This is a position pass where every fragment stores the xyz positions of the rendered geometry.
+      ![pos](./img/position.png)
+    * Geometry Normal
+      * This is a pass for the geometric normals coming straight from the mesh geometry
+      ![gnorm](./img/geometry_normal.png)
+    * Color Map
+      * This is the texture and color of the geometry again with no shading.  Only the texture color is applied.
+      ![colormap](./img/colormap.png)
+    * Normal Map
+      * This is similar to the texture pass but it contains the normal map texture for compting fake surface detail shading.
+      ![normalmap](./img/normalamap.png)
+    * Surface Normal
+      * This is a final calculation of the geometric normal with the normal map applied to it.
+      ![surfacenormal](./img/surface_normal.png)
+
+  * Final result with all passes combined
+    * After combining all the layers and computing the blinn-phong lighting model, here's the final result:  
+      ![blinn_phong](./img/blinn_phong.png)
+
+  * Final result with a toon shader
+    * Using the blinn-phong shader, we can extend it by computing the dot product of the surface normal and camera and accentuate the shading.  There are 3 color intensities chosen.  Black for perpendicular vectors, half color for facing geometry with less than 0.5 value and full color for less than 0.25.
+       ![toon](./img/toon.png)
 
+  * Post processing
+    * Bloom:  
+      By blurring the final rendered image and compositing it on top with a gl blend function, we can add a bloom post process effect.
+      There are 2 modes in the bloom shader implemented.  Square blur which is the default and gaussian blur which is computed for every       pixel. 
+      Here's the result:  
+      ![gaussian_bloom](./img/gaussian_bloom.png)
+
+    * Motion blur  
+      By sampling the camera motion and computing the difference transformation matrix, we can gereate motion vectors and use them to  generate blur in that direction.  Shader version 100 doesn't support matrix operations so we're forced to compute the difference beforehand which is not a bad idea since the shader will have to do less work over an unvarying variable, however the THREE.js precision isn't as good as expected so we get a few artifacts when the camera is slowing down to a complete stop.  Still the results are not too bad:  
+      ![motion_blur](./img/motion_blur.png)
+
+
+     * Optimizations:
+       Enabling scissor tests dramatically improves performance.  Rendering only the square that the fragments overlap the sphere radius.  
+       The initial mode with scissor test is shown here: 
+       ![scissor_test](./img/scissor_test.png)
+
+       We can even go further by rendering a sphere as an instance for the scissor test:  
+       ![instance_sphere](./img/instance_sphere.png)
+
+
+  * Performance analysis with scissor tests:  
+    * Running the scissor test on 20 lights has almost no impact on performance.  It's only when increase the number of lights to about 100 and more that we truly see the benefit of the scissor test.  Here are the results:  
+    ![benchmarks](./img/benchmarks.png)  
+
+    We can clearly see an improvement inperformance.  The time per frame is halved.  This is particularly noticeable when using gaussian bloom since the gaussian computation is heavy.
+
+
+    THANK YOU
+
+
+
+
 ### Credits
 
 * [Three.js](https://github.com/mrdoob/three.js) by [@mrdoob](https://github.com/mrdoob) and contributors

glsl/copy.frag.glsl

@@ -10,11 +10,15 @@ varying vec3 v_position;
 varying vec3 v_normal;
 varying vec2 v_uv;
 
+
 void main() {
     // TODO: copy values into gl_FragData[0], [1], etc.
     // You can use the GLSL texture2D function to access the textures using
     // the UV in v_uv.
 
     // this gives you the idea
-    // gl_FragData[0] = vec4( v_position, 1.0 );
+    gl_FragData[0] = vec4(v_position, 1.0);
+    gl_FragData[1] = vec4(v_normal, 1.0);
+    gl_FragData[2] = vec4(texture2D(u_colmap, v_uv, 0.01));
+    gl_FragData[3] = vec4(texture2D(u_normap, v_uv, 0.01));      
 }

glsl/copy.vert.glsl

@@ -18,4 +18,5 @@ void main() {
     v_position = a_position;
     v_normal = a_normal;
     v_uv = a_uv;
+
 }

glsl/deferred/ambient.frag.glsl

@@ -24,4 +24,7 @@ void main() {
     }
 
     gl_FragColor = vec4(0.1, 0.1, 0.1, 1);  // TODO: replace this
+
+    //gl_FragColor = gb3;
+
 }

glsl/deferred/blinnphong-pointlight.frag.glsl

@@ -9,9 +9,12 @@ uniform vec3 u_lightPos;
 uniform float u_lightRad;
 uniform sampler2D u_gbufs[NUM_GBUFFERS];
 uniform sampler2D u_depth;
+uniform vec3 u_cameraPos;
 
 varying vec2 v_uv;
 
+
+
 vec3 applyNormalMap(vec3 geomnor, vec3 normap) {
     normap = normap * 2.0 - 1.0;
     vec3 up = normalize(vec3(0.001, 1, 0.001));
@@ -21,11 +24,21 @@ vec3 applyNormalMap(vec3 geomnor, vec3 normap) {
 }
 
 void main() {
-    vec4 gb0 = texture2D(u_gbufs[0], v_uv);
-    vec4 gb1 = texture2D(u_gbufs[1], v_uv);
-    vec4 gb2 = texture2D(u_gbufs[2], v_uv);
-    vec4 gb3 = texture2D(u_gbufs[3], v_uv);
-    float depth = texture2D(u_depth, v_uv).x;
+	vec2 uv = vec2(gl_FragCoord.x / 800.0, gl_FragCoord.y / 600.0);  //hard coded screen size
+    vec4 gb0 = texture2D(u_gbufs[0], uv);
+    vec4 gb1 = texture2D(u_gbufs[1], uv);
+    vec4 gb2 = texture2D(u_gbufs[2], uv);
+    vec4 gb3 = texture2D(u_gbufs[3], uv);
+    float depth = texture2D(u_depth, uv).x;
+
+
+    //vec3 pos = gb0.xyz;     // World-space position
+    //vec3 geomnor = gb1.xyz;  // Normals of the geometry as defined, without normal mapping
+    //vec3 colmap = gb2.rgb;  // The color map - unlit "albedo" (surface color)
+    //vec3 normap = gb3.xyz;  // The raw normal map (normals relative to the surface they're on)
+    //vec3 nor = applyNormalMap (geomnor, normap);     // The true normals as we want to light them - with the normal map applied to the geometry normals (applyNormalMap above)
+
+
     // TODO: Extract needed properties from the g-buffers into local variables
 
     // If nothing was rendered to this pixel, set alpha to 0 so that the
@@ -35,5 +48,50 @@ void main() {
         return;
     }
 
-    gl_FragColor = vec4(0, 0, 1, 1);  // TODO: perform lighting calculations
+    //gl_FragColor = vec4(0, 0, 1, 1);  // TODO: perform lighting calculations
+
+    vec3 N = applyNormalMap(gb1.xyz, gb3.xyz);
+    //gl_FragColor = vec4(1.0, 1.0, 1.0, 1.0);
+
+    float shininess = 10.0;
+
+    // assign variables
+    vec3 vertPos = vec3(gb0);
+    //vec3 ambientColor = vec3(0.1, 0.1, 0.1);
+    vec3 diffuseColor = vec3(gb2);
+    //vec3 diffuseColor = vec3(1.0, 1.0, 1.0);
+    vec3 specColor = vec3(0.5, 0.5, 0.5);
+
+	vec3 normal = normalize(N);
+	vec3 lightDir = normalize(u_lightPos - vertPos);
+
+    float attenuation = max(0.0, 1.0 - length(u_lightPos - vertPos) / u_lightRad);
+
+	float lambertian = max(dot(lightDir,normal), 0.0);
+	float specular = 0.1;
+
+	if(lambertian > 0.0)
+	{
+
+		vec3 viewDir = -normalize(vertPos - u_cameraPos);
+
+		// blinn phong
+		vec3 halfDir = normalize(lightDir + viewDir);
+		float specAngle = max(dot(halfDir, normal), 0.0);
+		specular = min(pow(specAngle, shininess), 1.0);
+
+		vec3 reflectDir = reflect(-lightDir, normal);
+		specAngle = max(dot(reflectDir, viewDir), 0.0);
+
+		specular = min(pow(specAngle, shininess/4.0), 1.0);
+	}
+	vec3 color = u_lightCol * (lambertian * diffuseColor +
+							   specular * specColor);
+
+	gl_FragColor = vec4(color, 1.0) * attenuation;
+	//gl_FragColor = vec4(u_cameraPos, 1.0);
+
+
+    // gl_FragColor = vec4(1.0, 0.0, 0.0, 1.0);
+
 }

glsl/deferred/blinnphong-pointlightSphere.frag.glsl

@@ -0,0 +1,97 @@
+#version 100
+precision highp float;
+precision highp int;
+
+#define NUM_GBUFFERS 4
+
+uniform vec3 u_lightCol;
+uniform vec3 u_lightPos;
+uniform float u_lightRad;
+uniform sampler2D u_gbufs[NUM_GBUFFERS];
+uniform sampler2D u_depth;
+uniform vec3 u_cameraPos;
+
+varying vec2 v_uv;
+
+
+
+vec3 applyNormalMap(vec3 geomnor, vec3 normap) {
+    normap = normap * 2.0 - 1.0;
+    vec3 up = normalize(vec3(0.001, 1, 0.001));
+    vec3 surftan = normalize(cross(geomnor, up));
+    vec3 surfbinor = cross(geomnor, surftan);
+    return normap.y * surftan + normap.x * surfbinor + normap.z * geomnor;
+}
+
+void main() {
+	vec2 uv = vec2(gl_FragCoord.x / 800.0, gl_FragCoord.y / 600.0);  //hard coded screen size
+    vec4 gb0 = texture2D(u_gbufs[0], uv);
+    vec4 gb1 = texture2D(u_gbufs[1], uv);
+    vec4 gb2 = texture2D(u_gbufs[2], uv);
+    vec4 gb3 = texture2D(u_gbufs[3], uv);
+    float depth = texture2D(u_depth, uv).x;
+
+
+    //vec3 pos = gb0.xyz;     // World-space position
+    //vec3 geomnor = gb1.xyz;  // Normals of the geometry as defined, without normal mapping
+    //vec3 colmap = gb2.rgb;  // The color map - unlit "albedo" (surface color)
+    //vec3 normap = gb3.xyz;  // The raw normal map (normals relative to the surface they're on)
+    //vec3 nor = applyNormalMap (geomnor, normap);     // The true normals as we want to light them - with the normal map applied to the geometry normals (applyNormalMap above)
+
+
+    // TODO: Extract needed properties from the g-buffers into local variables
+
+    // If nothing was rendered to this pixel, set alpha to 0 so that the
+    // postprocessing step can render the sky color.
+    if (depth == 1.0) {
+        gl_FragColor = vec4(0, 0, 0, 0);
+        return;
+    }
+
+    //gl_FragColor = vec4(0, 0, 1, 1);  // TODO: perform lighting calculations
+
+    vec3 N = applyNormalMap(gb1.xyz, gb3.xyz);
+    //gl_FragColor = vec4(1.0, 1.0, 1.0, 1.0);
+
+    float shininess = 10.0;
+
+    // assign variables
+    vec3 vertPos = vec3(gb0);
+    //vec3 ambientColor = vec3(0.1, 0.1, 0.1);
+    vec3 diffuseColor = vec3(gb2);
+    //vec3 diffuseColor = vec3(1.0, 1.0, 1.0);
+    vec3 specColor = vec3(0.5, 0.5, 0.5);
+
+	vec3 normal = normalize(N);
+	vec3 lightDir = normalize(u_lightPos - vertPos);
+
+    float attenuation = max(0.0, 1.0 - length(u_lightPos - vertPos) / u_lightRad);
+
+	float lambertian = max(dot(lightDir,normal), 0.0);
+	float specular = 0.1;
+
+	if(lambertian > 0.0)
+	{
+
+		vec3 viewDir = -normalize(vertPos - u_cameraPos);
+
+		// blinn phong
+		vec3 halfDir = normalize(lightDir + viewDir);
+		float specAngle = max(dot(halfDir, normal), 0.0);
+		specular = min(pow(specAngle, shininess), 1.0);
+
+		vec3 reflectDir = reflect(-lightDir, normal);
+		specAngle = max(dot(reflectDir, viewDir), 0.0);
+
+		specular = min(pow(specAngle, shininess/4.0), 1.0);
+	}
+	vec3 color = u_lightCol * (lambertian * diffuseColor +
+							   specular * specColor);
+
+	gl_FragColor = vec4(color, 1.0) * attenuation;
+	//gl_FragColor = vec4(u_cameraPos, 1.0);
+
+
+    // gl_FragColor = vec4(1.0, 0.0, 0.0, 1.0);
+
+}

glsl/deferred/debug.frag.glsl

@@ -38,15 +38,15 @@ void main() {
     if (u_debug == 0) {
         gl_FragColor = vec4(vec3(depth), 1.0);
     } else if (u_debug == 1) {
-        // gl_FragColor = vec4(abs(pos) * 0.1, 1.0);
+        gl_FragColor = vec4(abs(pos) * 0.1, 1.0);
     } else if (u_debug == 2) {
-        // gl_FragColor = vec4(abs(geomnor), 1.0);
+        gl_FragColor = vec4(abs(geomnor), 1.0);
     } else if (u_debug == 3) {
-        // gl_FragColor = vec4(colmap, 1.0);
+        gl_FragColor = vec4(colmap, 1.0);
     } else if (u_debug == 4) {
-        // gl_FragColor = vec4(normap, 1.0);
+        gl_FragColor = vec4(normap, 1.0);
     } else if (u_debug == 5) {
-        // gl_FragColor = vec4(abs(nor), 1.0);
+        gl_FragColor = vec4(abs(nor), 1.0);
     } else {
         gl_FragColor = vec4(1, 0, 1, 1);
     }