Add compute shaders

[RandomGamingDev]

Increasing access

Although WebGL doesn't have official compute shaders, they can be emulated using a few vector shaders, a FBO, and fragment shaders for the actual calculations.

While p5.js's focus isn't computation, this would be perfect for many types of rendering (e.g. raytracing, raymarching, and certain types of culling). It wouldn't provide a speed benefit compared to doing it yourself, but it would mean less boilerplate being required, allow for computations for computation visualizations which are also popular in p5.js, make it easier to create more advanced graphics in p5.js, and also introduce a lot of beginners to the topic of compute shaders (part of p5.js's key principles is to help beginners, which is also part of why shaders, and attempts to make shaders easier, which is why I think this would work well).

Most appropriate sub-area of p5.js?

• Accessibility
• Color
• Core/Environment/Rendering
• Data
• DOM
• Events
• Image
• IO
• Math
• Typography
• Utilities
• WebGL
• Build process
• Unit testing
• Internationalization
• Friendly errors
• Other (specify if possible)

Feature request details

Create computer shader equivalents to createShader() and loadShader() (e.g. createComputeShader() and loadComputeShader()). p5.js would handle the boilerplate in terms of setting up the vertex shaders, part of the fragment shader, and FBO, meaning that the user would only get specific variable inputs and outputs, with the output getting written to a TypedArary buffer.

[Vaivaswat2244]

Hey, @RandomGamingDev,
maybe for making the computeShader feature, the following approaching might help,
webgl/p5.Shader.js

initComputeFBO(width, height) {
  const gl = this._renderer.GL;

  this._computeFramebuffer = gl.createFramebuffer();
  gl.bindFramebuffer(gl.FRAMEBUFFER, this._computeFramebuffer);

  // Creating Texture to store compute shader results
  this._computeTexture = gl.createTexture();
  gl.bindTexture(gl.TEXTURE_2D, this._computeTexture);
  gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, width, height, 0, gl.RGBA, gl.UNSIGNED_BYTE, null);

  // FBO implementation
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);

  
  gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0, gl.TEXTURE_2D, this._computeTexture, 0);

  // Check for FBO completeness
  if (gl.checkFramebufferStatus(gl.FRAMEBUFFER) !== gl.FRAMEBUFFER_COMPLETE) {
    console.error("Failed to initialize compute framebuffer");
  }

  gl.bindFramebuffer(gl.FRAMEBUFFER, null);
}

computeShader(width, height, callback) {
  const gl = this._renderer.GL;

 
  gl.bindFramebuffer(gl.FRAMEBUFFER, this._computeFramebuffer);
  gl.viewport(0, 0, width, height);
  if (callback) callback(this._computeTexture);
  gl.bindFramebuffer(gl.FRAMEBUFFER, null);
}

[Vaivaswat2244]

@davepagurek any advice on this? or something I should change?

[Rishab87]

@davepagurek , if no one's worknig on this can I try solving it?

[davepagurek]

hi @Rishab87 and everyone! I think before we jump right into implementation, there are a few details to iron out, which I'd love all of your help with if you're interested!

• The general behind the scenes setup: something like what @Vaivaswat2244 suggested is a good start if we're using a fragment shader to do computation. it's possible we even want to use regular p5.Framebuffers initialized with FLOAT data to piggyback off of existing rendering code to start with, but a raw WebGL approach could be ok too later on (see also @perminder-17's recent work on a minimal filter shader runner for use in 2D mode.) One other alternative, if the main goal is to get numbers readable by JavaScript, is to use WebGL 2 transform feedback and a vertex shader, but this is less good if you'd like to read data in another shader to keep everything on the GPU, so I lean towards a fragment shader approach too.
• The interface exposed to users: how do we expect users to specify inputs and outputs? For example, if a user is doing something like a particle simulation where each particle has a position and velocity, how do we pass in the existing state, what should the output of the shader be onto the data texture, and how are we letting the user retrieve that data?
  • Can we make a default shader and use shader hooks so that users are able to write only the body of their computer shader with as little boilerplate as possible?
  • An additional technical challenge: how, in the shader, do we want to handle outputting more values than we can fit in one pixel (assuming a fragment shader is doing the computation)?
• The object model: Is there just a global function to run a compute shader, and if users alternate running different compute jobs, do we resize/reuse a single texture under the hood? Would we instead make each compute task a separate object with its own internal texture that users create upfront and then reuse? (I think I lean towards the latter, especially if we have to end up storing info about how to interpret texture output data.)

Let me know what your thoughts are!