How can I improve my use of WebGL for GPU-based nonrealtime renders?

Question

I'll give more details below but my question is in the title. How can I improve the download feature so that it works more efficiently or reliably?

I've been working on an interactive 3D viewer. It renders in real-time but with many artifacts.
https://code.sololearn.com/W6e6xWN0T7Jv/#html

The download button on the right lets you render with more quality and at a higher resolution but also much slower. It works by following these steps:
1. Render a vertical strip of just a pixel or 2 in WebGL.
2. Draw that strip onto a 2D context in another canvas.
3. Perform a short setTimeout or requestAnimationFrame to let the browser process user input.
4. Go back to step 1 until the whole image is drawn.

One weird thing I noticed is that sometimes the step 3 mentioned above is VERY slow as in 10 seconds or more to receive the next timeout or requestAnimationFrame callback specifically on the last strip. I wasn't sure if the WebGL drawing was only blocking when I needed to download the data. I experimented with drawImage on a canvas element within the body and this weird problem disappeared. When I experimented with a document.createElement('canvas') element that was not included in the document, the delay on the last strip would happen again.

WebGL is normally used for realtime rendering which means I'm fighting against what it was designed for. That leads to a few specific problems. The WebGL context can be lost if a strip is too wide and therefore takes more than a couple seconds to render.

Answer 1

The way I interpeted your question is that you run into timing issues because your main thread is blocked by heavy computations.

If you render your canvas in a worker you don't need timeouts at all, just render the thing and pass it back to the main thread.

OffscreenCanvas seems to have access to GPU acceleration but my google-fu is failing me a bit, it seems not many people are using it.

I did find this, which shows how a regular canvas performs when the ui thread is busy (I think, there are no docs), versus an OffscreenCanvas: threejs.org/examples/webgl_worker_offscreencanvas

If I interpreted your question wrongly then my bad. But copying between canvasses and things like that probably also involves a fair amount of CPU work so offloading to another thread makes sense to me.

Answer 2

I have not tried this myself but I hear you can use canvas features inside web workers. If you offload your code there you will not block the main thread at all.

EDIT: Possibly you could use one of these? https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas

Answer 3

ok. I was already using an off screen canvas before posting the question.

I'll experiment with a web worker to see if WebGL can be invoked from a web worker and the WebGL context isn't lost while running a draw operation for over 15 seconds continuously.

I never have the main JavaScript thread locked up in my downloader because the waiting for the next strip is done with a setTimeout or a requestAnimation frame but if the webgl context isn't lost with a continuous draw operation over 15 seconds, that should optimize a few things and clean up the code I was otherwise using to draw the little strips.

Answer 4

The GPU is running at nearly 100% for me when performing that high quality render procedure even in the last strip when it sometimes takes several seconds to get the next callback. Web workers are run on the CPU which is very inactive, though. How would a web worker or an offscreen canvas help when the GPU is still shared?