The approach used is fairly simple:

Code the page as you normally would (progressive enhancement FTW)

Add a full-screen fixed canvas as background

Track the position of the DOM elements you want to port into the WebGL world

Init the Meshes/Shaders and once they’re ready, and hide the original DOM elements

When scrolling, keep the position of DOM elements and WebGL in sync

Here’s some simplified code snippets:

DOM

We then create a kapla component. Kapla is a little library we use internally to bridge the gap between DOM and JS. In this case we’re mainly using for its MutationObserver implementation.

trackable.js

We then use a little class to register / unregister the dom->gl bindings. See how our data-type will load the Button class.

dom.js

We also have our Button class that extends dom3D , which in turn extends threejs Object3D 🤯

(Notice how the material and the geometry are outside the instance, so that we can leverage some optimisation and reduce WebGL state changes)

button.js

Finally, there’s the dom3D class, which is the parent class used by all the elements. The component() mixin is from bidello , and it basically enhances the class, automatically calling the methods onResize and onRaf when necessary.

dom3d.js

The main magic happens inside of the updateSize, updatePosition and onRaf functions. Those methods make sure the WebGL element is exactly the same size and position of the DOM element.

The calculateUnitSize on the PerspectiveCamera will calculate the necessary width and height (in unit size) of an element (at position vec3(0, 0, 0) ) to completely fill the camera.