Simple rules can produce structured, complex systems. And beautiful images often follow. This is the core idea behind the Game of Life, a cellular automaton devised by British mathematician John Horton Conway in 1970. Often called just ‘Life’, it’s probably one of the most popular and well known examples of cellular automata. There are many examples and tutorials on the web that go over implementing it, like this one by Daniel Shiffman.
But in many of these examples this computation runs on the CPU, limiting the possible complexity and amount of cells in the system. So this article will go over implementing the Game of Life in WebGL which allows GPU-accelerated computations (= way more complex and detailed images). Writing WebGL on its own can be very painful so it’s going to be implemented using Three.js, a WebGL graphics library. This is going to require some advanced rendering techniques, so some basic familiarity with Three.js and GLSL would be helpful in order to follow along.
Cellular Automata
Conway’s game of life is what’s called a cellular automaton and it makes sense to consider a more abstract view of what that means. This relates to automata theory in theoretical computer science, but really it’s just about creating some simple rules. A cellular automaton is a model of a system that consists of automata, called cells, that are interlinked via some simple logic which allows modelling complex behaviour. A cellular automaton has the following characteristics:
Cells live on a grid which can be 1D or higher-dimensional (in our Game of Life it’s a 2D grid of pixels)
Each cell has only one current state. Our example only has two possibilities: 0 or 1 / dead or alive
Each cell has a neighbourhood, a list of adjacent cells
The basic working principle of a cellular automaton usually involves the following steps:
An initial (global) state is selected by assigning a state for each cell.
A new generation is created, according to some fixed rule that determines the new state of each cell in terms of:
The current state of the cell
The states of cells in its neighbourhood
The state of a cell together with its neighbourhood determine the state in the next generation
As already mentioned, the Game of Life is based on a 2D grid. In its initial state there are cells which are either alive or dead. We generate the next generation of cells according to only four rules:
Any live cell with fewer than two live neighbours dies as if caused by underpopulation.
Any live cell with two or three live neighbours lives on to the next generation.
Any live cell with more than three live neighbours dies, as if by overpopulation.
Any dead cell with exactly three live neighbours becomes a live cell, as if by reproduction.
Conway’s Game of Life uses a Moore neighbourhood, which is composed of the current cell and the eight cells that surround it, so those are the ones we’ll be looking at in this example. There are many variations and possibilities to this, and Life is actually Turing complete, but this post is about implementing it in WebGL with Three.js so we will stick to a rather basic version but feel free to research more.
Three.js
Now with most of the theory out of the way, we can finally start implementing the Game of Life.
Three.js is a pretty high-level WebGL library, but it lets you decide how deep you want to go. So it provides a lot of options to control the way scenes are structured and rendered and allows users to get close to the WebGL API by writing custom shaders in GLSL and passing Buffer Attributes.
In the Game of Life each cell needs information about its neighbourhood. But in WebGL all fragments are processed simultaneously by the GPU, so when a fragment shader is in the midst of processing one pixel, there’s no way it can directly access information about any other fragments. But there’s a workaround. In a fragment shader, if we pass a texture, we can easily query the neighbouring pixels in the texture as long as we know its width and height. This idea allows all kinds of post-processing effects to be applied to scenes.
We’ll start with the initial state of the system. In order to get any interesting results, we need non-uniform starting-conditions. In this example we’ll place cells randomly on the screen, so we’ll render a regular noise texture for the first frame. Of course we could initialise with another type of noise but this is the easiest way to get started.
/**
* Sizes
*/
const sizes = {
width: window.innerWidth,
height: window.innerHeight
};
/**
* Scenes
*/
//Scene will be rendered to the screen
const scene = new THREE.Scene();
/**
* Textures
*/
//The generated noise texture
const dataTexture = createDataTexture();
/**
* Meshes
*/
// Geometry
const geometry = new THREE.PlaneGeometry(2, 2);
//Screen resolution
const resolution = new THREE.Vector3(sizes.width, sizes.height, window.devicePixelRatio);
//Screen Material
const quadMaterial = new THREE.ShaderMaterial({
uniforms: {
uTexture: { value: dataTexture },
uResolution: {
value: resolution
}
},
vertexShader: document.getElementById('vertexShader').textContent,
fragmentShader: document.getElementById('fragmentShader').textContent
});
// Meshes
const mesh = new THREE.Mesh(geometry, quadMaterial);
scene.add(mesh);
/**
* Animate
*/
const tick = () => {
//The texture will get rendere to the default framebuffer
renderer.render(scene, camera);
// Call tick again on the next frame
window.requestAnimationFrame(tick);
};
tick();
This code simply initialises a Three.js scene and adds a 2D plane to fill the screen (the snippet doesn’t show all the basic boilerplate code). The plane is supplied with a Shader Material, that for now does nothing but display a texture in its fragment shader. In this code we generate a texture using a DataTexture. It would be possible to load an image as a texture too, in that case we would need to keep track of the exact texture size. Since the scene will take up the entire screen, creating a texture with the viewport dimensions seems like the simpler solution for this tutorial. Currently the scene will be rendered to the default framebuffer (the device screen).
When writing a WebGL application, whether using the vanilla API or a higher level library like Three.js, after setting up the scene the results are rendered to the default WebGL framebuffer, which is the device screen (as done above).
But there’s also the option to create framebuffers that render off-screen, to image buffers on the GPU’s memory. Those can then be used just like a regular texture for whatever purpose. This idea is used in WebGL when it comes to creating advanced post-processing effects such as depth-of-field, bloom, etc. by applying different effects on the scene once rendered. In Three.js we can do that by using THREE.WebGLRenderTarget. We’ll call our framebuffer renderBufferA.
/**
* Scenes
*/
//Scene will be rendered to the screen
const scene = new THREE.Scene();
//Create a second scene that will be rendered to the off-screen buffer
const bufferScene = new THREE.Scene();
/**
* Render Buffers
*/
// Create a new framebuffer we will use to render to
// the GPU memory
let renderBufferA = new THREE.WebGLRenderTarget(sizes.width, sizes.height, {
// Below settings hold the uv coordinates and retain precision.
minFilter: THREE.NearestFilter,
magFilter: THREE.NearestFilter,
format: THREE.RGBAFormat,
type: THREE.FloatType,
stencilBuffer: false
});
//Screen Material
const quadMaterial = new THREE.ShaderMaterial({
uniforms: {
//Now the screen material won't get a texture initially
//The idea is that this texture will be rendered off-screen
uTexture: { value: null },
uResolution: {
value: resolution
}
},
vertexShader: document.getElementById('vertexShader').textContent,
fragmentShader: document.getElementById('fragmentShader').textContent
});
//off-screen Framebuffer will receive a new ShaderMaterial
// Buffer Material
const bufferMaterial = new THREE.ShaderMaterial({
uniforms: {
uTexture: { value: dataTexture },
uResolution: {
value: resolution
}
},
vertexShader: document.getElementById('vertexShader').textContent,
//For now this fragment shader does the same as the one used above
fragmentShader: document.getElementById('fragmentShaderBuffer').textContent
});
/**
* Animate
*/
const tick = () => {
// Explicitly set renderBufferA as the framebuffer to render to
//the output of this rendering pass will be stored in the texture associated with renderBufferA
renderer.setRenderTarget(renderBufferA);
// This will the off-screen texture
renderer.render(bufferScene, camera);
mesh.material.uniforms.uTexture.value = renderBufferA.texture;
//This will set the default framebuffer (i.e. the screen) back to being the output
renderer.setRenderTarget(null);
//Render to screen
renderer.render(scene, camera);
// Call tick again on the next frame
window.requestAnimationFrame(tick);
};
tick();
Now there’s nothing to be seen because, while the scene is rendered, it’s rendered to an off-screen buffer.
We’ll need to access it as a texture in the animation loop to render the generated texture from the previous step to the fullscreen plane on our screen.
//In the animation loop before rendering to the screen
mesh.material.uniforms.uTexture.value = renderBufferA.texture;
And that’s all it takes to get back the noise, except now it’s rendered off-screen and the output of that render is used as a texture in the framebuffer that renders on to the screen.
Now that there’s data rendered to a texture, the shaders can be used to perform general computation using the texture data. Within GLSL, textures are read-only, and we can’t write directly to our input textures, we can only “sample” them. Using the off-screen framebuffer, however, we can use the output of the shader itself to write to a texture. Then, if we can chain together multiple rendering passes, the output of one rendering pass becomes the input for the next pass. So we create two off-screen buffers. This technique is called ping pong buffering. We create a kind of simple ring buffer, where after every frame we swap the off-screen buffer that is being read from with the off-screen buffer that is being written to. We can then use the off-screen buffer that was just written to, and display that to the screen. This lets us perform iterative computation on the GPU, which is useful for all kinds of effects.
To achieve it in THREE.js, first we need to create a second framebuffer. We will call it renderBufferB. Then the ping-pong technique is actually performed in the animation loop.
//Add another framebuffer
let renderBufferB = new THREE.WebGLRenderTarget(
sizes.width,
sizes.height,
{
minFilter: THREE.NearestFilter,
magFilter: THREE.NearestFilter,
format: THREE.RGBAFormat,
type: THREE.FloatType,
stencilBuffer: false
}
//At the end of each animation loop
// Ping-pong the framebuffers by swapping them
// at the end of each frame render
// Now prepare for the next cycle by swapping renderBufferA and renderBufferB
// so that the previous frame's *output* becomes the next frame's *input*
const temp = renderBufferA
renderBufferA = renderBufferB
renderBufferB = temp
//output becomes input
bufferMaterial.uniforms.uTexture.value = renderBufferB.texture;
)
Now the render buffers are swapped every frame, it’ll look the same but it’s possible to verify by logging out the textures that get passed to the on-screen plane each frame for example. Here’s a more in depth look at ping pong buffers in WebGL.
From here it’s about implementing the actual Game of Life. Since the rules are so simple, the resulting code isn’t very complicated either and there’s many good resources that go through coding it up, so I’ll only go over the key ideas. All the logic for this will happen in the fragment shader that gets rendered off-screen, which will provide the texture for the next frame.
As described earlier, we want to access neighbouring fragments (or pixels) via the texture that’s passed in. This is achieved in a nested for loop in the getNeighbours function. We skip our current cell and check the 8 surrounding pixels by sampling the texture at an offset. Then we check whether the pixels r value is above 0.5, which means it’s alive, and increment the count to represent the alive neighbours.
//GLSL in fragment shader
precision mediump float;
//The input texture
uniform sampler2D uTexture;
//Screen resolution
uniform vec3 uResolution;
// uv coordinates passed from vertex shader
varying vec2 vUvs;
float GetNeighbours(vec2 p) {
float count = 0.0;
for(float y = -1.0; y <= 1.0; y++) {
for(float x = -1.0; x <= 1.0; x++) {
if(x == 0.0 && y == 0.0)
continue;
// Scale the offset down
vec2 offset = vec2(x, y) / uResolution.xy;
// Apply offset and sample texture
vec4 lookup = texture2D(uTexture, p + offset);
// Accumulate the result
count += lookup.r > 0.5 ? 1.0 : 0.0;
}
}
return count;
}
Based on this count we can set the rules. (Note how we can use the standard UV coordinates here because the Texture we created in the beginning fills the screen. If we had initialised with an image texture of arbitrary dimensions, we’d need to scale coordinates according to its exact pixel size to get a value between 0.0 and 1.0)
//In the main function
vec3 color = vec3(0.0);
float neighbors = 0.0;
neighbors += GetNeighbours(vUvs);
bool alive = texture2D(uTexture, vUvs).x > 0.5;
//cell is alive
if(alive && (neighbors == 2.0 || neighbors == 3.0)) {
//Any live cell with two or three live neighbours lives on to the next generation.
color = vec3(1.0, 0.0, 0.0);
//cell is dead
} else if (!alive && (neighbors == 3.0)) {
//Any dead cell with exactly three live neighbours becomes a live cell, as if by reproduction.
color = vec3(1.0, 0.0, 0.0);
}
//In all other cases cell remains dead or dies so color stays at 0
gl_FragColor = vec4(color, 1.0);
And that’s basically it, a working Game of Life using only GPU shaders, written in Three.js. The texture will get sampled every frame via the ping pong buffers, which creates the next generation in our cellular automaton, so no additional variable tracking the time or frames needs to be passed for it to animate.
In summary, we first went over the basic ideas behind cellular automata, which is a very powerful model of computation used to generate complex behaviour. Then we were able to implement it in Three.js using ping pong buffering and framebuffers. Now there’s near endless possibilities for taking it further, try adding different rules or mouse interaction for example.
oneAPI is an open, unified programming model designed to simplify development and deployment of data-centric workloads across central processing units (CPUs), graphics processing units (GPUs), field-programmable gate arrays (FPGAs), and other accelerators. In a heterogeneous compute environment, developers need to understand the capabilities and limitations of each compute architecture to effectively match the appropriate workload to each compute device.
For a number of reasons, you cannot transfer high-throughput equipment and resource-intensive software home, but you can still organize high-quality remote access from anywhere at no extra cost. We are going to tell you about the first way we have tested for maintaining convenient remote management from almost any device.
What’s Up, Doc?
The average employee just needs to connect to the remote desktop using the RDP protocol in order to access corporate resources from a laptop, and herein lies the problem for IT specialists: ensuring security. If a specialist needs resource-intensive applications that use 3D acceleration, this is a problem of a completely different kind.
As technology advances and more organizations are implementing machine learning operations (MLOps), people are looking for ways to speed up processes. This is especially true for organizations working with deep learning (DL) processes which can be incredibly long to run. You can speed up this process by using graphical processing units (GPUs) on-premises or in the cloud.
GPUs are microprocessors that are specially designed to perform specific tasks. These units enable parallel processing of tasks and can be optimized to increase performance in artificial intelligence and deep learning processes.
If you’re familiar with the growth of ML/AI development in recent years, you’re likely to be aware of leveraging GPUs to speed up the intensive calculations required for tasks like Deep Learning. Using GPUs with Kubernetes allows you to extend the scalability of K8s to ML applications.
However, Kubernetes does not inherently have the ability to schedule GPU resources, so this approach requires the use of third-party device plugins. Additionally, there is no native way to determine utilization, per-device request statistics, or other metrics—this information is an important input to analyzing GPU efficiency and cost, which can be a significant expenditure.
Learn how to set up a GPU-enabled virtual server instance (VSI) on a Virtual Private Cloud (VPC) and deploy RAPIDS using IBM Schematics.
The GPU-enabled family of profiles provides on-demand, cost-effective access to NVIDIA GPUs. GPUs help to accelerate the processing time required for compute-intensive workloads, such as artificial intelligence (AI), machine learning, inferencing, and more. To use the GPUs, you need the appropriate toolchain - such as CUDA (an acronym for Compute Unified Device Architecture) - ready.
In this article, Dirk Weber explains the SVG `feDisplacementMap` filter primitive with a good number of examples to demonstrate the concept of animated displacement maps.
In this tutorial you’ll learn how CSS keyframes work from the ground up, and see how you can use them to build high-quality animations. By Josh W Comeau.
A simple color tool to help you find good color palettes for your web projects. This tools spits out modern CSS you can use right away in your projects.
Framebuffers are a key feature in WebGL when it comes to creating advanced graphical effects such as depth-of-field, bloom, film grain or various types of anti-aliasing and have already been covered in-depth here on Codrops. They allow us to “post-process” our scenes, applying different effects on them once rendered. But how exactly do they work?
By default, WebGL (and also Three.js and all other libraries built on top of it) render to the default framebuffer, which is the device screen. If you have used Three.js or any other WebGL framework before, you know that you create your mesh with the correct geometry and material, render it, and voilà, it’s visible on your screen.
However, we as developers can create new framebuffers besides the default one and explicitly instruct WebGL to render to them. By doing so, we render our scenes to image buffers in the video card’s memory instead of the device screen. Afterwards, we can treat these image buffers like regular textures and apply filters and effects before eventually rendering them to the device screen.
Here is a video breaking down the post-processing and effects in Metal Gear Solid 5: Phantom Pain that really brings home the idea. Notice how it starts by footage from the actual game rendered to the default framebuffer (device screen) and then breaks down how each framebuffer looks like. All of these framebuffers are composited together on each frame and the result is the final picture you see when playing the game:
So with the theory out of the way, let’s create a cool typography motion trail effect by rendering to a framebuffer!
Our skeleton app
Let’s render some 2D text to the default framebuffer, i.e. device screen, using threejs. Here is our boilerplate:
const LABEL_TEXT = 'ABC'
const clock = new THREE.Clock()
const scene = new THREE.Scene()
// Create a threejs renderer:
// 1. Size it correctly
// 2. Set default background color
// 3. Append it to the page
const renderer = new THREE.WebGLRenderer()
renderer.setClearColor(0x222222)
renderer.setClearAlpha(0)
renderer.setSize(innerWidth, innerHeight)
renderer.setPixelRatio(devicePixelRatio || 1)
document.body.appendChild(renderer.domElement)
// Create an orthographic camera that covers the entire screen
// 1. Position it correctly in the positive Z dimension
// 2. Orient it towards the scene center
const orthoCamera = new THREE.OrthographicCamera(
-innerWidth / 2,
innerWidth / 2,
innerHeight / 2,
-innerHeight / 2,
0.1,
10,
)
orthoCamera.position.set(0, 0, 1)
orthoCamera.lookAt(new THREE.Vector3(0, 0, 0))
// Create a plane geometry that spawns either the entire
// viewport height or width depending on which one is bigger
const labelMeshSize = innerWidth > innerHeight ? innerHeight : innerWidth
const labelGeometry = new THREE.PlaneBufferGeometry(
labelMeshSize,
labelMeshSize
)
// Programmaticaly create a texture that will hold the text
let labelTextureCanvas
{
// Canvas and corresponding context2d to be used for
// drawing the text
labelTextureCanvas = document.createElement('canvas')
const labelTextureCtx = labelTextureCanvas.getContext('2d')
// Dynamic texture size based on the device capabilities
const textureSize = Math.min(renderer.capabilities.maxTextureSize, 2048)
const relativeFontSize = 20
// Size our text canvas
labelTextureCanvas.width = textureSize
labelTextureCanvas.height = textureSize
labelTextureCtx.textAlign = 'center'
labelTextureCtx.textBaseline = 'middle'
// Dynamic font size based on the texture size
// (based on the device capabilities)
labelTextureCtx.font = `${relativeFontSize}px Helvetica`
const textWidth = labelTextureCtx.measureText(LABEL_TEXT).width
const widthDelta = labelTextureCanvas.width / textWidth
const fontSize = relativeFontSize * widthDelta
labelTextureCtx.font = `${fontSize}px Helvetica`
labelTextureCtx.fillStyle = 'white'
labelTextureCtx.fillText(LABEL_TEXT, labelTextureCanvas.width / 2, labelTextureCanvas.height / 2)
}
// Create a material with our programmaticaly created text
// texture as input
const labelMaterial = new THREE.MeshBasicMaterial({
map: new THREE.CanvasTexture(labelTextureCanvas),
transparent: true,
})
// Create a plane mesh, add it to the scene
const labelMesh = new THREE.Mesh(labelGeometry, labelMaterial)
scene.add(labelMesh)
// Start out animation render loop
renderer.setAnimationLoop(onAnimLoop)
function onAnimLoop() {
// On each new frame, render the scene to the default framebuffer
// (device screen)
renderer.render(scene, orthoCamera)
}
This code simply initialises a threejs scene, adds a 2D plane with a text texture to it and renders it to the default framebuffer (device screen). If we are execute it with threejs included in our project, we will get this:
Again, we don’t explicitly specify otherwise, so we are rendering to the default framebuffer (device screen).
Now that we managed to render our scene to the device screen, let’s add a framebuffer (THEEE.WebGLRenderTarget) and render it to a texture in the video card memory.
Rendering to a framebuffer
Let’s start by creating a new framebuffer when we initialise our app:
const clock = new THREE.Clock()
const scene = new THREE.Scene()
// Create a new framebuffer we will use to render to
// the video card memory
const renderBufferA = new THREE.WebGLRenderTarget(
innerWidth * devicePixelRatio,
innerHeight * devicePixelRatio
)
// ... rest of application
Now that we have created it, we must explicitly instruct threejs to render to it instead of the default framebuffer, i.e. device screen. We will do this in our program animation loop:
function onAnimLoop() {
// Explicitly set renderBufferA as the framebuffer to render to
renderer.setRenderTarget(renderBufferA)
// On each new frame, render the scene to renderBufferA
renderer.render(scene, orthoCamera)
}
As you can see, we are getting an empty screen, yet our program contains no errors – so what happened? Well, we are no longer rendering to the device screen, but another framebuffer! Our scene is being rendered to a texture in the video card memory, so that’s why we see the empty screen.
In order to display this generated texture containing our scene back to the default framebuffer (device screen), we need to create another 2D plane that will cover the entire screen of our app and pass the texture as material input to it.
First we will create a fullscreen 2D plane that will span the entire device screen:
// ... rest of initialisation step
// Create a second scene that will hold our fullscreen plane
const postFXScene = new THREE.Scene()
// Create a plane geometry that covers the entire screen
const postFXGeometry = new THREE.PlaneBufferGeometry(innerWidth, innerHeight)
// Create a plane material that expects a sampler texture input
// We will pass our generated framebuffer texture to it
const postFXMaterial = new THREE.ShaderMaterial({
uniforms: {
sampler: { value: null },
},
// vertex shader will be in charge of positioning our plane correctly
vertexShader: `
varying vec2 v_uv;
void main () {
// Set the correct position of each plane vertex
gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);
// Pass in the correct UVs to the fragment shader
v_uv = uv;
}
`,
fragmentShader: `
// Declare our texture input as a "sampler" variable
uniform sampler2D sampler;
// Consume the correct UVs from the vertex shader to use
// when displaying the generated texture
varying vec2 v_uv;
void main () {
// Sample the correct color from the generated texture
vec4 inputColor = texture2D(sampler, v_uv);
// Set the correct color of each pixel that makes up the plane
gl_FragColor = inputColor;
}
`
})
const postFXMesh = new THREE.Mesh(postFXGeometry, postFXMaterial)
postFXScene.add(postFXMesh)
// ... animation loop code here, same as before
As you can see, we are creating a new scene that will hold our fullscreen plane. After creating it, we need to augment our animation loop to render the generated texture from the previous step to the fullscreen plane on our screen:
function onAnimLoop() {
// Explicitly set renderBufferA as the framebuffer to render to
renderer.setRenderTarget(renderBufferA)
// On each new frame, render the scene to renderBufferA
renderer.render(scene, orthoCamera)
// 👇
// Set the device screen as the framebuffer to render to
// In WebGL, framebuffer "null" corresponds to the default
// framebuffer!
renderer.setRenderTarget(null)
// 👇
// Assign the generated texture to the sampler variable used
// in the postFXMesh that covers the device screen
postFXMesh.material.uniforms.sampler.value = renderBufferA.texture
// 👇
// Render the postFX mesh to the default framebuffer
renderer.render(postFXScene, orthoCamera)
}
After including these snippets, we can see our scene once again rendered on the screen:
Let’s recap the necessary steps needed to produce this image on our screen on each render loop:
Create renderTargetA framebuffer that will allow us to render to a separate texture in the users device video memory
Create our “ABC” plane mesh
Render the “ABC” plane mesh to renderTargetA instead of the device screen
Create a separate fullscreen plane mesh that expects a texture as an input to its material
Render the fullscreen plane mesh back to the default framebuffer (device screen) using the generated texture created by rendering the “ABC” mesh to renderTargetA
Achieving the persistence effect by using two framebuffers
We don’t have much use of framebuffers if we are simply displaying them as they are to the device screen, as we do right now. Now that we have our setup ready, let’s actually do some cool post-processing.
First, we actually want to create yet another framebuffer – renderTargetB, and make sure it and renderTargetA are let variables, rather then consts. That’s because we will actually swap them at the end of each render so we can achieve framebuffer ping-ponging.
“Ping-ponging” in WebGl is a technique that alternates the use of a framebuffer as either input or output. It is a neat trick that allows for general purpose GPU computations and is used in effects such as gaussian blur, where in order to blur our scene we need to:
Render it to framebuffer A using a 2D plane and apply horizontal blur via the fragment shader
Render the result horizontally blurred image from step 1 to framebuffer B and apply vertical blur via the fragment shader
Swap framebuffer A and framebuffer B
Keep repeating steps 1 to 3 and incrementally applying blur until desired gaussian blur radius is achieved.
Here is a small chart illustrating the steps needed to achieve ping-pong:
So with that in mind, we will render the contents of renderTargetA into renderTargetB using the postFXMesh we created and apply some special effect via the fragment shader.
Let’s kick things off by creating our renderTargetB:
let renderBufferA = new THREE.WebGLRenderTarget(
// ...
)
// Create a second framebuffer
let renderBufferB = new THREE.WebGLRenderTarget(
innerWidth * devicePixelRatio,
innerHeight * devicePixelRatio
)
Next up, let’s augment our animation loop to actually do the ping-pong technique:
function onAnimLoop() {
// 👇
// Do not clear the contents of the canvas on each render
// In order to achieve our ping-pong effect, we must draw
// the new frame on top of the previous one!
renderer.autoClearColor = false
// 👇
// Explicitly set renderBufferA as the framebuffer to render to
renderer.setRenderTarget(renderBufferA)
// 👇
// Render the postFXScene to renderBufferA.
// This will contain our ping-pong accumulated texture
renderer.render(postFXScene, orthoCamera)
// 👇
// Render the original scene containing ABC again on top
renderer.render(scene, orthoCamera)
// Same as before
// ...
// ...
// 👇
// Ping-pong our framebuffers by swapping them
// at the end of each frame render
const temp = renderBufferA
renderBufferA = renderBufferB
renderBufferB = temp
}
If we are to render our scene again with these updated snippets, we will see no visual difference, even though we do in fact alternate between the two framebuffers to render it. That’s because, as it is right now, we do not apply any special effects in the fragment shader of our postFXMesh.
Let’s change our fragment shader like so:
// Sample the correct color from the generated texture
// 👇
// Notice how we now apply a slight 0.005 offset to our UVs when
// looking up the correct texture color
vec4 inputColor = texture2D(sampler, v_uv + vec2(0.005));
// Set the correct color of each pixel that makes up the plane
// 👇
// We fade out the color from the previous step to 97.5% of
// whatever it was before
gl_FragColor = vec4(inputColor * 0.975);
With these changes in place, here is our updated program:
Let’s break down one frame render of our updated example:
We render renderTargetB result to renderTargetA
We render our “ABC” text to renderTargetA, compositing it on top of renderTargetB result in step 1 (we do not clear the contents of the canvas on new renders, because we set renderer.autoClearColor = false)
We pass the generated renderTargetA texture to postFXMesh, apply a small offset vec2(0.002) to its UVs when looking up the texture color and fade it out a bit by multiplying the result by 0.975
We render postFXMesh to the device screen
We swap renderTargetA with renderTargetB (ping-ponging)
For each new frame render, we will repeat steps 1 to 5. This way, the previous target framebuffer we rendered to will be used as an input to the current render and so on. You can clearly see this effect visually in the last demo – notice how as the ping-ponging progresses, more and more offset is being applied to the UVs and more and more the opacity fades out.
Applying simplex noise and mouse interaction
Now that we have implemented and can see the ping-pong technique working correctly, we can get creative and expand on it.
Instead of simply adding an offset in our fragment shader as before:
Let’s actually use simplex noise for more interesting visual result. We will also control the direction using our mouse position.
Here is our updated fragment shader:
// Pass in elapsed time since start of our program
uniform float time;
// Pass in normalised mouse position
// (-1 to 1 horizontally and vertically)
uniform vec2 mousePos;
// <Insert snoise function definition from the link above here>
// Calculate different offsets for x and y by using the UVs
// and different time offsets to the snoise method
float a = snoise(vec3(v_uv * 1.0, time * 0.1)) * 0.0032;
float b = snoise(vec3(v_uv * 1.0, time * 0.1 + 100.0)) * 0.0032;
// Add the snoise offset multiplied by the normalised mouse position
// to the UVs
vec4 inputColor = texture2D(sampler, v_uv + vec2(a, b) + mousePos * 0.005);
We also need to specify mousePos and time as inputs to our postFXMesh material shader:
Finally let’s make sure we attach a mousemove event listener to our page and pass the updated normalised mouse coordinates from Javascript to our GLSL fragment shader:
// ... initialisation step
// Attach mousemove event listener
document.addEventListener('mousemove', onMouseMove)
function onMouseMove (e) {
// Normalise horizontal mouse pos from -1 to 1
const x = (e.pageX / innerWidth) * 2 - 1
// Normalise vertical mouse pos from -1 to 1
const y = (1 - e.pageY / innerHeight) * 2 - 1
// Pass normalised mouse coordinates to fragment shader
postFXMesh.material.uniforms.mousePos.value.set(x, y)
}
// ... animation loop
With these changes in place, here is our final result. Make sure to hover around it (you might have to wait a moment for everything to load):
Framebuffers are a powerful tool in WebGL that allows us to greatly enhance our scenes via post-processing and achieve all kinds of cool effects. Some techniques require more then one framebuffer as we saw and it is up to us as developers to mix and match them however we need to achieve our desired visuals.
I encourage you to experiment with the provided examples, try to render more elements, alternate the “ABC” text color between each renderTargetA and renderTargetB swap to achieve different color mixing, etc.
In the first demo, you can see a specific example of how this typography effect could be used and the second demo is a playground for you to try some different settings (just open the controls in the top right corner).
GPUs Continue to Expand Application Use in Artificial Intelligence and Machine Learning
Artificial intelligence (AI) is set to transform global productivity, working patterns, and lifestyles and create enormous wealth. Research firm Gartner expects the global AI economy to increase from about $1.2 trillion last year to about $3.9 Trillion by 2022, while McKinsey sees it delivering global economic activity of around $13 trillion by 2030. And of course, this transformation is fueled by the powerful Machine Learning (ML) tools and techniques such as Deep Reinforcement Learning (DRL), Generative Adversarial Networks (GAN), Gradient-boosted-tree models (GBM), Natural Language Processing (NLP), and more.
Most of the success in modern AI and ML systems is dependent on their ability to process massive amounts of raw data in a parallel fashion using task-optimized hardware. In fact, the modern resurgence of AI started with the 2012 ImageNet competition where deep-learning algorithms demonstrated an eye-popping increment in the image classification accuracy over their non-deep-learning counterparts (algorithms). However, along with clever programming and mathematical modeling,the use of specialized hardware played a significant role in this early success.
While many people shy away from writing vanilla WebGL and immediately jump to frameworks such as three.js or PixiJS, it is possible to achieve great visuals and complex animation with relatively small amounts of code. Today, I would like to present core WebGL concepts while programming some simple 2D visuals. This article assumes at least some higher-level knowledge of WebGL through a library.
Please note: WebGL2 has been around for years, yet Safari only recently enabled it behind a flag. It is a pretty significant upgrade from WebGL1 and brings tons of new useful features, some of which we will take advantage of in this tutorial.
What are we going to build
From a high level standpoint, to implement our 2D metaballs we need two steps:
Draw a bunch of rectangles with radial linear gradient starting from their centers and expanding to their edges. Draw a lot of them and alpha blend them together in a separate framebuffer.
Take the resulting image with the blended quads from step #1, scan its pixels one by one and decide the new color of the pixel depending on its opacity. For example – if the pixel has opacity smaller then 0.5, render it in red. Otherwise render it in yellow and so on.
Left: Multiple quads rendered with radial gradient, alpha blended and rendered to a texture. Right: Post-processing on the generated texture and rendering the result to the device screen. Conditional coloring of each pixel based on opacity.
Don’t worry if these terms don’t make a lot of sense just yet – we will go over each of the steps needed in detail. Let’s jump into the code and start building!
Bootstrapping our program
We will start things by
Creating a HTMLCanvasElement, sizing it to our device viewport and inserting it into the page DOM
Setting the correct WebGL viewport and the background color for our scene
Starting a requestAnimationFrame loop that will draw our scene as fast as the device allows. The speed is determined by various factors such as the hardware, current CPU / GPU workloads, battery levels, user preferences and so on. For smooth animation we are going to aim for 60FPS.
/* Create our canvas and obtain it's WebGL2RenderingContext */
const canvas = document.createElement('canvas')
const gl = canvas.getContext('webgl2')
/* Handle error somehow if no WebGL2 support */
if (!gl) {
// ...
}
/* Size our canvas and listen for resize events */
resizeCanvas()
window.addEventListener('resize', resizeCanvas)
/* Append our canvas to the DOM and set its background-color with CSS */
canvas.style.backgroundColor = 'black'
document.body.appendChild(canvas)
/* Issue first frame paint */
requestAnimationFrame(updateFrame)
function updateFrame (timestampMs) {
/* Set our program viewport to fit the actual size of our monitor with devicePixelRatio into account */
gl.viewport(0, 0, canvas.width, canvas.height)
/* Set the WebGL background colour to be transparent */
gl.clearColor(0, 0, 0, 0)
/* Clear the current canvas pixels */
gl.clear(gl.COLOR_BUFFER_BIT)
/* Issue next frame paint */
requestAnimationFrame(updateFrame)
}
function resizeCanvas () {
/*
We need to account for devicePixelRatio when sizing our canvas.
We will use it to obtain the actual pixel size of our viewport and size our canvas to match it.
We will then downscale it back to CSS units so it neatly fills our viewport and we benefit from downsampling antialiasing
We also need to limit it because it can really slow our program. Modern iPhones have devicePixelRatios of 3. This means rendering 9x more pixels each frame!
More info: https://webglfundamentals.org/webgl/lessons/webgl-resizing-the-canvas.html
*/
const dpr = devicePixelRatio > 2 ? 2 : devicePixelRatio
canvas.width = innerWidth * dpr
canvas.height = innerHeight * dpr
canvas.style.width = `${innerWidth}px`
canvas.style.height = `${innerHeight}px`
}
Drawing a quad
The next step is to actually draw a shape. WebGL has a rendering pipeline, which dictates how does the object you draw and its corresponding geometry and material end up on the device screen. WebGL is essentially just a rasterising engine, in the sense that you give it properly formatted data and it produces pixels for you.
The full rendering pipeline is out of the scope for this tutorial, but you can read more about it here. Let’s break down what exactly we need for our program:
Defining our geometry and its attributes
Each object we draw in WebGL is represented as a WebGLProgram running on the device GPU. It consists of input variables and vertex and fragment shader to operate on these variables. The vertex shader responsibility is to position our geometry correctly on the device screen and fragment shader’s responsibility is to control its appearance.
It’s up to us as developers to write our vertex and fragment shaders, compile them on the device GPU and link them in a GLSL program. Once we have successfully done this, we must query this program’s input variable locations that were allocated on the GPU for us, supply correctly formatted data to them, enable them and instruct them how to unpack and use our data.
To render our quad, we need 3 input variables:
a_position will dictate the position of each vertex of our quad geometry. We will pass it as an array of 12 floats, i.e. 2 triangles with 3 points per triangle, each represented by 2 floats (x, y). This variable is an attribute, i.e. it is obviously different for each of the points that make up our geometry.
a_uv will describe the texture offset for each point of our geometry. They too will be described as an array of 12 floats. We will use this data not to texture our quad with an image, but to dynamically create a radial linear gradient from the quad center. This variable is also an attribute and will too be different for each of our geometry points.
u_projectionMatrix will be an input variable represented as a 32bit float array of 16 items that will dictate how do we transform our geometry positions described in pixel values to the normalised WebGL coordinate system. This variable is a uniform, unlike the previous two, it will not change for each geometry position.
We can take advantage of Vertex Array Object to store the description of our GLSL program input variables, their locations on the GPU and how should they be unpacked and used.
WebGLVertexArrayObjects or VAOs are 1st class citizens in WebGL2, unlike in WebGL1 where they were hidden behind an optional extension and their support was not guaranteed. They let us type less, execute fewer WebGL bindings and keep our drawing state into a single, easy to manage object that is simpler to track. They essentially store the description of our geometry and we can reference them later.
We need to write the shaders in GLSL 3.00 ES, which WebGL2 supports. Our vertex shader will be pretty simple:
/*
Pass in geometry position and tex coord from the CPU
*/
in vec4 a_position;
in vec2 a_uv;
/*
Pass in global projection matrix for each vertex
*/
uniform mat4 u_projectionMatrix;
/*
Specify varying variable to be passed to fragment shader
*/
out vec2 v_uv;
void main () {
/*
We need to convert our quad points positions from pixels to the normalized WebGL coordinate system
*/
gl_Position = u_projectionMatrix * a_position;
v_uv = a_uv;
}
At this point, after we have successfully executed our vertex shader, WebGL will fill in the pixels between the points that make up the geometry on the device screen. The way the space between the points is filled depends on what primitives are we using for drawing – WebGL supports points, lines and triangles.
We as developers do not have control over this step.
After it has rasterised our geometry, it will execute our fragment shader on each generated pixel. The fragment shader responsibility is the final appearance of each generated pixel and wether it should even be rendered. Here is our fragment shader:
/*
Set fragment shader float precision
*/
precision highp float;
/*
Consume interpolated tex coord varying from vertex shader
*/
in vec2 v_uv;
/*
Final color represented as a vector of 4 components - r, g, b, a
*/
out vec4 outColor;
void main () {
/*
This function will run on each each pixel generated by our quad geometry
*/
/*
Calculate the distance for each pixel from the center of the quad (0.5, 0.5)
*/
float dist = distance(v_uv, vec2(0.5)) * 2.0;
/*
Invert and clamp our distance from 0.0 to 1.0
*/
float c = clamp(1.0 - dist, 0.0, 1.0);
/*
Use the distance to generate the pixel opacity. We have to explicitly enable alpha blending in WebGL to see the correct result
*/
outColor = vec4(vec3(1.0), c);
}
Let’s write two utility methods: makeGLShader() to create and compile our GLSL shaders and makeGLProgram() to link them into a GLSL program to be ran on the GPU:
/*
Utility method to create a WebGLShader object and compile it on the device GPU
https://developer.mozilla.org/en-US/docs/Web/API/WebGLShader
*/
function makeGLShader (shaderType, shaderSource) {
/* Create a WebGLShader object with correct type */
const shader = gl.createShader(shaderType)
/* Attach the shaderSource string to the newly created shader */
gl.shaderSource(shader, shaderSource)
/* Compile our newly created shader */
gl.compileShader(shader)
const success = gl.getShaderParameter(shader, gl.COMPILE_STATUS)
/* Return the WebGLShader if compilation was a success */
if (success) {
return shader
}
/* Otherwise log the error and delete the faulty shader */
console.error(gl.getShaderInfoLog(shader))
gl.deleteShader(shader)
}
/*
Utility method to create a WebGLProgram object
It will create both a vertex and fragment WebGLShader and link them into a program on the device GPU
https://developer.mozilla.org/en-US/docs/Web/API/WebGLProgram
*/
function makeGLProgram (vertexShaderSource, fragmentShaderSource) {
/* Create and compile vertex WebGLShader */
const vertexShader = makeGLShader(gl.VERTEX_SHADER, vertexShaderSource)
/* Create and compile fragment WebGLShader */
const fragmentShader = makeGLShader(gl.FRAGMENT_SHADER, fragmentShaderSource)
/* Create a WebGLProgram and attach our shaders to it */
const program = gl.createProgram()
gl.attachShader(program, vertexShader)
gl.attachShader(program, fragmentShader)
/* Link the newly created program on the device GPU */
gl.linkProgram(program)
/* Return the WebGLProgram if linking was successfull */
const success = gl.getProgramParameter(program, gl.LINK_STATUS)
if (success) {
return program
}
/* Otherwise log errors to the console and delete fauly WebGLProgram */
console.error(gl.getProgramInfoLog(program))
gl.deleteProgram(program)
}
And here is the complete code snippet we need to add to our previous code snippet to generate our geometry, compile our shaders and link them into a GLSL program:
const canvas = document.createElement('canvas')
/* rest of code */
/* Enable WebGL alpha blending */
gl.enable(gl.BLEND)
gl.blendFunc(gl.SRC_ALPHA, gl.ONE_MINUS_SRC_ALPHA)
/*
Generate the Vertex Array Object and GLSL program
we need to render our 2D quad
*/
const {
quadProgram,
quadVertexArrayObject,
} = makeQuad(innerWidth / 2, innerHeight / 2)
/* --------------- Utils ----------------- */
function makeQuad (positionX, positionY, width = 50, height = 50, drawType = gl.STATIC_DRAW) {
/*
Write our vertex and fragment shader programs as simple JS strings
!!! Important !!!!
WebGL2 requires GLSL 3.00 ES
We need to declare this version on the FIRST LINE OF OUR PROGRAM
Otherwise it would not work!
*/
const vertexShaderSource = `#version 300 es
/*
Pass in geometry position and tex coord from the CPU
*/
in vec4 a_position;
in vec2 a_uv;
/*
Pass in global projection matrix for each vertex
*/
uniform mat4 u_projectionMatrix;
/*
Specify varying variable to be passed to fragment shader
*/
out vec2 v_uv;
void main () {
gl_Position = u_projectionMatrix * a_position;
v_uv = a_uv;
}
`
const fragmentShaderSource = `#version 300 es
/*
Set fragment shader float precision
*/
precision highp float;
/*
Consume interpolated tex coord varying from vertex shader
*/
in vec2 v_uv;
/*
Final color represented as a vector of 4 components - r, g, b, a
*/
out vec4 outColor;
void main () {
float dist = distance(v_uv, vec2(0.5)) * 2.0;
float c = clamp(1.0 - dist, 0.0, 1.0);
outColor = vec4(vec3(1.0), c);
}
`
/*
Construct a WebGLProgram object out of our shader sources and link it on the GPU
*/
const quadProgram = makeGLProgram(vertexShaderSource, fragmentShaderSource)
/*
Create a Vertex Array Object that will store a description of our geometry
that we can reference later when rendering
*/
const quadVertexArrayObject = gl.createVertexArray()
/*
1. Defining geometry positions
Create the geometry points for our quad
V6 _______ V5 V3
| / /|
| / / |
| / / |
V4 |/ V1 /______| V2
We need two triangles to form a single quad
As you can see, we end up duplicating vertices:
V5 & V3 and V4 & V1 end up occupying the same position.
There are better ways to prepare our data so we don't end up with
duplicates, but let's keep it simple for this demo and duplicate them
Unlike regular Javascript arrays, WebGL needs strongly typed data
That's why we supply our positions as an array of 32 bit floating point numbers
*/
const vertexArray = new Float32Array([
/*
First set of 3 points are for our first triangle
*/
positionX - width / 2, positionY + height / 2, // Vertex 1 (X, Y)
positionX + width / 2, positionY + height / 2, // Vertex 2 (X, Y)
positionX + width / 2, positionY - height / 2, // Vertex 3 (X, Y)
/*
Second set of 3 points are for our second triangle
*/
positionX - width / 2, positionY + height / 2, // Vertex 4 (X, Y)
positionX + width / 2, positionY - height / 2, // Vertex 5 (X, Y)
positionX - width / 2, positionY - height / 2 // Vertex 6 (X, Y)
])
/*
Create a WebGLBuffer that will hold our triangles positions
*/
const vertexBuffer = gl.createBuffer()
/*
Now that we've created a GLSL program on the GPU we need to supply data to it
We need to supply our 32bit float array to the a_position variable used by the GLSL program
When you link a vertex shader with a fragment shader by calling gl.linkProgram(someProgram)
WebGL (the driver/GPU/browser) decide on their own which index/location to use for each attribute
Therefore we need to find the location of a_position from our program
*/
const a_positionLocationOnGPU = gl.getAttribLocation(quadProgram, 'a_position')
/*
Bind the Vertex Array Object descriptior for this geometry
Each geometry instruction from now on will be recorded under it
To stop recording after we are done describing our geometry, we need to simply unbind it
*/
gl.bindVertexArray(quadVertexArrayObject)
/*
Bind the active gl.ARRAY_BUFFER to our WebGLBuffer that describe the geometry positions
*/
gl.bindBuffer(gl.ARRAY_BUFFER, vertexBuffer)
/*
Feed our 32bit float array that describes our quad to the vertexBuffer using the
gl.ARRAY_BUFFER global handle
*/
gl.bufferData(gl.ARRAY_BUFFER, vertexArray, drawType)
/*
We need to explicitly enable our the a_position variable on the GPU
*/
gl.enableVertexAttribArray(a_positionLocationOnGPU)
/*
Finally we need to instruct the GPU how to pull the data out of our
vertexBuffer and feed it into the a_position variable in the GLSL program
*/
/*
Tell the attribute how to get data out of positionBuffer (ARRAY_BUFFER)
*/
const size = 2 // 2 components per iteration
const type = gl.FLOAT // the data is 32bit floats
const normalize = false // don't normalize the data
const stride = 0 // 0 = move forward size * sizeof(type) each iteration to get the next position
const offset = 0 // start at the beginning of the buffer
gl.vertexAttribPointer(a_positionLocationOnGPU, size, type, normalize, stride, offset)
/*
2. Defining geometry UV texCoords
V6 _______ V5 V3
| / /|
| / / |
| / / |
V4 |/ V1 /______| V2
*/
const uvsArray = new Float32Array([
0, 0, // V1
1, 0, // V2
1, 1, // V3
0, 0, // V4
1, 1, // V5
0, 1 // V6
])
/*
The rest of the code is exactly like in the vertices step above.
We need to put our data in a WebGLBuffer, look up the a_uv variable
in our GLSL program, enable it, supply data to it and instruct
WebGL how to pull it out:
*/
const uvsBuffer = gl.createBuffer()
const a_uvLocationOnGPU = gl.getAttribLocation(quadProgram, 'a_uv')
gl.bindBuffer(gl.ARRAY_BUFFER, uvsBuffer)
gl.bufferData(gl.ARRAY_BUFFER, uvsArray, drawType)
gl.enableVertexAttribArray(a_uvLocationOnGPU)
gl.vertexAttribPointer(a_uvLocationOnGPU, 2, gl.FLOAT, false, 0, 0)
/*
Stop recording and unbind the Vertex Array Object descriptior for this geometry
*/
gl.bindVertexArray(null)
/*
WebGL has a normalized viewport coordinate system which looks like this:
Device Viewport
------- 1.0 ------
| | |
| | |
-1.0 --------------- 1.0
| | |
| | |
------ -1.0 -------
However as you can see, we pass the position and size of our quad in actual pixels
To convert these pixels values to the normalized coordinate system, we will
use the simplest 2D projection matrix.
It will be represented as an array of 16 32bit floats
You can read a gentle introduction to 2D matrices here
https://webglfundamentals.org/webgl/lessons/webgl-2d-matrices.html
*/
const projectionMatrix = new Float32Array([
2 / innerWidth, 0, 0, 0,
0, -2 / innerHeight, 0, 0,
0, 0, 0, 0,
-1, 1, 0, 1,
])
/*
In order to supply uniform data to our quad GLSL program, we first need to enable the GLSL program responsible for rendering our quad
*/
gl.useProgram(quadProgram)
/*
Just like the a_position attribute variable earlier, we also need to look up
the location of uniform variables in the GLSL program in order to supply them data
*/
const u_projectionMatrixLocation = gl.getUniformLocation(quadProgram, 'u_projectionMatrix')
/*
Supply our projection matrix as a Float32Array of 16 items to the u_projection uniform
*/
gl.uniformMatrix4fv(u_projectionMatrixLocation, false, projectionMatrix)
/*
We have set up our uniform variables correctly, stop using the quad program for now
*/
gl.useProgram(null)
/*
Return our GLSL program and the Vertex Array Object descriptor of our geometry
We will need them to render our quad in our updateFrame method
*/
return {
quadProgram,
quadVertexArrayObject,
}
}
/* rest of code */
function makeGLShader (shaderType, shaderSource) {}
function makeGLProgram (vertexShaderSource, fragmentShaderSource) {}
function updateFrame (timestampMs) {}
We have successfully created a GLSL program quadProgram, which is running on the GPU, waiting to be drawn on the screen. We also have obtained a Vertex Array Object quadVertexArrayObject, which describes our geometry and can be referenced before we draw. We can now draw our quad. Let’s augment our updateFrame() method like so:
function updateFrame (timestampMs) {
/* rest of our code */
/*
Bind the Vertex Array Object descriptor of our quad we generated earlier
*/
gl.bindVertexArray(quadVertexArrayObject)
/*
Use our quad GLSL program
*/
gl.useProgram(quadProgram)
/*
Issue a render command to paint our quad triangles
*/
{
const drawPrimitive = gl.TRIANGLES
const vertexArrayOffset = 0
const numberOfVertices = 6 // 6 vertices = 2 triangles = 1 quad
gl.drawArrays(drawPrimitive, vertexArrayOffset, numberOfVertices)
}
/*
After a successful render, it is good practice to unbind our
GLSL program and Vertex Array Object so we keep WebGL state clean.
We will bind them again anyway on the next render
*/
gl.useProgram(null)
gl.bindVertexArray(null)
/* Issue next frame paint */
requestAnimationFrame(updateFrame)
}
And here is our result:
We can use the great SpectorJS Chrome extension to capture our WebGL operations on each frame. We can look at the entire command list with their associated visual states and context information. Here is what it takes to render a single frame with our updateFrame() call:
A screenshot of all the steps we implemented to render a single quad. (Click to see a larger version)
Some gotchas:
We declare the vertices positions of our triangles in a counter clockwise order. This is important.
We need to explicitly enable blending in WebGL and specify it’s blend operation. For our demo we will use gl.ONE_MINUS_SRC_ALPHA as a blend function (multiplies all colors by 1 minus the source alpha value).
In our vertex shader you can see we expect the input variable a_position to be vector with 4 components (vec4), while in Javascript we specify only 2 items per vertex. That’s because the default attribute value is 0, 0, 0, 1. It doesn’t matter that you’re only supplying x and y from your attributes. z defaults to 0 and w defaults to 1.
As you can see, WebGL is a state machine, where you have to constantly bind stuff before you are able to work on it and you always have to make sure you unbind it afterwards. Consider how in the code snippet above we supplied a Float32Array with out positions to the vertexBuffer:
const vertexArray = new Float32Array([/* ... */])
const vertexBuffer = gl.createBuffer()
/* Bind our vertexBuffer to the global binding WebGL bind point gl.ARRAY_BUFFER */
gl.bindBuffer(gl.ARRAY_BUFFER, vertexBuffer)
/* At this point, gl.ARRAY_BUFFER represents vertexBuffer */
/* Supply data to our vertexBuffer using the gl.ARRAY_BUFFER binding point */
gl.bufferData(gl.ARRAY_BUFFER, vertexArray, gl.STATIC_DRAW)
/* Do a bunch of other stuff with the active gl.ARRAY_BUFFER (vertexBuffer) here */
// ...
/* After you have done your work, unbind it */
gl.bindBuffer(gl.ARRAY_BUFFER, null)
This is totally opposite of Javascript, where this same operation would be expressed like this for example (pseudocode):
const vertexBuffer = gl.createBuffer()
vertexBuffer.addData(vertexArray)
vertexBuffer.setDrawOperation(gl.STATIC_DRAW)
// etc.
Coming from Javascript background, initially I found WebGL’s state machine way of doing things by constantly binding and unbinding really odd. One must exercise good discipline and always make sure to unbind stuff after using it, even in trivial programs like ours! Otherwise you risk things not working and hard to track bugs.
Drawing lots of quads
We have successfully rendered a single quad, but in order to make things more interesting and visually appealing, we need to draw more.
As we saw already, we can easily create new geometries with different position using our makeQuad() utility helper. We can pass them different positions and radiuses and compile each one of them into a separate GLSL program to be executed on the GPU. This will work, however:
As we saw in our update loop method updateFrame, to render our quad on each frame we must:
Use the correct GLSL program by calling gl.useProgram()
Bind the correct VAO describing our geometry by calling gl.bindVertexArray()
Issue a draw call with correct primitive type by calling gl.drawArrays()
So 3 WebGL commands in total.
What if we want to render 500 quads? Suddenly we jump to 500×3 or 1500 individual WebGL calls on each frame of our animation. If we want 1000quads we jump up to 3000 individual calls, without even counting all of the preparation WebGL bindings we have to do before our updateFrame loop starts.
Geometry Instancing is a way to reduce these calls. It works by letting you tell WebGL how many times you want the same thing drawn (the number of instances) with minor variations, such as rotation, scale, position etc. Examples include trees, grass, crowd of people, boxes in a warehouse, etc.
Just like VAOs, instancing is a 1st class citizen in WebGL2 and does not require extensions, unlike WebGL1. Let’s augment our code to support geometry instancing and render 1000 quads with random positions.
First of all, we need to decide on how many quads we want rendered and prepare the offset positions for each one as a new array of 32bit floats. Let’s do 1000 quads and positions them randomly in our viewport:
/* rest of code */
/* How many quads we want rendered */
const QUADS_COUNT = 1000
/*
Array to store our quads positions
We need to layout our array as a continuous set
of numbers, where each pair represents the X and Y
or a single 2D position.
Hence for 1000 quads we need an array of 2000 items
or 1000 pairs of X and Y
*/
const quadsPositions = new Float32Array(QUADS_COUNT * 2)
for (let i = 0; i < QUADS_COUNT; i++) {
/*
Generate a random X and Y position
*/
const randX = Math.random() * innerWidth
const randY = Math.random() * innerHeight
/*
Set the correct X and Y for each pair in our array
*/
quadsPositions[i * 2 + 0] = randX
quadsPositions[i * 2 + 1] = randY
}
/*
We also need to augment our makeQuad() method
It no longer expects a single position, rather an array of positions
*/
const {
quadProgram,
quadVertexArrayObject,
} = makeQuad(quadsPositions)
/* rest of code */
Instead of a single position, we will now pass an array of positions into our makeQuad() method. Let’s augment this method to receive our offsets array as a new variable input a_offset to our shaders which will contain the correct XY offset for a particular instance. To do this, we need to prepare our offsets as a new WebGLBuffer and instruct WebGL how to upack them, just like we did for a_position and a_uv
function makeQuad (quadsPositions, width = 70, height = 70, drawType = gl.STATIC_DRAW) {
/* rest of code */
/*
Add offset positions for our individual instances
They are declared and used in exactly the same way as
"a_position" and "a_uv" above
*/
const offsetsBuffer = gl.createBuffer()
const a_offsetLocationOnGPU = gl.getAttribLocation(quadProgram, 'a_offset')
gl.bindBuffer(gl.ARRAY_BUFFER, offsetsBuffer)
gl.bufferData(gl.ARRAY_BUFFER, quadsPositions, drawType)
gl.enableVertexAttribArray(a_offsetLocationOnGPU)
gl.vertexAttribPointer(a_offsetLocationOnGPU, 2, gl.FLOAT, false, 0, 0)
/*
HOWEVER, we must add an additional WebGL call to set this attribute to only
change per instance, instead of per vertex like a_position and a_uv above
*/
const instancesDivisor = 1
gl.vertexAttribDivisor(a_offsetLocationOnGPU, instancesDivisor)
/*
Stop recording and unbind the Vertex Array Object descriptor for this geometry
*/
gl.bindVertexArray(null)
/* rest of code */
}
We need to augment our original vertexArray responsible for passing data into our a_position GLSL variable. We no longer need to offset it to the desired position like in the first example, now the a_offset variable will take care of this in the vertex shader:
const vertexArray = new Float32Array([
/*
First set of 3 points are for our first triangle
*/
-width / 2, height / 2, // Vertex 1 (X, Y)
width / 2, height / 2, // Vertex 2 (X, Y)
width / 2, -height / 2, // Vertex 3 (X, Y)
/*
Second set of 3 points are for our second triangle
*/
-width / 2, height / 2, // Vertex 4 (X, Y)
width / 2, -height / 2, // Vertex 5 (X, Y)
-width / 2, -height / 2 // Vertex 6 (X, Y)
])
We also need to augment our vertex shader to consume and use the new a_offset input variable we pass from Javascript:
const vertexShaderSource = `#version 300 es
/* rest of GLSL code */
/*
This input vector will change once per instance
*/
in vec4 a_offset;
void main () {
/* Account a_offset in the final geometry posiiton */
vec4 newPosition = a_position + a_offset;
gl_Position = u_projectionMatrix * newPosition;
}
/* rest of GLSL code */
`
And as a final step we need to change our drawArrays call in our updateFrame to drawArraysInstanced to account for instancing. This new method expects the exact same arguments and adds instanceCount as last one:
And with all these changes, here is our updated example:
Even though we increased the amount of rendered objects by 1000x, we are still making 3 WebGL calls on each frame. That’s a pretty great performance win!
All WebGL calls needed to draw our 1000 quads in a single updateFrame()call. Note the amount of needed calls did not increase from the previous example thanks to instancing.
Post Processing with a fullscreen quad
Now that we have our 1000 quads successfully rendering to the device screen on each frame, we can turn them into metaballs. As we established, we need to scan the pixels of the picture we generated in the previous steps and determine the alpha value of each pixel. If it is below a certain threshold, we discard it, otherwise we color it.
To do this, instead of rendering our scene directly to the screen as we do right now, we need to render it to a texture. We will do our post processing on this texture and render the result to the device screen.
Post-Processing is a technique used in graphics that allows you to take a current input texture, and manipulate its pixels to produce a transformed image. This can be used to apply shiny effects like volumetric lighting, or any other filter type effect you’ve seen in applications like Photoshop or Instagram.
The basic technique for creating these effects is pretty straightforward:
A WebGLTexture is created with the same size as the canvas and attached as a color attachment to a WebGLFramebuffer. At the beginning of our updateFrame() method, the framebuffer is set as the render target, and the entire scene is rendered normally to it.
Next, a full-screen quad is rendered to the device screen using the texture generated in step 1 as an input. The shader used during the rendering of the quad is what contains the post-process effect.
/* rest of code */
const renderTexture = makeTexture()
const framebuffer = makeFramebuffer(renderTexture)
function makeTexture (textureWidth = canvas.width, textureHeight = canvas.height) {
/*
Create the texture that we will use to render to
*/
const targetTexture = gl.createTexture()
/*
Just like everything else in WebGL up until now, we need to bind it
so we can configure it. We will unbind it once we are done with it.
*/
gl.bindTexture(gl.TEXTURE_2D, targetTexture)
/*
Define texture settings
*/
const level = 0
const internalFormat = gl.RGBA
const border = 0
const format = gl.RGBA
const type = gl.UNSIGNED_BYTE
/*
Notice how data is null. That's because we don't have data for this texture just yet
We just need WebGL to allocate the texture
*/
const data = null
gl.texImage2D(gl.TEXTURE_2D, level, internalFormat, textureWidth, textureHeight, border, format, type, data)
/*
Set the filtering so we don't need mips
*/
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR)
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE)
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE)
return renderTexture
}
function makeFramebuffer (texture) {
/*
Create and bind the framebuffer
*/
const fb = gl.createFramebuffer()
gl.bindFramebuffer(gl.FRAMEBUFFER, fb)
/*
Attach the texture as the first color attachment
*/
const attachmentPoint = gl.COLOR_ATTACHMENT0
gl.framebufferTexture2D(gl.FRAMEBUFFER, attachmentPoint, gl.TEXTURE_2D, targetTexture, level)
}
We have successfully created a texture and attached it as color attachment to a framebuffer. Now we can render our scene to it. Let’s augment our updateFrame()method:
function updateFrame () {
gl.viewport(0, 0, canvas.width, canvas.height)
gl.clearColor(0, 0, 0, 0)
gl.clear(gl.COLOR_BUFFER_BIT)
/*
Bind the framebuffer we created
From now on until we unbind it, each WebGL draw command will render in it
*/
gl.bindFramebuffer(gl.FRAMEBUFFER, framebuffer)
/* Set the offscreen framebuffer background color to be transparent */
gl.clearColor(0.2, 0.2, 0.2, 1.0)
/* Clear the offscreen framebuffer pixels */
gl.clear(gl.COLOR_BUFFER_BIT)
/*
Code for rendering our instanced quads here
*/
/*
We have successfully rendered to the framebuffer at this point
In order to render to the screen next, we need to unbind it
*/
gl.bindFramebuffer(gl.FRAMEBUFFER, null)
/* Issue next frame paint */
requestAnimationFrame(updateFrame)
}
Let’s take a look at our result:
As you can see, we get an empty screen. There are no errors and the program is running just fine – keep in mind however that we are rendering to a separate framebuffer, not the default device screen framebuffer!
Our program produces black screen, since we are rendering to the offscreen framebuffer
In order to display our offscreen framebuffer back on the screen, we need to render a fullscreen quad and use the framebuffer’s texture as an input.
Creating a fullscreen quad and displaying our texture on it
Let’s create a new quad. We can reuse our makeQuad() method from the above snippets, but we need to augment it to support instancing optionally and be able to put vertex and fragment shader sources as outside argument variables. This time we need only one quad and the shaders we need for it are different.
Take a look at the updated makeQuad()signature:
/* rename our instanced quads program & VAO */
const {
quadProgram: instancedQuadsProgram,
quadVertexArrayObject: instancedQuadsVAO,
} = makeQuad({
instancedOffsets: quadsPositions,
/*
We need different set of vertex and fragment shaders
for the different quads we need to render, so pass them from outside
*/
vertexShaderSource: instancedQuadVertexShader,
fragmentShaderSource: instancedQuadFragmentShader,
/*
support optional instancing
*/
isInstanced: true,
})
Let’s use the same method to create a new fullscreen quad and render it. First our vertex and fragment shader:
const fullscreenQuadVertexShader = `#version 300 es
in vec4 a_position;
in vec2 a_uv;
uniform mat4 u_projectionMatrix;
out vec2 v_uv;
void main () {
gl_Position = u_projectionMatrix * a_position;
v_uv = a_uv;
}
`
const fullscreenQuadFragmentShader = `#version 300 es
precision highp float;
/*
Pass our texture we render to as an uniform
*/
uniform sampler2D u_texture;
in vec2 v_uv;
out vec4 outputColor;
void main () {
/*
Use our interpolated UVs we assigned in Javascript to lookup
texture color value at each pixel
*/
vec4 inputColor = texture(u_texture, v_uv);
/*
0.5 is our alpha threshold we use to decide if
pixel should be discarded or painted
*/
float cutoffThreshold = 0.5;
/*
"cutoff" will be 0 if pixel is below 0.5 or 1 if above
step() docs - https://thebookofshaders.com/glossary/?search=step
*/
float cutoff = step(cutoffThreshold, inputColor.a);
/*
Let's use mix() GLSL method instead of if statement
if cutoff is 0, we will discard the pixel by using empty color with no alpha
otherwise, let's use black with alpha of 1
mix() docs - https://thebookofshaders.com/glossary/?search=mix
*/
vec4 emptyColor = vec4(0.0);
/* Render base metaballs shapes */
vec4 borderColor = vec4(1.0, 0.0, 0.0, 1.0);
outputColor = mix(
emptyColor,
borderColor,
cutoff
);
/*
Increase the treshold and calculate new cutoff, so we can render smaller shapes again, this time in different color and with smaller radius
*/
cutoffThreshold += 0.05;
cutoff = step(cutoffThreshold, inputColor.a);
vec4 fillColor = vec4(1.0, 1.0, 0.0, 1.0);
/*
Add new smaller metaballs color on top of the old one
*/
outputColor = mix(
outputColor,
fillColor,
cutoff
);
}
`
Let’s use them to create and link a valid GLSL program, just like when we rendered our instances:
const {
quadProgram: fullscreenQuadProgram,
quadVertexArrayObject: fullscreenQuadVAO,
} = makeQuad({
vertexShaderSource: fullscreenQuadVertexShader,
fragmentShaderSource: fullscreenQuadFragmentShader,
isInstanced: false,
width: innerWidth,
height: innerHeight
})
/*
Unlike our instances GLSL program, here we need to pass an extra uniform - a "u_texture"!
Tell the shader to use texture unit 0 for u_texture
*/
gl.useProgram(fullscreenQuadProgram)
const u_textureLocation = gl.getUniformLocation(fullscreenQuadProgram, 'u_texture')
gl.uniform1i(u_textureLocation, 0)
gl.useProgram(null)
Finally we can render the fullscreen quad with the result texture as an uniform u_texture. Let’s change our updateFrame() method:
function updateFrame () {
gl.bindFramebuffer(gl.FRAMEBUFFER, framebuffer)
/* render instanced quads here */
gl.bindFramebuffer(gl.FRAMEBUFFER, null)
/*
Render our fullscreen quad
*/
gl.bindVertexArray(fullscreenQuadVAO)
gl.useProgram(fullscreenQuadProgram)
/*
Bind the texture we render to as active TEXTURE_2D
*/
gl.bindTexture(gl.TEXTURE_2D, renderTexture)
{
const drawPrimitive = gl.TRIANGLES
const vertexArrayOffset = 0
const numberOfVertices = 6 // 6 vertices = 2 triangles = 1 quad
gl.drawArrays(drawPrimitive, vertexArrayOffset, numberOfVertices)
}
/*
Just like everything else, unbind our texture once we are done rendering
*/
gl.bindTexture(gl.TEXTURE_2D, null)
gl.useProgram(null)
gl.bindVertexArray(null)
requestAnimationFrame(updateFrame)
}
And here is our final result (I also added a simple animation to make the effect more apparent):
And here is the breakdown of one updateFrame() call:
You can clearly see how we render our 1000 instanced quads in separate framebuffer in steps 1 to 3. We then draw and manipulate the resulting texture to a fullscreen quad that we render in steps 4 to 7.
Aliasing issues
On my 2016 MacBook Pro with retina display I can clearly see aliasing issues with our current example. If we are to add bigger radiuses and blow our animation to fullscreen the problem will become only more noticeable.
The issue comes from the fact we are rendering to a 8bit gl.UNSIGNED_BYTE texture. If we want to increase the detail, we need to switch to floating point textures (32 bit float gl.RGBA32F or 16 bit float gl.RGBA16F). The catch is that these textures are not supported on all hardware and are not part of WebGL2 core. They are available through optional extensions, that we need to check if exist.
The extensions we are interested in to render to 32bit floating point textures are
EXT_color_buffer_float
OES_texture_float_linear
If these extensions are present on the user device, we can use internalFormat = gl.RGBA32F and textureType = gl.FLOAT when creating our render textures. If they are not present, we can optionally fallback and render to 16bit floating textures. The extensions we need in that case are:
EXT_color_buffer_half_float
OES_texture_half_float_linear
If these extensions are present, we can use internalFormat = gl.RGBA16F and textureType = gl.HALF_FLOAT for our render texture. If not, we will fallback to what we have used up until now – internalFormat = gl.RGBA and textureType = gl.UNSIGNED_BYTE.
Here is our updated makeTexture() method:
function makeTexture (textureWidth = canvas.width, textureHeight = canvas.height) {
/*
Initialize internal format & texture type to default values
*/
let internalFormat = gl.RGBA
let type = gl.UNSIGNED_BYTE
/*
Check if optional extensions are present on device
*/
const rgba32fSupported = gl.getExtension('EXT_color_buffer_float') && gl.getExtension('OES_texture_float_linear')
if (rgba32fSupported) {
internalFormat = gl.RGBA32F
type = gl.FLOAT
} else {
/*
Check if optional fallback extensions are present on device
*/
const rgba16fSupported = gl.getExtension('EXT_color_buffer_half_float') && gl.getExtension('OES_texture_half_float_linear')
if (rgba16fSupported) {
internalFormat = gl.RGBA16F
type = gl.HALF_FLOAT
}
}
/* rest of code */
/*
Pass in correct internalFormat and textureType to texImage2D call
*/
gl.texImage2D(gl.TEXTURE_2D, level, internalFormat, textureWidth, textureHeight, border, format, type, data)
/* rest of code */
}
And here is our updated result:
Conclusion
I hope I managed to showcase the core principles behind WebGL2 with this demo. As you can see, the API itself is low-level and requires quite a bit of typing, yet at the same time is really powerful and let’s you draw complex scenes with fine-grained control over the rendering.
Writing production ready WebGL requires even more typing, checking for optional features / extensions and handling missing extensions and fallbacks, so I would advise you to use a framework. At the same time, I believe it is important to understand the key concepts behind the API so you can successfully use higher level libraries like threejs and dig into their internals if needed.
I am a big fan of twgl, which hides away much of the verbosity of the API, while still being really low level with a small footprint. This demo’s code can easily be reduced by more then half by using it.
I encourage you to experiment around with the code after reading this article, plug in different values, change the order of things, add more draw commands and what not. I hope you walk away with a high level understanding of core WebGL2 API and how it all ties together, so you can learn more on your own.
There’s a slew of artists and creative coders on social media who regularly post satisfying, hypnotic looping animations. One example is Dave Whyte, also known as @beesandbombs on Twitter. In this tutorial I’ll explain how to recreate one of his more popular recent animations, which I’ve dubbed “Breathing Dots”. Here’s the original animation:
Dave says he uses Processing for his animations, but I’ll be using react-three-fiber (R3F) which is a React renderer for Three.js. Why am I using a 3D library for a 2D animation? Well, R3F provides a powerful declarative syntax for WebGL graphics and grants you access to useful Three.js features such as post-processing effects. It lets you do a lot with few lines of code, all while being highly modular and re-usable. You can use whatever tool you like, but I find the combined ecosystems of React and Three.js make R3F a robust tool for general purpose graphics.
I use an adapted Codesandbox template running Create React App to bootstrap my R3F projects; You can fork it by clicking the button above to get a project running in a few seconds. I will assume some familiarity with React, Three.js and R3F for the rest of the tutorial. If you’re totally new, you might want to start here.
Step 1: Observations
First things first, we need to take a close look at what’s going on in the source material. When I look at the GIF, I see a field of little white dots. They’re spread out evenly, but the pattern looks more random than a grid. The dots are moving in a rhythmic pulse, getting pulled towards the center and then flung outwards in a gentle shockwave. The shockwave has the shape of an octagon. The dots aren’t in constant motion, rather they seem to pause at each end of the cycle. The dots in motion look really smooth, almost like they’re melting. We need to zoom in to really understand what’s going on here. Here’s a close up of the corners during the contraction phase:
Top left
Bottom right
Interesting! The moving dots are split into red, green, and blue parts. The red part points in the direction of motion, while the blue part points away from the motion. The faster the dot is moving, the farther these three parts are spread out. As the colored parts overlap, they combine into a solid white color. Now that we understand what exactly we want to produce, lets start coding.
Step 2: Making Some Dots
If you’re using the Codesandbox template I provided, you can strip down the main App.js to just an empty scene with a black background:
import React from 'react'
import { Canvas } from 'react-three-fiber'
export default function App() {
return (
<Canvas>
<color attach="background" args={['black']} />
</Canvas>
)
}
Our First Dot
Let’s create a component for our dots, starting with just a single white circle mesh composed of a CircleBufferGeometry and MeshBasicMaterial
Add the <Dots /> component inside the canvas, and you should see a white octagon appear onscreen. Our first dot! Since it’ll be tiny, it doesn’t matter that it’s not very round.
But wait a second… Using a color picker, you’ll notice that it’s not pure white! This is because R3F sets up color management by default which is great if you’re working with glTF models, but not if you need raw colors. We can disable the default behavior by setting colorManagement={false} on our canvas.
More Dots
We need approximately 10,000 dots to fully fill the screen throughout the animation. A naive approach at creating a field of dots would be to simply render our dot mesh a few thousand times. However, you’ll quickly notice that this destroys performance. Rendering 10,000 of these chunky dots brings my gaming rig down to a measly 5 FPS. The problem is that each dot mesh incurs its own draw call, which means the CPU needs to send 10,000 (largely redundant) instructions to the GPU every frame.
The solution is to use instanced rendering, which means the CPU can tell the GPU about the dot shape, material, and the locations of all 10,000 instances in a single draw call. Three.js offers a helpful InstancedMesh class to facilitate instanced rendering of a mesh. According to the docs it accepts a geometry, material, and integer count as constructor arguments. Let’s convert our regular old mesh into an <instancedMesh> , starting with just one instance. We can leave the geometry and material slots as null since the child elements will fill them, so we only need to specify the count.
Hey, where did it go? The dot disappeared because of how InstancedMesh is initialized. Internally, the .instanceMatrix stores the transformation matrix of each instance, but it’s initialized with all zeros which squashes our mesh into the abyss. Instead, we should start with an identity matrix to get a neutral transformation. Let’s get a reference to our InstancedMesh and apply the identity matrix to the first instance inside of useLayoutEffect so that it’s properly positioned before anything is painted to the screen.
function Dots() {
const ref = useRef()
useLayoutEffect(() => {
// THREE.Matrix4 defaults to an identity matrix
const transform = new THREE.Matrix4()
// Apply the transform to the instance at index 0
ref.current.setMatrixAt(0, transform)
}, [])
return (
<instancedMesh ref={ref} args={[null, null, 1]}>
<circleBufferGeometry />
<meshBasicMaterial />
</instancedMesh>
)
}
Great, now we have our dot back. Time to crank it up to 10,000. We’ll increase the instance count and set the transform of each instance along a centered 100 x 100 grid.
for (let i = 0; i < 10000; ++i) {
const x = (i % 100) - 50
const y = Math.floor(i / 100) - 50
transform.setPosition(x, y, 0)
ref.current.setMatrixAt(i, transform)
}
We should also decrease the circle radius to 0.15 to better fit the grid proportions. We don’t want any perspective distortion on our grid, so we should set the orthographic prop on the canvas. Lastly, we’ll lower the default camera’s zoom to 20 to fit more dots on screen.
The result should look like this:
Although you can’t notice yet, it’s now running at a silky smooth 60 FPS
Adding Some Noise
There’s a variety of ways to distribute points on a surface beyond a simple grid. “Poisson disc sampling” and “centroidal Voronoi tessellation” are some mathematical approaches that generate slightly more natural distributions. That’s a little too involved for this demo, so let’s just approximate a natural distribution by turning our square grid into hexagons and adding in small random offsets to each point. The positioning logic now looks like this:
// Place in a grid
let x = (i % 100) - 50
let y = Math.floor(i / 100) - 50
// Offset every other column (hexagonal pattern)
y += (i % 2) * 0.5
// Add some noise
x += Math.random() * 0.3
y += Math.random() * 0.3
Step 3: Creating Motion
Sine waves are the heart of cyclical motion. By feeding the clock time into a sine function, we get a value that oscillates between -1 and 1. To get the effect of expansion and contraction, we want to oscillate each point’s distance from the center. Another way of thinking about this is that we want to dynamically scale each point’s intial position vector. Since we should avoid unnecessary computations in the render loop, let’s cache our initial position vectors in useMemo for re-use. We’re also going to need that Matrix4 in the loop, so let’s cache that as well. Finally, we don’t want to overwrite our initial dot positions, so let’s cache a spare Vector3 for use during calculations.
For simplicity let’s scrap the useLayoutEffect call and configure all the matrix updates in a useFrame loop. Remember that in R3F, the useFrame callback receives the same arguments as useThree including the Three.js clock, so we can access a dynamic time through clock.elapsedTime. We’ll add some simple motion by copying each instance position into our scratch vector, scaling it by some factor of the sine wave, and then copying that to the matrix. As mentioned in the docs, we need to set .needsUpdate to true on the instanced mesh’s .instanceMatrix in the loop so that Three.js knows to keep updating the positions.
The raw sine wave follows a perfectly round, circular motion. However, as we observed earlier:
The dots aren’t in constant motion, rather they seem to pause at each end of the cycle.
This calls for a different, more boxy looking wave with longer plateaus and shorter transitions. A search through the digital signal processing StackExchange produces this post with the equation for a rounded square wave. I’ve visualized the equation here and animated the delta parameter, watch how it goes from smooth to boxy:
The equation translates to this Javascript function:
const roundedSquareWave = (t, delta, a, f) => {
return ((2 * a) / Math.PI) * Math.atan(Math.sin(2 * Math.PI * t * f) / delta)
}
Swapping out our Math.sin call for the new wave function with a delta of 0.1 makes the motion more snappy, with time to rest in between:
Ripples
How do we use this wave to make the dots move at different speeds and create ripples? If we change the input to the wave based on the dot’s distance from the center, then each ring of dots will be at a different phase causing the surface to stretch and squeeze like an actual wave. We’ll use the initial distances on every frame, so let’s cache and return the array of distances in our useMemo callback:
Then, in the useFrame callback we subtract a factor of the distance from the t (time) variable that gets plugged into the wave. That looks like this:
That already looks pretty cool!
The Octagon
Our ripple is perfectly circular, how can we make it look more octagonal like the original? One way to approximate this effect is by combining a sine or cosine wave with our distance function at an appropriate frequency (8 times per revolution). Watch how changing the strength of this wave changes the shape of the region:
A strength of 0.5 is a pretty good balance between looking like an octagon and not looking too wavy. That change can happen in our initial distance calculations:
It’ll take some additional tweaks to really see the effect of this. There’s a few places that we can focus our adjustments on:
Influence of point distance on wave phase
Influence of point distance on wave roundness
Frequency of the wave
Amplitude of the wave
It’s a bit of educated trial and error to make it match the original GIF, but after fiddling with the wave parameters and multipliers eventually you can get something like this:
When previewing in full screen, the octagonal shape is now pretty clear.
Step 4: Post-processing
We have something that mimics the overall motion of the GIF, but the dots in motion don’t have the same color shifting effect that we observed earlier. As a reminder:
The moving dots are split into red, green, and blue parts. The red part points in the direction of motion, while the blue part points away from the motion. The faster the dot is moving, the farther these three parts are spread out. As the colored parts overlap, they combine into a solid white color.
We can achieve this effect using the post-processing EffectComposer built into Three.js, which we can conveniently tack onto the scene without any changes to the code we’ve already written. If you’re new to post-processing like me, I highly recommend reading this intro guide from threejsfundamentals. In short, the composer lets you toss image data back and forth between two “render targets” (glorified image textures), applying shaders and other operations in between. Each step of the pipeline is called a “pass”. Typically the first pass performs the initial scene render, then there are some passes to add effects, and by default the final pass writes the resulting image to the screen.
An example: motion blur
Here’s a JSFiddle from Maxime R that demonstrates a naive motion blur effect with the EffectComposer. This effect makes use of a third render target in order to preserve a blend of previous frames. I’ve drawn out a diagram to track how image data moves through the pipeline (read from the top down):
First, the scene is rendered as usual and written to rt1 with a RenderPass. Most passes will automatically switch the read and write buffers (render targets), so our next pass will read what we just rendered in rt1 and write to rt2. In this case we use a ShaderPass configured with a BlendShader to blend the contents of rt1 with whatever is stored in our third render target (empty at first, but it eventually accumulates a blend of previous frames). This blend is written to rt2 and another swap automatically occurs. Next, we use a SavePass to save the blend we just created in rt2 back to our third render target. The SavePass is a little unique in that it doesn’t swap the read and write buffers, which makes sense since it doesn’t actually change the image data. Finally, that same blend in rt2 (which is still the read buffer) gets read into another ShaderPass set to a CopyShader, which simply copies its input into the output. Since it’s the last pass on the stack, it automatically gets renderToScreen=true which means that its output is what you’ll see on screen.
Working with post-processing requires some mental gymnastics, but hopefully this makes some sense of how different components like ShaderPass, SavePass, and CopyPass work together to apply effects and preserve data between frames.
RGB Delay Effect
A simple RGB color shifting effect involves turning our single white dot into three colored dots that get farther apart the faster they move. Rather than trying to compute velocities for all the dots and passing them to the post-processing stack, we can cheat by overlaying previous frames:
This turns out to be a very similar problem as the motion blur, since it requires us to use additional render targets to store data from previous frames. We actually need two extra render targets this time, one to store the image from frame n-1 and another for frame n-2. I’ll call these render targets delay1 and delay2.
Here’s a diagram of the RGB delay effect:
A circle containing a value X represents the individual frame for delay X.
The trick is to manually disable needsSwap on the ShaderPass that blends the colors together, so that the proceeding SavePass re-reads the buffer that holds the current frame rather than the colored composite. Similarly, by manually enabling needsSwap on the SavePass we ensure that we read from the colored composite on the final ShaderPass for the end result. The other tricky part is that since we’re placing the current frame’s contents in the delay2 buffer (as to not lose the contents of delay1 for the next frame), we need to swap these buffers each frame. It’s easiest to do this outside of the EffectComposer by swapping the references to these render targets on the ShaderPass and SavePass within the render loop.
Implementation
This is all very abstract, so let’s see what this means in practice. In a new file (Effects.js), start by importing the necessary passes and shaders, then extending the classes so that R3F can access them declaratively.
import { useThree, useFrame, extend } from 'react-three-fiber'
import { EffectComposer } from 'three/examples/jsm/postprocessing/EffectComposer'
import { ShaderPass } from 'three/examples/jsm/postprocessing/ShaderPass'
import { SavePass } from 'three/examples/jsm/postprocessing/SavePass'
import { CopyShader } from 'three/examples/jsm/shaders/CopyShader'
import { RenderPass } from 'three/examples/jsm/postprocessing/RenderPass'
extend({ EffectComposer, ShaderPass, SavePass, RenderPass })
We’ll put our effects inside a new component. Here is what a basic effect looks like in R3F:
All that this does is render the scene to the canvas. Let’s start adding in the pieces from our diagram. We’ll need a shader that takes in 3 textures and respectively blends the red, green, and blue channels of them. The vertexShader of a post-processing shader always looks the same, so we only really need to focus on the fragmentShader. Here’s what the complete shader looks like:
With the shader ready to roll, we’ll then memo-ize our helper render targets and set up some additional refs to hold constants and references to our other passes.
By stating args={[triColorMix, 'tDiffuse1']} on the blend pass, we indicate that the composer’s read buffer should be passed as the tDiffuse1 uniform in our custom shader. The behavior of these passes is unfortunately not documented, so you sometimes need to poke through the source files to figure this stuff out.
Finally, we’ll need to modify the render loop to swap between our spare render targets and plug them in as the remaining 2 uniforms:
All the pieces for our RGB delay effect are in place. Here’s a demo of the end result on a simpler scene with one white dot moving back and forth:
Putting it all together
As you’ll notice in the previous sandbox, we can make the effect take hold by simply plopping the <Effects /> component inside the canvas. After doing this, we can make it look even better by adding an anti-aliasing pass to the effect composer.
While outside the scope of this tutorial, I’ve added an interactive demo variant which responds to mouse clicks and cursor position. This variant uses react-spring v9 to smoothly reposition the focus point of the dots. Check it out in the “Demo 2” page of the demo linked at the top of this page, and play around with the source code to see if you can add other forms of interactivity.
Step 5: Sharing Your Work
I highly recommend publicly sharing the things you create. It’s a great way to track your progress, share your learning with others, and get feedback. I wouldn’t be writing this tutorial if I hadn’t shared my work! For perfect loops you can use the use-capture hook to automate your recording. If you’re sharing to Twitter, consider converting to a GIF to avoid compression artifacts. Here’s a thread from @arc4g explaining how they create smooth 50 FPS GIFs for Twitter.
I hope you learned something about Three.js or react-three-fiber from this tutorial. Many of the animations I see online follow a similar formula of repeated shapes moving in some mathematical rhythm, so the principles here extend beyond just rippling dots. If this inspired you to create something cool, tag me in it so I can see!
So I recently purchased 2 5k2k monitors to improve workflow for game development. I noticed once I run both displays the GPU gets hot and eventually the display goes black and the GPU fans go crazy. I also noticed that before that happens the GPU fans do not turn on to cool it down. Could this be an issue that can be fixed through an option the GPU might have? Or is the GPU not enough to keep both displays on?
The GPU starts getting hot and as soon as I display both screens, even though I dont run heavy GPU applications and sometimes it does before I do run them or minutes after I run something like Photoshop.
GPU is an RTX 1080 Ti.
If this the GPU isnt powerful enough, do you think an RTX 3080 might be enough?
Or if I use 1070ti to focus only on the displays and dedicate the 3080 GPU to the heavy applications.
Sorry for my ignorance in advance, I havent encounter this issue before so Ive never really looked into any of this.
The other day, I was looking to read an Arrow buffer on GPU using Python, but as far as I could tell, none of the provided pyarrow packages on conda or pip are built with CUDA support. Like many of the packages in the compiled-C-wrapped-by-Python ecosystem, Apache Arrow is thoroughly documented, but the number of permutations of how you could choose to build pyarrow with CUDA support quickly becomes overwhelming.
In this post, I’ll show you how to build pyarrow with CUDA support on Ubuntu using Docker and virtualenv. These directions are approximately the same as the official Apache Arrow docs, but here, I explain them step-by-step and show only the single build toolchain I used.
Animations have come a long way, continuously providing developers with better tools. CSS Animations, in particular, have defined the ground floor to solve the majority of uses cases. However, there are some animations that require a little bit more work.
You probably know that animations should run on the composite layer. (I won’t extend myself here, but if you want to know more, check this article.) That means animating transform or opacity properties that don’t trigger layout or paint layers. Animating properties like height and width is a big no-no, as they trigger those layers, which force the browser to recalculate styles.
On top of that, even when animating transform properties, if you want to truly hit 60 FPS animations, you probably should get a little help from JavaScript, using the FLIP technique for extra smoother animations!
However, the problem of using transform for expandable animations is that the scale function isn’t exactly the same as animating width/height properties. It creates a skewed effect on the content, as all elements get stretched (when scaling up) or squeezed (when scaling down).
So, because of that, my go-to solution has been (and probably still is, for reasons I will detail later), technique #3 from Brandon Smith’s article. This still has a transition on height, but uses Javascript to calculate the content size, and force a transition using requestAnimationFrame. At OutSystems, we actually used this to build the animation for the OutSystems UI Accordion Pattern.
Generating keyframes with JavaScript
Recently, I stumbled on another great article from Paul Lewis, that details a new solution for expanding and collapsing animations, which motivated me to write this article and spread this technique around.
Using his words, the main idea consists of generating dynamic keyframes, stepping…
[…] from 0 to 100 and calculate what scale values would be needed for the element and its contents. These can then be boiled down to a string, which can be injected into the page as a style element.
To achieve this, there are three main steps.
Step 1: Calculate the start and end states
We need to calculate the correct scale value for both states. That means we use getBoundingClientRect() on the element that will serve as a proxy for the start state, and divide it with the value from the end state. It should be something like this:
Now, we need to run a for loop, using the number of frames needed as the length. (It shouldn’t really be less than 60 to ensure a smooth animation.) Then, in each iteration, we calculate the correct easing value, using an ease function:
function ease (v, pow=4) {
return 1 - Math.pow(1 - v, pow);
}
let easedStep = ease(i / frame);
With that value, we’ll get the scale of the element on the current step, using the following math:
const xScale = x + (1 - x) * easedStep;
const yScale = y + (1 - y) * easedStep;
You can check the example of a Menu from Paul Lewis article, on Codepen (courtesy of Chris):
Building an expandable section
After grasping these baseline concepts, I wanted to check if I could apply this technique to a different use case, like a expandable section.
We only need to animate the height in this case, specifically on the function to calculate scales. We’re getting the Y value from the section title, to serve as the collapsed state, and the whole section to represent the expanded state:
_calculateScales () {
var collapsed = this._sectionItemTitle.getBoundingClientRect();
var expanded = this._section.getBoundingClientRect();
// create css variable with collapsed height, to apply on the wrapper
this._sectionWrapper.style.setProperty('--title-height', collapsed.height + 'px');
this._collapsed = {
y: collapsed.height / expanded.height
}
}
Since we want the expanded section to have absolute positioning (in order to avoid it taking space when in a collapsed state), we are setting the CSS variable for it with the collapsed height, applied on the wrapper. That will be the only element with relative positioning.
Next comes the function to create the keyframes: _createEaseAnimations(). This doesn’t differ much from what was explained above. For this use case, we actually need to create four animations:
The animation to expand the wrapper
The counter-expand animation on the content
The animation to collapse the wrapper
The counter-collapse animation on the content
We follow the same approach as before, running a for loop with a length of 60 (to get a smooth 60 FPS animation), and create a keyframe percentage, based on the eased step. Then, we push it to the final animations strings:
We start by creating a style tag to hold the finished animations. As this is built as a constructor, to be able to easily add multiple patterns, we want to have all these generated animations on the same stylesheet. So, first, we validate if the element exists. If not, we create it and add a meaningful class name. Otherwise, you would end up with a stylesheet for each section expandable, which is not ideal.
var sectionEase = document.querySelector('.section-animations');
if (!sectionEase) {
sectionEase = document.createElement('style');
sectionEase.classList.add('section-animations');
}
Speaking of that, you may already be wondering, “Hmm, if we have multiple expandable sections, wouldn’t they still be using the same-named animation, with possibly wrong values for their content?”
You’re absolutely right! So, to prevent that, we are also generating dynamic animation names. Cool, right?
We make use of the index passed to the constructor from the for loop when making the querySelectorAll('.section') to add a unique element to the name:
var sectionExpandAnimationName = "sectionExpandAnimation" + index;
var sectionExpandContentsAnimationName = "sectionExpandContentsAnimation" + index;
Then we use this name to set a CSS variable on the current expandable section. As this variable is only in this scope, we just need to set the animation to the new variable in the CSS, and each pattern will get its respective animation-name value.
The rest of the script is related to adding event listeners, functions to toggle the collapse/expand status and some accessibility improvements.
About the HTML and CSS: it needs a little bit of extra work to make the expandable functionality work. We need an extra wrapper to be the relative element that doesn’t animate. The expandable children have an absolute position so that they don’t occupy space when collapsed.
Remember, since we need to make counter animations, we make it scale full size in order to avoid a skew effect on the content.
I would like to highlight the importance of the animation-timing-functionproperty. It should be set to linear or step-end to avoid easing between each keyframe.
The will-change property — as you probably know — will enable GPU acceleration for the transform animation for an even smoother experience. And using the contains property, with a value of contents, will help the browser treat the element independently from the rest of the DOM tree, limiting the area before it recalculates the layout, style, paint and size properties.
We use visibility and opacity to hide the content, and stop screen readers to access it, when collapsed.
And finally, we have our section expandable! Here’s the complete code and demo for you to check:
Performance check
Anytime we work with animations, performance ought to be in the back of our mind. So, let’s use developer tools to check if all this work was worthy, performance-wise. Using the Performance tab (I’m using Chrome DevTools), we can analyze the FPS and the CPU usage, during the animations.
And the results are great!
The higher the green bar, the higher the frames. And there’s no junk either, which would be signed by red sections.
Using the FPS meter tool to check the values at greater detail, we can see that it constantly hits the 60 FPS mark, even with abusive usage.
Final considerations
So, what’s the verdict? Does this replace all other methods? Is this the “Holy Grail” solution?
In my opinion, no.
But… that’s OK, really! It’s another solution on the list. And, as is true with any other method, it should be analyzed if it’s the best approach for the use-case.
This technique definitely has its merits. As Paul Lewis says, this does take a lot of work to prepare. But, on the flip side, we only need to do it once, when the page loads. During interactions, we are merely toggling classes (and attributes in some cases, for accessibility).
However, this brings some limitations to the UI of the elements. As you could see for the expandable section element, the counter-scale makes it much more reliable for absolute and off-canvas elements, like floating-actions or menus. It’s also difficult to styled borders because it’s using overflow: hidden.
Nevertheless, I think there is tons of potential with this approach. Let me know what you think!
Texture projection is a way of mapping a texture onto a 3D object and making it look like it was projected from a single point. Think of it as the batman symbol projected onto the clouds, with the clouds being our object and the batman symbol being our texture. It’s used both in games and visual effects, and more parts of the creative world. Here is a talk by Yi-Wen Lin which contains some other cool examples.
Looks neat, huh? Let’s achieve this in Three.js!
Minimum viable example
First, let’s set up our scene. The setup code is the same in every Three.js project, so I won’t go into details here. You can go to the official guide and get familiar with it if you haven’t done that before. I personally use some utils from threejs-modern-app, so I don’t need to worry about the boilerplate code.
So, first we need a camera from which to project the texture from.
const camera = new THREE.PerspectiveCamera(45, 1, 0.01, 3)
camera.position.set(-1, 1.2, 1.5)
camera.lookAt(0, 0, 0)
Then, we need our object on which we will project the texture. To do projection mapping, we will write some custom shader code, so let’s create a new ShaderMaterial:
// create the mesh with the projected material
const geometry = new THREE.BoxGeometry(1, 1, 1)
const material = new THREE.ShaderMaterial({
uniforms: {
texture: { value: assets.get(textureKey) },
},
vertexShader: '',
fragShader: '',
})
const box = new THREE.Mesh(geometry, material)
However, since we may need to use our projected material multiple times, we can put it in a component by itself and use it like this:
class ProjectedMaterial extends THREE.ShaderMaterial {
constructor({ camera, texture }) {
// ...
}
}
const material = new ProjectedMaterial({
camera,
texture: assets.get(textureKey),
})
Let’s write some shaders!
In the shader code we’ll basically sample the texture as if it was projected from the camera. Unfortunately, this involves some matrix multiplication. But don’t be scared! I’ll explain it in a simple, easy to understand way. If you want to dive deeper into the subject, here is a really good article about it.
In the vertex shader, we have to treat each vertex as if it’s being viewed from the projection camera, so we just use the projection camera’s projectionMatrix and viewMatrix instead of the ones from the scene camera. We pass this transformed position into the fragment shader using a varying variable.
In the fragment shader, we have to transform the position from world space into clip space. We do this by dividing the vector by its .w component. The GLSL built-in function texture2DProj (or the newer textureProj) does this internally also.
In the same line, we also transform from clip space range, which is [-1, 1], to the uv lookup range, which is [0, 1]. We use this variable to later sample from the texture.
Notice that we wrote some code to project the texture only on the faces of the cube that are facing the camera. By default, every face gets the texture projected onto, so we check if the face is actually facing the camera by looking at the dot product of the normal and the camera direction. This technique is really common in lighting, here is an article if you want to read more about this topic.
// this makes sure we don't render the texture also on the back of the object
vec3 projectorDirection = normalize(projPosition - vWorldPosition.xyz);
float dotProduct = dot(vNormal, projectorDirection);
if (dotProduct < 0.0) {
outColor = vec4(color, 1.0);
}
First part down, we now want to make it look like the texture is actually sticked on the object.
We do this simply by saving the object’s position at the beginning, and then we use it instead of the updated object position to do the calculations of the projection, so that if the object moves afterwards, the projection doesn’t change.
We can store the object initial model matrix in the uniform savedModelMatrix, and so our calculations become:
We can expose a project() function which sets the savedModelMatrix with the object’s current modelMatrix.
export function project(mesh) {
// make sure the matrix is updated
mesh.updateMatrixWorld()
// we save the object model matrix so it's projected relative
// to that position, like a snapshot
mesh.material.uniforms.savedModelMatrix.value.copy(mesh.matrixWorld)
}
And here is our final result:
That’s it! Now the cube looks like it has a texture slapped onto it! This can scale up and work with any kind of 3D model, so let’s make a more interesting example.
More appealing example
For the previous example we created a new camera from which to project, but what if we would use the same camera that renders the scene to project? This way we would see exactly the 2D image! This is because the point of projection coincides with the view point.
Also, let’s try projecting onto multiple objects:
That looks interesting! However, as you can see from the example, the image looks kinda warped, this is because the texture is stretched to fill the camera frustum. But what if we would like to retain the image’s original proportion and dimensions?
Also we didn’t take lighting into consideration at all. There needs to be some code in the fragment shader which tells how the surface is lighted regarding the lights we put in the scene.
Furthermore, what if we would like to project onto a much bigger number of objects? The performance would quickly drop. That’s where GPU instancing comes to aid! Instancing moves the the heavy work onto the GPU, and Three.js recently implemented an easy-to-use API for it. The only requirement is that all of the instanced objects must have the same geometry and material. Luckily, this is our case! All of the objects have the same geometry and material, the only difference is the savedModelMatrix, since each object had a different position when it was projected on. But we can pass that as a uniform to every instance like in this Three.js example.
Things starts to get complicated, but don’t worry! I already coded this stuff and put it in a library, so it’s easier to use and you don’t have to rewrite the same things each time! It’s called three-projected-material, go check it out if you’re interested in how I overcame the remaining challenges.
We’re gonna use the library from this point on.
Useful example
Now that we can project onto and animate a lot of objects, let’s try making something actually useful out of it.
For example, let’s try integrating this into a slideshow, where the images are projected onto a ton of 3D objects, and then the objects are animated in an interesting way.
For the first example, the inspiration comes from Refik Anadol. He does some pretty rad stuff. However, we can’t do full-fledged simulations with velocities and forces like him, we need to have control over the object’s movement; we need it to arrive in the right place at the right time.
We achieve this by putting the object on some trails: we define a path the object has to follow, and we animate the object on that path. Here is a Stack Overflow answer that explains how to do it.
Tip: you can access this mode by putting ?debug at the end of the URL of each demo.
To do the projection, we
Move the elements to the middle point
Do the texture projection calling project()
Put the elements back to the start
This happens synchronously, so the user won’t see anything.
Now we have the freedom to model these paths any way we want!
But first, we have to make sure that at the middle point, the elements will cover the image’s area properly. To do this I used the poisson-disk sampling algorithm, which distributes the points more evenly on a surface rather than random positioning them.
this.points = poissonSampling([this.width, this.height], 7.73, 9.66) // innerradius and outerradius
// here is what this.points looks like,
// the z component is 0 for every one of them
// [
// [
// 2.4135735314978937, --> x
// 0.18438944023363374 --> y
// ],
// [
// 2.4783704056100464,
// 0.24572635574719284
// ],
// ...
Now let’s take a look at how the paths are generated in the first demo. In this demo, there is a heavy of use of perlin noise (or rather its open source counterpart, open simplex noise). Notice also the mapRange() function (map() in processing) which basically maps a number from one interval to another. Another library that does this is d3-scale with its d3.scaleLinear(). Some easing functions are also used.
const segments = 51 // must be odds so we have the middle frame
const halfIndex = (segments - 1) / 2
for (let i = 0; i < segments; i++) {
const offsetX = mapRange(i, 0, segments - 1, startX, endX)
const noiseAmount = mapRangeTriple(i, 0, halfIndex, segments - 1, 1, 0, 1)
const frequency = 0.25
const noiseAmplitude = 0.6
const noiseY = noise(offsetX * frequency) * noiseAmplitude * eases.quartOut(noiseAmount)
const scaleY = mapRange(eases.quartIn(1 - noiseAmount), 0, 1, 0.2, 1)
const offsetZ = mapRangeTriple(i, 0, halfIndex, segments - 1, startZ, 0, endZ)
// offsetX goes from left to right
// scaleY shrinks the y before and after the center
// noiseY is some perlin noise on the y axis
// offsetZ makes them enter from behind a little bit
points.push(new THREE.Vector3(x + offsetX, y * scaleY + noiseY, z + offsetZ))
}
Another thing we can work on is the delay with which each element arrives. We also use Perlin noise here, which makes it look like they arrive in “clusters”.
const frequency = 0.5
const delay = (noise(x * frequency, y * frequency) * 0.5 + 0.5) * delayFactor
We use perlin noise also in the waving effect, which modifies each point of the curve giving it a “flag waving” effect.
const { frequency, speed, amplitude } = this.webgl.controls.turbulence
const z = noise(x * frequency - time * speed, y * frequency) * amplitude
point.z = targetPoint.z + z
For the mouse interaction instead, we check if the point of the path is closer than a certain radius, if so, we calculate a vector which goes from the mouse point to the path point. We then move the path point a little bit along that vector’s direction. We use the lerp() function for this, which returns the interpolated value in the range specified, at one specific percentage. For example 0.2 means at 20%.
// displace the curve points
if (point.distanceTo(this.mousePoint) < displacement) {
const direction = point.clone().sub(this.mousePoint)
const displacementAmount = displacement - direction.length()
direction.setLength(displacementAmount)
direction.add(point)
point.lerp(direction, 0.2) // magic number
}
// and move them back to their original position
if (point.distanceTo(targetPoint) > 0.01) {
point.lerp(targetPoint, 0.27) // magic number
}
The remaining code handles the slideshow style animation, go check out the source code if you’re interested!
In the other two demos I used some different functions to shape the paths the elements move on, but overall the code is pretty similar.
Final words
I hope this article was easy to understand and simple enough to give you some insight into texture projection techniques. Make sure to check out the code on GitHub and download it! I made sure to write the code in an easy to understand manner with plenty of comments.
Let me know if something is still unclear and feel free to reach out to me on Twitter @marco_fugaro!
Hope this was fun to read and that you learned something along the way! Cheers!
It’s a famous fact that bitcoin mining hardware has changed by leaps and bounds lately due to the growth of new central processing units in the marketplace. The new machines may conduct Bitcoin processing at a faster rate when compared with the computers of yesteryear.
Furthermore, they consume less power. Field programming team array processors are connected with CPUs to boost their computing power. While selecting hardware for Bitcoin processing, ensure it includes a large hash rate that would deliver spectacular results to your users. According to experts, the rate of data processing is measured in mega hash rates each second, or GIGA hash speeds per second.
As a columnar database with both strong CPU and GPU performance, the OmniSci platform is well suited for Extract-Load-Transform (ELT) pipelines (as well as the data science workloads we more frequently demonstrate). In this blog post, I’ll demonstrate an example ELT workflow, along with some helpful tips when merging various files with drifting data schemas. If you’re not familiar with the two major data processing workflows, the next section briefly outlines the history and reasoning for ETL-vs-ELT; if you’re just interested in the mechanics of doing ELT in OmniSci, you can skip to the “Baywheels Bikeshare Data” section.
A Brief History of ETL vs. ELT for Loading Data
From the first computerized databases in the 1960s, the Extract-Transform-Load (ETL) data processing methodology has been an integral part of running a data-driven business. Historically, storing and processing data was too expensive to be accumulating data without knowing what you were going to do with it, so a process, such as the following. would occur each day:
Since I first became aware of the energy in this community, I felt the urge to be more engaged in this ‘digital avant-garde landscape’ that is being cultivated by the amazing people behind Codrops, Awwwards, CSSDA, The FWA, Webby Awards, etc. That energy has propelled me to set up this new portfolio, which acted as a way of putting my feet into the water and getting used to the temperature.
I see this community being responsible for pushing the limits of what is possible on the web, fostering the right discussions and empowering the role of creative developers and creative designers across the world.
With this in mind, it’s difficult not to think of the great art movements of the past and their role in mediating change. You can easily draw a parallel between this digital community and the Impressionists artists in the last century, or as well the Bauhaus movement leading our society into modernism a few decades ago. What these periods have in common is that they’re pushing the boundaries of what is possible, of what is the new standard, doing so through relentless experimentation. The result of that is the world we live in, the products we interact with, and the buildings we inhabit.
The websites that are awarded today, are so because they are innovating in some aspects, and those innovations eventually become a new standard. We can see that in the apps used by millions of people, in consumer websites, and so on. That is the impact that we make.
I’m not saying that a new interaction featured on a new portfolio launched last week is going to be in the hands of millions of people across the globe in the following week, although constantly pushing these interactions to its limits will scale it and eventually make these things adopted as new standards. This is the kind of responsibility that is in our hands.
Open Source
We decided to be transparent and take a step forward in making this entire project open source so people can learn how to make the things we created. We are both interested in supporting the community, so feel free to ask us questions on Twitter or Instagram (@brunoarizio and @lhbzr), we welcome you to do so!
With the portfolio, we took a meticulous approach to motion and collaborated to devise deliberate interactions that have a ‘realness’ to it, especially on the main page.
The mix of the bending animation with the distortion effect was central to making the website ‘tactile’. It is meant to feel good when you shuffle through the projects, and since it was published we received a lot of messages from people saying how addictive the navigation is.
A lot of my new ideas come from experimenting with shaders and filters in After Effects, and just after I find what I’m looking for — the ‘soul’ of the project — I start to add the ‘grid layer’ and begin to structure the typography and other elements.
In this project, before jumping to Sketch, I started working with a variety of motion concepts in AE, and that’s when the version with the convection bending came in and we decided to take it forward. So we can pretty much say that the project was born from motion, not from a layout in this matter. After the main idea was solid enough, I took it to Sketch, designed a simple grid and applied the typography.
Collaboration
Working in collaboration with Luis was so productive. This is the second (of many to come) projects working together and I can safely say that we had a strong connection from start to finish, and that was absolutely important for the final results. It wasn’t a case in which the designer creates the layouts and hands them over to a developer and period. This was a nuanced relationship of constant feedback. We collaborated daily from idea to production, and it was fantastic how dev and design had this keen eye for perfectionism.
From layout to code we were constantly fine-tuning every aspect: from the cursor kinetics to making overhaul layout changes and finding the right tone for the easing curves and the noise mapping on the main page.
When you design a portfolio, especially your own, it feels daunting since you are free to do whatever you want. But the consequence is that this will dictate how people will see your work, and what work you will be doing shortly after. So making the right decisions deliberately and predicting its impact is mandatory for success.
Technical Breakdown
Luis Henrique Bizarro, Creative Developer — @lhbzr
Motion Reference
This was the video of the motion reference that Bruno shared with me when he introduced me his ideas for his portfolio. I think one of the most important things when starting a project like this with the idea of implementing a lot of different animations, is to create a little prototype in After Effects to drive the developer to achieve similar results using code.
That’s my favorite stack to work with right now; it gives me a lot of freedom to focus on animations and interactions instead of having to follow guidelines of a specific framework.
In this particular project, most of the code was written from scratch using ECMAScript 2015+ features like Classes, Modules, and Promises to handle the route transitions and other things in the application.
In this case study, we’ll be focusing on the WebGL implementation, since it’s the core animation of the website and the most interesting thing to talk about.
1. How to measure things in Three.js
This specific subject was already covered in other articles of Codrops, but in case you’ve never heard of it before, when you’re working with Three.js, you’ll need to make some calculations in order to have values that represent the correct sizes of the viewport of your browser.
In my last projects, I’ve been using this Gist by Florian Morel, which is basically a calculation that uses your camera field-of-view to return the values for the height and width of the Three.js environment.
I usually store these two variables in the wrapper class of my applications, this way we just need to pass it to the constructor of other elements that will use it.
In the embed below, you have a very simple implementation of a PlaneBufferGeometry that covers 75% of the height and width of your viewport using this solution.
2. Uploading textures to the GPU and using them in Three.js
In order to avoid the textures to be processed in runtime while the user is navigating through the website, I consider a very good practice to upload all images to the GPU immediately when they’re ready. On Bruno’s portfolio, this process happens during the preloading of the website. (Kudos to Fabio Azevedo for introducing me this concept a long time ago in previous projects.)
Another two good additions, in case you don’t want Three.js to resize and process the images you’re going to use as textures, are disabling mipmaps and change how the texture is sampled by changing the generateMipmaps and minFilter attributes.
The method .initTexture() was introduced back in the newest versions of Three.js in the WebGLRenderer class, so make sure to update to the latest version of the library to be able to use this feature.
But my texture is looking stretched! The default behavior of Three.js map attribute from MeshBasicMaterial is to make your image fit into the PlaneBufferGeometry. This happens because of the way the library handles 3D models. But in order to keep the original aspect ratio of your image, you’ll need to do some calculations as well.
There’s a lot of different solutions out there that don’t use GLSL shaders, but in our case we’ll also need them to implement our animations. So let’s implement the aspect ratio calculations in our fragment shader that will be created for the ShaderMaterial class.
So, all you need to do is pass your Texture to your ShaderMaterial via the uniforms attribute. In the fragment shader, you’ll be able to use all variables passed via the uniforms attribute.
In Three.js Uniform documentation you have a good reference of what happens internally when you pass the values. For example, if you pass a Vector2, you’ll be able to use a vec2 inside your shaders.
We need two vec2 variables to do the aspect ratio calculations: the image resolution and the resolution of the renderer. After passing them to the fragment shader, we just need to implement our calculations.
In this snippet we’re using template strings to represent the code of our shaders only to keep it simple when using CodeSandbox, but I highly recommend using glslify to split your shaders into multiple files to keep your code more organized in a more robust development environment.
We’re all good now with the images! Our images are preserving their original aspect ratio and we also have control over how much space they’ll use in our viewport.
3. How to implement infinite scrolling
Infinite scrolling can be something very challenging, but in a Three.js environment the implementation is smoother than it’d be without WebGL by using CSS transforms and HTML elements, because you don’t need to worry about storing the original position of the elements and calculate their distance to avoid browser repaints.
Overall, a simple logic for the infinite scrolling should follow these two basic rules:
If you’re scrolling down, your elements move up — when your first element isn’t on the screen anymore, you should move it to the end of the list.
If you’re scrolling up, your elements move to down — when your last element isn’t on the screen anymore, you should move it to the start of the list.
Sounds reasonable right? So, first we need to detect in which direction the user is scrolling.
The variable this.scroll.values.target is responsible for defining to which scroll position the user wants to go. Then the variable this.position.current represents the current position of your scroll, it goes smoothly to the value of the target with the * 0.1 multiplication.
After detecting the direction the user is scrolling towards, we just store the current position to the this.position.previous variable, this way we’ll also have the right direction value inside the requestAnimationFrame.
Now we need to implement the checking method to make our items have the expected behavior based on the direction of the scroll and their position. In order to do so, you need to implement a method like this one below:
The website motion reference has four different animations happening while the user is scrolling:
Movement on the z-axis: the image moves from the back to the front.
Bending on the z-axis: the image bends a little bit depending on its position.
Image scaling: the image scales slightly when moving out of the screen.
Image distortion: the image is distorted when we start scrolling.
My approach to implementing the animations was to use a calculation of the element position divided by the viewport height, giving me a percentage number between -1 and 1. This way I’ll be able to map this percentage into other values inside the ShaderMaterial instance.
The implementation of the z-axis animation is pretty simple, because it can be done directly with JavaScript using this.position.z from Mesh, so the code for this animation looks like this:
The implementation of the bending animation is slightly more complex, we need to use the vertex shaders to bend our PlaneBufferGeometry. I’ve choose distortion as the value to control this animation inside the shaders. Then we also pass two other parameters distortionX and distortionY which controls the amount of distortion of the x and y axis.
We hope you liked the Case Study we’ve written together, if you have any questions, feel free to ask us on Twitter or Instagram (@brunoarizio and @lhbzr), we would be very happy to receive your feedback.
