Throughout 2022 I spent a lot of time learning about Bezier splines, MIDI, procedural mesh generation, Unity’s C# jobs system and using compute buffers to render particle geometry on the GPU.

My end goal was to make my own tool to generate audio-visual short films with. Ultimately I moved engines to Unreal and pivoted to making live performance based experiences vs. offline videos (another post on that later), but architecting and building out these systems in Unity taught me many things!

The first iteration was very slow and cumbersome. I used the Mesh API to construct tubes from bezier splines, but it was very slow as soon as you started to get anything decently complex. Not mention, lots of bugs and also the UX of the system was cumbersome and difficult to use. However, I came up with a lot of the visual design that would go into future iterations of this work, not to mention the system I built for creating procedural animation curves from MIDI would not change:

Soon after I rebuilt the system into V2, which used the C# Jobs system to build out meshes (using the advanced Mesh API to set vertex and index buffer data directly) and to calculate the needed information from the bezier splines. This solved a lot of the performance issues, but also the UX/design of the code base was still a bit cumbersome and I couldn’t imagine using that system to build a ton of content on top of in the future. However, I was definitely getting closer.

V3 was the final version, and I’m still pretty happy with how it ended up. The crux of the system is based around the concept of a ‘Pathway’, which is simply a bezier spline (an array of bezier curve structs) but with some extra properties tucked into a custom struct.

A PathwaySet is a set of Pathways, and a PathwaySetCreator is a custom class that maps user defined parameters into a PathwaySet. Over time I developed a few different types of PathwaySetCreators to create sets of Pathways in interesting ways.

Every PathwaySetCreator is based off of an abstract class that has functionality for converting all of its Pathways into a native array (or compute buffer) of PathPoints. These points contain a position and an up/right/forward vector. Once these points are created, they can be used by other systems I built to create procedural meshes (like tubes along pathways) or to render particles in a VFX Graph system.

This process of rasterizing Bezier curves into discrete points could be done every frame thanks to the speed of using the multi-threaded Jobs system - meaning I could get immediate feedback from altering my Pathways in editor.

Below are some animations I created using my system:

All of this code runs in the editor, so there is no need to hit play! In addition, I designed everything to be scrub-able on a timeline, meaning everything is deterministic and pre-defined, and all animations are based on either procedurally created animation curves (using MIDI for timing) or deterministic mathematical functions. You may ask why I did this, well the reason was because my end goal here was to make a short film, and I wanted the ability to really dial in every shot by being able to pause and scrub all the visual content on the timeline. Future iterations of this in Unreal engine (for a future post), will be more event based, to allow for me ot do live control in an un-deterministic way.