Duration: January - February 2018
Role: Technical Designer
Tools: Unity, TouchDesigner, Python
About
This was a physical installation project I worked on with my buddy Chris Rojas for Nike’s new launch of their Nike React fly knit shoe, where they took over a space in downtown Chicago, converting it into an immersive playground centered around their new shoe.
For this installation, up to three participants run on a giant floor treadmill together and see themselves represented as running avatars on the LED panels via Kinect tracking. At the end of the experience, they receive a video “boomerang” of their running.
We were hired by Portland based design agency Enjoy the Weather to handle all of these components regarding the avatar visualization, tracking and ultimately the boomerang video as well.
Most of the heavy lifting was in prototyping how we could track multiple runners and give them solid looking skeletal avatars only through multiple Kinect trackers. Our initial assumption was 3D point cloud visualizations, but as the art direction became more defined, we learned that we had to visualize the runners with a specific 2D character art style from a Chicago based artist. This meant figuring out how best to map 3D skeletal data onto a 2D avatar. I did a lot of experimentation into this inside of Unity, but in the end, given the highly stylized and deliberate art style, the simplest solution worked best, to just track the root position of the runners and attach a self animating 2D avatar to that position. Often times simple is best.
For the boomerang video generation, I used TouchDesigner, Python and FFmpeg. TouchDesigner would capture the video and create a copy of it in reverse, then with Python and FFmpeg I would splice them together and save out a new file. I had to use multi-threaded Python code to ensure that there was no frame hitch when the video file was being processed / saved, something I had to teach myself last minute on-site under a time pressure, but hey that’s part of the fun (sometimes).
