Duration: October 2017 - 2 days of work

Role: Designer

Tools: Unity (built-in renderer), HoloLens

The audio design agency Listen approached me to help pitch an interactive installation piece to Brian Eno where users wear a HoloLens and create tones which are visualized as floating holographic orbs, essentially a 3D version of Brian Eno's well known "Bloom"  iPhone app but 3D and in mixed reality. 

My role was in creating a prototype on the HoloLens that could be demonstrated to Peter Chilvers and Brian Eno to convey the idea and demonstrate a proof of concept.

The prototype was a success, and a full multi-user experience has been built out by We Are Listen for the public to experience at The Transformatorhuis in Amsterdam http://bloomopenspace.com/

Below is a video of the prototype I built (the sounds were created by Listen):

The prototype explored these areas:

• Visualizing the tone of the sound (shader, animation and FX look development)

• How you create the sound producing orbs

  • Placing the orb on the location of the user's hand

  • Placing the orb in front of the users gaze. (in both cases the air tap gesture is used to initiate the action of creating an orb)

  • What is the best input method for the hardware's limitations?

• Spatial sound design

• UX

  • How many orbs can be active at once?

  • How do we ensure a seamless experience if sounds must be capped at a certain number?

  • How do we map input to the different pitches of sounds? How do we help the user discover how this mapping works?