From July 2013 to February 2017 I worked on HoloLens as a Technical Artist for the platform design team, working between the roles of experience designer, 3D artist and technical artist.
We designed and helped deliver the first version of the Windows Mixed Reality operating system that ships with the developer edition of HoloLens V1.
It was a very unique experience I got the privilege of having, working with a team of incredibly intelligent and talented people on some very futuristic hardware. During my 4 years I got to see the evolution of three different generations of HoloLens prototype hardware as well as the concept of Mixed Reality being more ingrained into the public consciousness.
Mixed reality refers to the merging of the analog and digital worlds, where digital content leaves the confines of a screen to live in the physical world. To achieve this, the HoloLens brings together see-through lenses, numerous sensors, cameras and microphones all in a form factor that fits on your head, no cables required. Hardware without an operating system is not very helpful though, and building one for the HoloLens means building an OS that runs on a 3D rendering engine and that leverages and exposes all of the new sensors and inputs in a well designed and user friendly way.
prototyping
Prototyping for the holographic OS in Unity. This includes scripting, building assets, lighting, rendering, animation, working with the HoloLens's hand tracking and head tracking inputs and surface reconstruction.
asset creation
Building the assets (models, textures, animations, materials, state machines, scripts) that became the final components of the operating system
pipeline
Light weight Maya python scripts to get the job done. Research into audio middleware technologies and audio engines
Look development
Designing in Unity, Maya and Houdini visual styles for HoloLens assets
Keyboard asset, using gaze and gesture interaction models. Designed and built to accommodate a small FOV
Teaching the gaze interaction model by looking gems that produce sound when you gaze over them
Manipulating a hologram with gesture interaction model. Cursor states reflect current actions.