Designing for hands, gesture and VR

How do we design and develop a good user experience for an entirely new technology, where the rules aren’t written yet?

At Ultraleap, we’re tasked with establishing best UX practice for hand-tracking – technology that lets you control VR experiences and touchless displays by gesturing in the air using only your hands. This means new products like touch-free public kiosks, or pilot and surgeon training in VR without any controllers at all.

This whole new way of interacting has meant our teams need to figure out the fundamentals of user experience design all over again, because no-one else has yet. How does a menu panel work when it’s in three dimensions? How do you use a webpage when you’re standing two feet away from it? And how can we ensure a good experience for all, when everyone’s hands, habits and expectations are slightly different from one another?

In this talk, I’ll tell the story of how researchers, designers and developers at Ultraleap learned to adapt how we work to suit this unique experiences, and how we pulled together to start figuring out the next big step in interaction design. I’ll show real projects and prototypes, and give a candid explanation of what we learned, and how we went about it.