Gesture Learning

Apple products use the same gesture vocabulary for input. Certainly, unifying gestures across devices helps their adoption. However, in the context of a population with hugely variable abilities, achieving broad or general impact requires the adaptation or customization of input gestures for each individual. As a first step, we aim to make touchscreens truly accessible to populations with involuntary motion, such as Cerebral Palsy patients or elderly individuals with essential tremors. We are building a system that learns and recognizes both intended and unintended multi-touch gestures. By recognizing the latter, we can customize accessibility for individuals exhibiting involuntary movements.