Page 30

EETE MAY 2013

HAPTICS & USER INTERFACES Critical factors in air-mouse system design By Steve Scheirey Motion control is coming to Smart TVs in the form of in-air mice. MEMS sensors and sensor fusion algorithms provide the foundation for wireless, 3D cursor control, which bring exciting new interaction opportunities to the TV. While the basic interaction design for these air-mice comes from the familiar PC mouse, there are vital, additional considerations which impact the implementation. This article will dive into what separates an in air-mouse from a standard mouse, and reveal how systems engineers are solving some of the most pressing challenges facing in-air cursor control. A desk mouse and an air-mouse are similar but different. So let’s start with the foundation for in-air, 3D motion control: the mouse. The design of the mouse as an input device for point-and-click control is based on decades of research and testing by human factors engineers. It was first created in the 1960s, adopted for commercial use in the 1980s, and is still in wide use today because it solves a fundamental problem for point-and-click control of a graphical user interface (GUI) on a display. The two most important and distinguishing characteristics of all computer mice are bounded pointing and non-linear ballistics cursor movement. Bounded pointing, also known as relative pointing, is a feature whereby the mouse cursor stays bounded on the screen at all times. Bounded pointing was an explicit choice by human factors engineers who determined that it was better to decouple the movement of the mouse from the cursor on the screen. The advantage is that the cursor is always visible. This gives the user a feeling that they are in control and haven’t “lost the cursor”. Non-linear ballistics cursor movement means that cursor movement is not a 1:1 mapping with mouse movement. When the user moves the mouse fast, the cursor accelerates and moves with a larger velocity toward the target. When the user moves the mouse slower, the cursor slows down and moves with a smaller velocity. This helps in two ways. The acceleration Fig 1: Target latency ranges for cursor control. means less effort is required to move the cursor across the screen. The deceleration enables the user to more effectively hit smaller targets on the screen. These mouse dynamics form the foundation of air-mouse cursor control. Smart TVs present a similar GUI on a display to that found on a PC, and therefore bounded pointing using nonlinear ballistics cursor movement, just like a PC mouse, were chosen for the interaction architecture. However, these dynamics were necessary but not sufficient for usable in-air operation in the living room. This is because there are other important, non-obvious and non-trivial algorithms required to make an airmouse user friendly. These requirements result from the different ways in which a PC mouse and TV air-mouse are used. The PC mouse is used in a well-defined, standard setting—a user sitting at a desk using a PC in front of them, while interaction with the PC is the main focus of activity. The air-mouse is typically used in a living room, where a user can be sitting, standing or lying down, holding the controller at any angle, and where their main goal is to find and consume content. This contrast creates three key challenges for air-mouse systems engineers, and solving these challenges will be the focus of the rest of the article. Challenge 1: an air-mouse has no simple wired connection The numerous potential positions of the user in the room make a wired connection between the air-mouse and the PC impossible. Therefore, communication must take place over RF link, with a trade-off between RF packet rate and battery life. In addition, 3D in-air movement must be mapped and translated into a 2D cursor, which has to be processed in the OS and shown on screen. This introduces a significant issue in the system: latency. Humans expect instantaneous results to physical actions, and therefore expect immediate response when using motion control. But given the complicated data collection, transmission and processing needed to map motion to cursor control as described briefly above. The question becomes: “how fast is fast enough?” Our studies show that the target latency of the motion control system should be 30-50ms. Above this level, latency is noticeable to even novice users and usability is affected – see figure 1. In testing we have observed decreasing target selection effectiveness with significant overshoot in systems with latencies of 60ms and above. Associated with this overshoot and lack of accuracy was an increase in user fatigue, making the latency targets an essential foundation for a successful motion control system. Challenge 2: An air-mouse does not have the benefit of a flat surface with friction. The lack of the desk which the PC mouse rests on, and the friction which stabilizes movement, increases the impact of unintentional movement on cursor control. Specifically, hand tremor and movement caused by button clicks in air-mice is not only distracting but incredibly harmful to the user’s ability to navigate the interface. So it is essential to remove human tremor and make the cursor more stable on the screen during in-air operation. A common approach to this is using a simple low pass filter to eliminate all movements with an input velocity of under 5°/ second. However, this also affects small intentional movements, creating what is known as a motion ‘dead zone’ and damaging Steve Scheirey is Vice President, TV Software and Services at Hillcrest Labs - www.HillcrestLabs.com 30 Electronic Engineering Times Europe May 2013 www.electronics-eetimes.com


EETE MAY 2013
To see the actual publication please follow the link above