Page 34

EETE MAY 2013

HAPTICS & USER INTERFACES Exploiting depth sensing for 3D interfaces and complex image analysis By Michael Delueg Nowadays intelligent sensor networks have entered many application fields ranging from building and industrial automation to traffic management or medical applications. Intelligent sensors are able to process the sensor inputs on the sensor node and trigger actions autonomously. This allows intelligent sensor networks with information sharing even on bandwidth constrained networks or serial busses. The availability of powerful embedded processors makes it possible to process the input of high-bandwidth sensors like image sensors in real-time for complex image analysis tasks. With the advent of the Kinect as input unit for their console, Microsoft has sparked developments in the area of gesture recognition, novel user interfaces and depth sensors. A depth sensor is an image sensor which provides distance information for each pixel. They usually don’t provide color information, but an accurate and robust 3D representation of the scene with X/Y/Z coordinates for each pixel. This eliminates computational overhead for obtaining the 3D data in the next processing stages which is needed for a stereo vision system. Since the sensor output is already robust 3D data analysis functions can be implemented efficiently – see figure 1. There are a couple of depth-sensing technologies in existence. Bluetechnix uses PMD (Photonic Mixer Device) sensors from pmdtechnologies based on the Time-of-Flight (ToF) technology which is more robust than Kinect’s structured light technology. ToF-sensors use a LIDAR (LIght Detection and Ranging) approach for distance measurement. The target is illuminated by an active IR light source and the distance to the object is calculated based on the backscattered light. The PMD sensor chip uses a modulated light source with a frequency range of 5 to 30MHz. The phase shift between the emitted light and the reflected light at the receiver in conjunction with the known modulation frequency and speed of light can be used to calculate the distance to the object. The depth resolution of the sensor is in the centimeter range or below in good conditions. The use of a modulated IR light and the phase measurement make the sensor robust against difficult ambient light conditions. The whole scene is captured in one shot for high frame rates up to 160fps, limiting motion artifacts. System design considerations A system design for an intelligent 3D sensor based on this technology needs to address the following topics: illumination, sensor, optics, power, processing unit and connectivity. While Time-of-Flight is a scalable technology, the choice of field of view (FoV) and range is an important decision. The sensor’s lens and the beam of the illumination LEDs must be adjusted accordingly to illuminate the complete FoV of the sensor evenly. Typical setups for close range applications like a gesture control have a FoV of 90° or more and a range below 1m while people tracking applications might go up to 3-5m but still need to cover the same FoV. At greater distances the FoV usually becomes Fig. 1: Basic schematics of a Time-of-Flight system. smaller because too much power is needed to illuminate the area and the area a single sensor pixel represents becomes too big for meaningful analysis. Today’s ToF sensors feature a resolution of 160x120 pixels. The achievable range is mostly limited by the amount of power which is available to illuminate the whole scene and still get enough reflected light from the target to get accurate measurements. For each scenario the system designer has to balance an equation of range, FoV and frame rate versus power consumption, heat generation and system costs to achieve an optimal result. The processing unit controls all functions of the sensor and the illumination unit. It processes the raw sensor data and performs enhancements like lens correction or noise reduction. The additional resources can be dedicated on the user application. In terms of connectivity different scenarios are possible. PC based applications may rely on USB to stream the complete image data and power the sensor over USB. Ethernet allows long cable lengths and high bandwidth and the integration into existing IP security infrastructures. In sensor networks bus topologies play an important part when covering large areas with multiple sensors in a line or grid. A 4-wire cable with RS485 plus power supply is a robust choice for chaining multiple intelligent 3D sensors to cover a large room or monitor a conveyor belt. For all connectivity choices the needed bandwidth can scale from several kBit/s to several MBit/s depending on the amount of processing done on the sensor. People tracking and counting Multiple applications for such an intelligent 3D sensors deal with people counting, tracking and behaviour analysis. In opposition to HMI applications like gesture recognition these applications cannot rely on the cooperation of the people and require very robust sensor data. A good example for such a use case is people counting in public transportation which provides very important data sets for the operators – see figure 2. During rush hour light barriers or infrared sensors have difficulties to Michael Delueg is product Manager at bluetechnix - www.bluetechnix.com 34 Electronic Engineering Times Europe May 2013 www.electronics-eetimes.com


EETE MAY 2013
To see the actual publication please follow the link above