Time-of-flight hand posture recognition using compact normalized histogram

Gesture recognition plays a key role in Human Computer Interaction (HCI) to let users interact with devices in a touchless and intuitive way. However, sensors often produce huge amounts of data, are power-hungry, or uncomfortable, making it difficult to integrate them into standalone wearable devices. In this work, we exploit a new Time-of-Flight (ToF) sensor by STMicroelectronics to perform still gesture recognition through a Convolutional Neural Network (CNN). With just 215 mW in full power mode, the new multizone ToF sensor provides two types of data: an 8×8 Distance Map (DM) and a Compact Normalized Histogram (CNH), which is a histogram with the number of photons collected by each pixel at different time slots. We use both data to train two different CNNs. Results are then compared in terms of accuracy and weight size, showing that CNH is not only better than DM in terms of accuracy (96% vs 88%), but also requires less memory (9.1 kB vs 18.8 kB). Finally, we embed the best CNH model into two different microcontrollers: an ARM Cortex-M4 and an ARM Cortex-M7 with a built-in hardware accelerator. We then compare the embedded models on occupied memory (RAM and Flash) and inference time. In conclusion, we demonstrate how CNN built using CNH performs well in recognizing static hand posture and with significantly less memory consumption for network weights. These results disclose the potential of CNH data from ToF sensors for ultra-lite gesture recognition in low-power wearables.