Distant displays such as interactive Public Displays or Interactive Television require new interaction techniques as traditional input devices may be limited or missing in these contexts. Free hand interaction, as sensed with computer vision techniques, presents a promising interaction technique. This project with Gilles Bailly presents the adaptation of three menu techniques for free hand interaction: Linear menu, Marking menu and Finger-Count menu.

The first study based on a Wizard-of-OZ protocol focuses on Finger-Counting postures in front of interactive television and public displays. It reveals that participants do choose the most efficient gestures neither before nor after the experiment. Results are used to develop a Finger-Count recognizer.

The second experiment shows that all techniques achieve satisfactory accuracy. It also shows that Finger-Count requires more mental demand than other techniques.

We developed an algorithm demonstrated in the following video to identify individual finger tips from depth image streams that are provided by the Kinect camera.

Results of this project were published at INTERACT 2011: