IT specialists have created a new app that gives the possibility to users of operating their smartphone using gestures. This innovation opens the path for new ranges of interactions with these devices.

The new app was developed at ETH Zurich by the Professor Otmar Hilliges and his staff. The app works by holding the phone in one hand while moving the other hand around, in the air above the phone’s camera, with gestures similar to the sign language. By imitating the firing of a pistol, for instance, the user can change browser tab, shooting enemy planes while playing, or change the type of maps’ view. Spread the fingers could scroll the pages of a book or make a map automatically zooming in a section. The new technology was made possible by Jie Song, a Master’s student part of the Computer Science professor Hilleges’ team, who developed a new algorithm later included in the app. The app was presented at the UIST symposium in Honolulu, Hawaii.

When using the app, the smartphone registers the environment around through the camera. It recognises the shapes and the movements without taking in account depth or colour, and then compares the information acquired with the stored memory. Consequently, according to the comparison result, the program executes the related command. The program also recognise the distance between mobile and moving hand, warning the user when it is not enough or when there is too much space in between. The program actually recognises six gesture patterns and succeeds in executing the related programmes. The researchers are working to expand the number of classification schemes in order to increase the possible commands and the effective interaction. This story offers an insight of the fact that even though humans are always moving toward technology shifting the world through a virtual reality, sometimes technology allows devices to step back toward humans, interacting halfway.

Written by: Pietro Paolo Frigenti