Many applications recently developed, and still in development, wish to make information systems and machines capable of identifying their users and considering their individual needs and emotions. Technical Research Centre of Finland Ltd studied how ordinary consumers could reliably verify the operation of systems by using human senses.

One day, AI systems and machines will progress to a point where they will be able to deeply understand the action of their users. Nowadays, traces of such process are already existing. For instance, AI devices are capable of generating an image of what humans are watching on a screen simply by analysing brain activity. Similarly, these machines can understand emotions of humans simply by analysing their faces’ microexpressions.

In the Human Verifiable Computing project, VTT used augmented and virtual reality to develop solutions for building trust between people and systems and facilitating the verification of information security. This is a key aspect of the digital future, in which interaction between people and computers will become an effortless part of everyday life. “Augmented and virtual reality technologies let us make fuller use of our senses and enable the constant mutual evaluation of reliability between humans and machines,” says Senior Scientist Kimmo Halunen of VTT. Making cryptographically verifiable computing available to human users was a key part of the project.

The project addressed functionalities involving computing verified with human senses. For instance, augmented reality was used to distribute single-use passwords, which could then be used through voice recognition. Then, it was also utilized to give multisensory feedback by showing visual instructions to a maintenance worker who turns a valve and receives an error message if the valve is operated incorrectly. The message can be implemented as an interactive image and also presented through audio on the user’s smart glasses. In addition, haptic feedback can be provided by making the user’s smart watch or other mobile device vibrate.

The results of the project indicate that the basic technology required for the verification of computing with the human senses is already available. The combination of augmented reality and safety information will also enable new services. Most of the times, current cryptographic methods and protocols are applied to communication between machines. VTT recognised that Including the user in the interaction will indeed require more research and system and application development, as well as more study of human behaviour.

Written by: Pietro Paolo Frigenti

Journal Reference: Halunen, K., Latvala, O. M., Karvonen, H., Häikiö, J., Valli, S., Federley, M., & Peltola, J. (2017). Human Verifiable Computing in Augmented and Virtual Realities.