Jamming with a robot drummer while he posts pictures on Facebook? It is happening and it is really cool. Researchers have created a robot capable of playing drums along with human keyboard players, with the possibility of taking pictures of the jam sessions and post them on Facebook.

The research has been carried out by scientists at the Queen Mary University of London, investigating he interaction between humans and robot and the effect of social media on such relationship.

The establishment of strong bonds between humans and robots require the humans to be strongly engaged with the robot and to ‘trust’ it, recognising in it a certain level of social presence. Music seems to be the ideal tool to foster engagement, according to researchers, and this led the team to develop a robotic drummer, called Mortimer, capable of composing music responsively to human pianists’ real-time performances.

To make the robot even more real and the interaction even more deep, Mortimer’s developers added the capacity of taking pictures and posting them on Facebook with a supporting comment while tagging the keyboard player.

The research was published in IEEE Transactions on Cognitive and Developmental Systems.

Lead author Louis McCallum, from Queen Mary’s School of Electronic Engineering and Computer Science, explained: “We’d previously uncovered new and exciting findings that suggested open-ended creative activities could be a strong bedrock to build long-term human-robot relationships. This particular research sought to examine whether the relationships that were initially developed face-to-face, but under lab conditions, could be extended to the more open, but virtual, realm of social media.”

To carry out the study, two groups of participants were defined. The first group was sent a Facebook friend request from Mortimer allowing the robot to tag them in pictures taken during the session. The second group, instead, was not sent a request and had no contact with the robot except from the jam sessions.

Participants performed in six weekly sessions in a controlled studio environment and were told to stay for a minimum of 20 minutes but, if willing, could optionally stay for up to 45.

They were welcomed by Mortimer, who communicates via speech synthesis software, and could use a tablet to interact with him.

During each session, a webcam in the lab automatically took a picture of Mortimer and the participant playing and was posted with an automatically generated accompanying comment. In some instances, the participants also took a selfie with Mortimer and posted it to their own Facebook accounts.

From the Facebook data, there were considerably more ‘likes’ for posts made by a user in contrast with the automatic ones of Mortimer’s that the user was tagged in.

Dr McCallum observed: “One of the most interesting findings was that posts by human participants about the music sessions between them carried significantly more weight within their networks than posts by the robot itself. This suggests a discerning approach to generated posts that is especially relevant in today’s world of social media bots, automated content and fake news.”

The research showed that the time spent with the robot increased over the study but session length for the group who were Facebook friends with Mortimer reduced over time. They suggest this may be because the participants had additional contact with Mortimer outside the sessions.

Co-author of the study Professor Peter McOwan added: “There are signs of high engagement, such as high self-reported repeat interaction, across all participants that strengthen previous results about the use of music as a good base for improving long-term human-robot relationships. But we found the effects of extending the relationship into the virtual world were less pronounced than expected. This doesn’t mean that virtual interaction doesn’t help, but maybe the quality of the interaction needs to be improved.”

The study was funded by The Engineering and Physical Sciences Research Council (EPSRC).

Written by: Pietro Paolo Frigenti

Journal Reference: McCallum, L. McOwan, P. W. (2017). Extending Human-Robot Relationships Based in Music with Virtual Presence. IEEE Transactions on Cognitive and Developmental Systems; DOI: 10.1109/TCDS.2017.2779218