A computer can predict positive or negative changes in relationships, through the analysis of the two partners’ tone of voice, with about 79% accuracy. The computer algorithm obtained more accurate results than relationships expert in analysing marital success of couples with serious marital issues. The study was published in Proceedings of Interspeech on September 6, 2015.

The first step of the research involved recording hundred of conversation from more than one hundred couples during their marriage therapy session for two years. Then the marital status of the same couples was tracked for five years to analyse the progresses.

The team that carried out the research was lead by Shrikanth Narayanan and Panayiotis Georgiou of the USC Viterbi School of Engineering with their doctoral student Md Nasir and supported by the collaboration of Brian Baucom of University of Utah. The researchers developed an algorithm capable of breaking down the audio recordings into acoustic features through the use of speech-processing techniques. Some examples of the features are intensity, pitch, “jitter” and “shimmer” among the participants, as well as the tracking of warbles in the voice to outline important moments of high emotion.

“What you say is not the only thing that matters, it’s very important how you say it. Our study confirms that it holds for a couple’s relationship as well,” Nasir explains. The information acquired as acoustic features allowed the program to recognise the subjects’ communicative state and the variations in that state during a single therapy and over the different therapies.

The features were analysed in a complex way, studying the impact of a partner upon the other over the different therapy sessions.

“It’s not just about studying your emotions,” Narayanan explains. “It’s about studying the impact of what your partner says on your emotions.”

“Looking at one instance of a couple’s behaviour limits our observational power,” Georgiou added. “However, looking at multiple points in time and looking at both the individuals and the dynamics of the dyad can help identify trajectories of the their relationship.”

Once all the regulations were set, the new program was tested in a comparison with behavioural analysis made by human experts who, in the process of coding, associated to them positive factors like “acceptance” and negative qualities like “blame”. The researcher found that analysing the voice directly, instead of the behavioural codes made by the experts, gave more accurate information about the couples’ future.

“Psychological practitioners and researchers have long known that the way that partners talk about and discuss problems has important implications for the health of their relationships. However, the lack of efficient and reliable tools for measuring the important elements in those conversations has been a major impediment in their widespread clinical use. These findings represent a major step forward in making objective measurement of behaviour practical and feasible for couple therapists,” Baucom explained.

The team is now organising the next study, planning to use language, for example spoken words, and nonverbal language, such as body language, to predict how effective treatments will be, through a framework for computationally understanding human behaviour developed by Narayanan.

Written by: Pietro Paolo Frigenti