Published on in Vol 2, No 1 (2016): December

Real-Time Tailoring of Depression Counseling by Conversational Agent

Real-Time Tailoring of Depression Counseling by Conversational Agent

Real-Time Tailoring of Depression Counseling by Conversational Agent

Authors of this article:

Lazlo Ring 1 Author Orcid Image ;   Timothy Bickmore 1 Author Orcid Image ;   Paola Pedrelli 2


1Northeastern University, Boston, MA, United States

2Massachusetts General Hospital, Boston, MA, United States

Corresponding Author:

Lazlo Ring

Northeastern University

1282 Bolyston Street

Apt 516

Boston, MA, 02215

United States

Phone: 1 917 783 6965


Background: Depression is the leading cause of disability in the world, with over 350 million people affected worldwide. To address this, many researchers have developed computer-based therapy systems to treat this condition. However, many of these systems have been shown to have significantly worse retention than face-to-face counseling, limiting their effectiveness. We believe this discrepancy is partially due to these systems’ inability to respond to a user’s emotional state in real time.

Objective: We have developed an affectively aware virtual agent for depression counseling: an animated computer character that responds to users’ affective states in real time during virtual therapy sessions. The virtual agent guides users through a manualized cognitive behavioral therapy (CBT) intervention, while sensing users’ affect from their facial expressions and changes in speech prosody. During this simulated face-to-face conversation, the agent provides real-time tailored responses in response to sensed user affect. The automatically generated responses corresponded to empathic statements that allowed participants to pause the interaction in order to calm and compose themselves and reflective statements that rephrased emotionally sensitive content elicited from participants to emulate empathic listening.

Methods: We conducted a quasi-experimental pilot study to assess acceptance of the system by individuals with mild to moderate depression. Ten individuals (5 male, 5 female, aged 18-28) participated in a single 30-minute CBT-based session with the affectively aware virtual agent. We measured depression (Patient Health Questionnaire-9, PHQ-9) and state anxiety before and after the session and self-reported attitudes towards the system on scale measures and via a semi-structured interview.

Results: Participants scored between 5 and 15 (mean=6.6, SD=2.7) on baseline depression measures (PHQ-9). No significant changes were found in pre-post tests for depression (pre-interaction mean=8.0, SD=4.62, post-interaction mean=6.4, SD=3.13) or anxiety (pre-interaction mean=23.3, SD=6.46, post-interaction mean=21.78, SD=7.61), although both trended towards improvement. Agent ratings were generally neutral across the board, with satisfaction (mean=4.5, SD=1.35), desire to continue using (mean=4.2, SD=1.6), trust (mean=3.9, SD=1.66), and likability (mean=4.4, SD=1.7) scoring around the mid-point. Thematic analysis of the semi-structured interviews indicated that 50% of participants reported that the agent evoked emotional responses in them during the interactions and that 70% of participants felt the agent understood their emotional state. Participants stated that this was caused by the agent presenting them with “information they did not realize” (Patient 5) and that the agent “… understood my emotions because I felt that it gave me the right responses” (Patient 8).

Conclusions: This study suggest that it is possible for a virtual agent to evoke and respond to a user’s emotional state in real time during counseling sessions. Although we did not design this study to produce significant changes in depressive symptoms due to its size and duration, the majority of users expressed that they felt the agent understood their emotional state and responded appropriately.

iproc 2016;2(1):e27



Multimedia Appendix 1


PDF File (Adobe PDF File), 10MB

Edited by T Hale; submitted 01.06.16; peer-reviewed by CHS Scientific Program Committee; accepted 02.08.16; published 30.12.16


©Lazlo Ring, Timothy Bickmore, Paola Pedrelli. Originally published in Iproceedings (, 30.12.2016.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in Iproceedings, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.