Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, March 11, 2019 at 4:00 PM to 4:30 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?


Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 30.12.16 in Vol 2, No 1 (2016): December

This paper is in the following e-collection/theme issue:


    Real-Time Tailoring of Depression Counseling by Conversational Agent

    1Northeastern University, Boston, MA, United States

    2Massachusetts General Hospital, Boston, MA, United States

    Corresponding Author:

    Lazlo Ring

    Northeastern University

    1282 Bolyston Street

    Apt 516

    Boston, MA, 02215

    United States

    Phone: 1 917 783 6965



    Background: Depression is the leading cause of disability in the world, with over 350 million people affected worldwide. To address this, many researchers have developed computer-based therapy systems to treat this condition. However, many of these systems have been shown to have significantly worse retention than face-to-face counseling, limiting their effectiveness. We believe this discrepancy is partially due to these systems’ inability to respond to a user’s emotional state in real time.

    Objective: We have developed an affectively aware virtual agent for depression counseling: an animated computer character that responds to users’ affective states in real time during virtual therapy sessions. The virtual agent guides users through a manualized cognitive behavioral therapy (CBT) intervention, while sensing users’ affect from their facial expressions and changes in speech prosody. During this simulated face-to-face conversation, the agent provides real-time tailored responses in response to sensed user affect. The automatically generated responses corresponded to empathic statements that allowed participants to pause the interaction in order to calm and compose themselves and reflective statements that rephrased emotionally sensitive content elicited from participants to emulate empathic listening.

    Methods: We conducted a quasi-experimental pilot study to assess acceptance of the system by individuals with mild to moderate depression. Ten individuals (5 male, 5 female, aged 18-28) participated in a single 30-minute CBT-based session with the affectively aware virtual agent. We measured depression (Patient Health Questionnaire-9, PHQ-9) and state anxiety before and after the session and self-reported attitudes towards the system on scale measures and via a semi-structured interview.

    Results: Participants scored between 5 and 15 (mean=6.6, SD=2.7) on baseline depression measures (PHQ-9). No significant changes were found in pre-post tests for depression (pre-interaction mean=8.0, SD=4.62, post-interaction mean=6.4, SD=3.13) or anxiety (pre-interaction mean=23.3, SD=6.46, post-interaction mean=21.78, SD=7.61), although both trended towards improvement. Agent ratings were generally neutral across the board, with satisfaction (mean=4.5, SD=1.35), desire to continue using (mean=4.2, SD=1.6), trust (mean=3.9, SD=1.66), and likability (mean=4.4, SD=1.7) scoring around the mid-point. Thematic analysis of the semi-structured interviews indicated that 50% of participants reported that the agent evoked emotional responses in them during the interactions and that 70% of participants felt the agent understood their emotional state. Participants stated that this was caused by the agent presenting them with “information they did not realize” (Patient 5) and that the agent “… understood my emotions because I felt that it gave me the right responses” (Patient 8).

    Conclusions: This study suggest that it is possible for a virtual agent to evoke and respond to a user’s emotional state in real time during counseling sessions. Although we did not design this study to produce significant changes in depressive symptoms due to its size and duration, the majority of users expressed that they felt the agent understood their emotional state and responded appropriately.

    iproc 2016;2(1):e27



    This poster was presented at the Connected Health Symposium 2016, October 20-21, Boston, MA, United States. The poster is displayed as an image in Figure 1 and as a PDF in Multimedia Appendix 1.

    Figure 1. Poster.
    View this figure

    Multimedia Appendix 1


    PDF File (Adobe PDF File), 10MB

    Edited by T Hale; submitted 01.06.16; peer-reviewed by CHS Scientific Program Committee; accepted 02.08.16; published 30.12.16

    ©Lazlo Ring, Timothy Bickmore, Paola Pedrelli. Originally published in Iproceedings (, 30.12.2016.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in Iproceedings, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.