Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, December 24 through Wednesday, December 26 inclusive. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 27.10.15 in Vol 1, No 1 (2015): October

This paper is in the following e-collection/theme issue:

    Abstract

    Adapting A Unified Electronic Health Record Usability Framework for Evaluation of Connected Health Care Technologies Linking Mobile Data

    1The University of Texas Health Science Center at Houston, School of Nursing; School of Biomedical Informatics, Houston, TX, United States

    2The University of Texas Health Science Center at Houston, School of Biomedical Informatics, Houston, TX, United States

    Corresponding Author:

    Jing Wang, MPH, PhD

    The University of Texas Health Science Center at Houston

    School of Nursing; School of Biomedical Informatics

    6901 Bertner Avenue

    Son 614

    Houston, TX, 77030

    United States

    Phone: 1 7135009022

    Fax:1 7135002142

    Email:


    ABSTRACT

    Background: Evidence-based, objective, and systematic usability evaluation is key to successful connected health care technologies.  The increases in patient-facing mobile health technologies not only offer convenience for patients in managing their own health and/or chronic conditions, but also offer the opportunity for health care providers to access patient behaviors and patient-centered outcomes at ease. Thus, it is of significance to link and present patient-facing mobile device data to their health care providers in a secure and uninterrupted way that will facilitate workflow and promote patient provider communication, rather than drawing providers away from patients. This prompts increasing efforts developing connected health care technologies linking mobile data to electronic health record systems guided by user-centered design and redesign principles. However, lack of scientific, objective, and systematic usability evaluation put connected health care technologies at risk for low adoption and eventual failure.

    Objective: Learning from lessons in electronic health record usability evaluation, we propose to adapt an existing unified framework, TURF, for electronic health record usability evaluation to guide the design, redesign, and usability evaluation of connected health care technologies linking mobile data to electronic health record systems or other provider-facing Web-based evaluation tools.

    Methods: TURF, a unified framework of electronic health record usability, involves three dimensions: useful, usable, and satisfying; and four key components: task, user, representation, and function. Each dimension and component is described with theoretical underpinnings along with examples of how usability can be measured.

    Results: Specific adaptation of TURF that’s unique for connected health technologies include (1) user analysis for “satisfying” dimension will need to include both users using and mobile health users who’s feeding data into the system; (2) function analysis for “useful” dimension will need to consider functions/data wanted by the providers, functions actually used in real activities, functions/data available from mobile devices and with agreement from patients, functions/data available in interfaces within connected health care technologies ; (3) representation analysis for “usable” dimension need to consider correct representation of data from mobile devices in connected interface; (4) task analysis for “usable” dimension will highlight learnability, efficiency (time on task, steps on task, task success, mental effort), and error prevention and recovery (occurrence rate, error recovery rate). Real world interruptions, team dynamics, and multitasking should also be considered during evaluation of connected health care technologies.

    Conclusions: An adapted framework is proposed to offer objective, evidence-based, and systematic usability evaluation to guide the design and redesign of interfaces connecting mobile data with electronic health record systems and Web-based evaluation tools.

    iProc 2015;1(1):e20

    doi:10.2196/iproc.4477

    KEYWORDS



    Introduction

    Connected health care technologies have significant values to both health care professional and patients [1]. Patient-facing technology tools including smartphone apps, wearable activity trackers, bluetooth enabled glucometers, wireless weight scales have significantly increased in the market and gained popular demands with potential to facilitate patient management of their own health. An example of these mobile tools are the use of electronic diaries for patient self-monitoring of diet and physical activity for both fitness purposes and can also be for chronic disease self-management. Self-monitoring is cornerstone of a successful behavioral lifestyle intervention for obesity and diabetes [2,3]. Mobile-based electronic diaries made self-monitoring easier and convenient for individuals. Accessing such mobile collected self-monitoring information can provide valuable information for their providers not only in setting individualized and realistic behavioral goals, but also in following up with these behavioral goals in an efficient and effective way that can help patient achieve successful behavior change and ultimately lead to better health care outcomes. Thus, it is of significance to link and present patient-facing mobile device data to their health care providers in a secure and uninterrupted way that will facilitate workflow and promote patient provider communication, rather than drawing providers away from patients. This prompts increasing efforts developing connected health care technologies linking mobile data to electronic health record systems guided by user-centered design and redesign principles. However, lack of scientific, objective, and systematic usability evaluation put connected health care technologies at risk for low adoption and eventual system failure.

     

    With lessons learned in electronic health record implementation across the United States, evidence-based, objective, and systematic usability evaluation is key to successful connected health care technologies. A review in health care information technology usability studies also pointed out that a theoretical framework to guide usability evaluation is essential [4]. Therefore, we propose to adapt an existing unified framework -TURF- for electronic health record usability evaluation to guide the design, redesign, and usability evaluation of connected health care technologies linking mobile data to electronic health record systems or other provider-facing Web-based evaluation tools. In summary, this paper will present the adapted TURF framework as a coherent, unified framework to guide objective and systematic usability evaluation [5].

     


    Methods

    TURF, standing for “task, user, representation, and function,” is a unified framework of EHR usability, that is “(1) a theory for describing, explaining, and predicting usability differences; (2) a method for defining, evaluating, and measuring usability objectively; (3) a process for designing built-in good usability; and (4) once fully developed, a potential principle for developing EHR usability guidelines and standards.”5 TURF defines usability as “how useful, usable, and satisfying a system is for the intended users to accomplish goals in the work domain by performing certain sequences of tasks” [5].

     

    TURF involves three dimensions: useful, usable, and satisfying; and four key components: task, user, representation, and function. Each dimension and component is described with theoretical underpinnings along with examples of how usability can be measured. The relationships between each dimension and components are depicted in Figure 1.

     

    Figure 1. The original TURF framework for Electronic Health Record Usability (from Zhang, 2011)[5].
    View this figure

    Results

    Adapted TURF for Connected Health Technologies

    Under adapted TURF, usability for connected health care technologies involving mobile data is defined as useful, usable, and satisfying for intended connected interface users to accomplish goals in the work domain by performing certain sequences of tasks, as well as useful, usable, and satisfying for mobile users feeding data into the connected health care technologies.

     

    User satisfaction is often measured by survey questions evaluating users’ perceptions or ratings on a scale. It is an important step, but it often gives individuals wrong impression that usability is subjective, unreliable, and useless for system design and redesign. TURF has both subjective and objective measures of usability. Both measures of usability will have to be conducted to give a complete picture of usability. 

    We presented each dimension and measure of usability under adapted TURF framework in Table 1.

    Table 1. Usability dimensions and measures of usability under adapted TURF for connected health care technologies.
    View this table

     

    User Analysis

    User analysis for “satisfying” dimension will need to include both users using and mobile health users who are feeding data into the system. User analysis is the first step of usability evaluation; it involves steps of identifying the types of users and characteristics of users that are using the connected interfaces (often being key health care professionals) as well as mobile technology users (often being patients or supporting personnel).

    Functional Analysis

    Function analysis for “useful” dimension will need to consider functions or data wanted by the providers, functions actually used in real activities, functions or data available from mobile devices and with agreement from patients, functions or data available in interfaces within connected health care technologies

    Representational Analysis

    Representation analysis for “usable” dimension need to consider correct representation of data from mobile devices in connected interface. Heuristic evaluation is often used for representation analysis. It is an easy to learn, easy to use, discounted usability evaluation method that involves expert review of usability violations against established usability principles [5]. Adapted TURF framework proposes to use the 12 principles including consistency, visibility, match, minimalist, memory, feedback, flexibility, message, error, closure, undo, language, control, and document [6]. Particular attention is to be paid to the consistency and match between mobile information and connected interface. 

    Task Analysis

    Task analysis for “usable” dimension will highlight learnability, efficiency (time on task, steps on task, task success, mental effort), and error prevention and recovery (occurrence rate, error recovery rate). A connected health care technology is considered as easy to learn if the number of trials to reach a certain performance level, number of items that need to memorized, number of sequences of steps that need to be memorized are all minimized.


    Discussion

    Principal Findings

    We are the first to propose and present a unified framework to guide the design and redesign of connected health care technologies. Future work should expand the framework in real world settings considering real world interruptions, team dynamics, and multitasking. Both subjective and objective dimensions of usability should be applied during development of connected health care technologies involving mobile data. A software called turf is guided by the original TURF framework to semi-automate the usability evaluation process and make it simple and straightforward for usability testing professionals. We anticipate that this software can also be used to support usability evaluation of connected health care technologies.

    Conclusions

    An adapted framework is proposed to offer objective, evidence-based, and systematic usability evaluation to guide the design and redesign of interfaces connecting mobile data with electronic health record systems and Web-based evaluation tools.

    Acknowledgments

    This project was funded by the Robert Wood Johnson Foundation Nurse Faculty Scholars Program and partly supported by the Grant No. 10510592 for Patient-Centered Cognitive Support under the Strategic Health IT Advanced Research Projects (SHARP) from the Office of the National Coordinator for Health Information Technology. We would like to thank our colleagues Muhammad Valji, Amy Franklin, Min Zhu, and students for their assistance and feedback.

    Conflicts of Interest

    None declared.

    References

    1. Kvedar J, Coye MJ, Everett W. Connected health: a review of technologies and strategies to improve patient care with telemedicine and telehealth. Health Aff (Millwood) 2014 Feb;33(2):194-199. [CrossRef] [Medline]
    2. Wang J, Sereika SM, Chasens ER, Ewing LJ, Matthews JT, Burke LE. Effect of adherence to self-monitoring of diet and physical activity on weight loss in a technology-supported behavioral intervention. Patient Prefer Adherence 2012;6:221-226 [FREE Full text] [CrossRef] [Medline]
    3. Burke LE, Wang J, Sevick MA. Self-monitoring in weight loss: a systematic review of the literature. J Am Diet Assoc 2011 Jan;111(1):92-102 [FREE Full text] [CrossRef] [Medline]
    4. Yen PY, Bakken S. Review of health information technology usability study methodologies. J Am Med Inform Assoc 2012;19(3):413-422 [FREE Full text] [CrossRef] [Medline]
    5. Zhang J, Walji MF. TURF: toward a unified framework of EHR usability. J Biomed Inform 2011 Dec;44(6):1056-1067 [FREE Full text] [CrossRef] [Medline]
    6. Zhang J, Johnson TR, Patel VL, Paige DL, Kubose T. Using usability heuristics to evaluate patient safety of medical devices. J Biomed Inform 2003;36(1-2):23-30 [FREE Full text] [Medline]

    Edited by G Eysenbach, T Hale; submitted 14.05.15; peer-reviewed by S Agboola, PY Yen, J Kvedar, S Bakken; accepted 20.07.15; published 27.10.15

    ©Jing Wang, Jiajie Zhang. Originally published in JMIR Mhealth and Uhealth (http://www.iproc.org), 27.10.2015.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.