This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.
Background
Evidence-based, objective, and systematic usability evaluation is key to successful connected health care technologies. The increases in patient-facing mobile health technologies not only offer convenience for patients in managing their own health and/or chronic conditions, but also offer the opportunity for health care providers to access patient behaviors and patient-centered outcomes at ease. Thus, it is of significance to link and present patient-facing mobile device data to their health care providers in a secure and uninterrupted way that will facilitate workflow and promote patient provider communication, rather than drawing providers away from patients. This prompts increasing efforts developing connected health care technologies linking mobile data to electronic health record systems guided by user-centered design and redesign principles. However, lack of scientific, objective, and systematic usability evaluation put connected health care technologies at risk for low adoption and eventual failure.
Objective
Learning from lessons in electronic health record usability evaluation, we propose to adapt an existing unified framework, TURF, for electronic health record usability evaluation to guide the design, redesign, and usability evaluation of connected health care technologies linking mobile data to electronic health record systems or other provider-facing Web-based evaluation tools.
Methods
TURF, a unified framework of electronic health record usability, involves three dimensions: useful, usable, and satisfying; and four key components: task, user, representation, and function. Each dimension and component is described with theoretical underpinnings along with examples of how usability can be measured.
Results
Specific adaptation of TURF that’s unique for connected health technologies include (1) user analysis for “satisfying” dimension will need to include both users using and mobile health users who’s feeding data into the system; (2) function analysis for “useful” dimension will need to consider functions/data wanted by the providers, functions actually used in real activities, functions/data available from mobile devices and with agreement from patients, functions/data available in interfaces within connected health care technologies ; (3) representation analysis for “usable” dimension need to consider correct representation of data from mobile devices in connected interface; (4) task analysis for “usable” dimension will highlight learnability, efficiency (time on task, steps on task, task success, mental effort), and error prevention and recovery (occurrence rate, error recovery rate). Real world interruptions, team dynamics, and multitasking should also be considered during evaluation of connected health care technologies.
Conclusions
An adapted framework is proposed to offer objective, evidence-based, and systematic usability evaluation to guide the design and redesign of interfaces connecting mobile data with electronic health record systems and Web-based evaluation tools.
mobile healthconnected healthelectronic health records, usability evaluationIntroduction
Connected health care technologies have significant values to both health care professional and patients [1]. Patient-facing technology tools including smartphone apps, wearable activity trackers, bluetooth enabled glucometers, wireless weight scales have significantly increased in the market and gained popular demands with potential to facilitate patient management of their own health. An example of these mobile tools are the use of electronic diaries for patient self-monitoring of diet and physical activity for both fitness purposes and can also be for chronic disease self-management. Self-monitoring is cornerstone of a successful behavioral lifestyle intervention for obesity and diabetes [2,3]. Mobile-based electronic diaries made self-monitoring easier and convenient for individuals. Accessing such mobile collected self-monitoring information can provide valuable information for their providers not only in setting individualized and realistic behavioral goals, but also in following up with these behavioral goals in an efficient and effective way that can help patient achieve successful behavior change and ultimately lead to better health care outcomes. Thus, it is of significance to link and present patient-facing mobile device data to their health care providers in a secure and uninterrupted way that will facilitate workflow and promote patient provider communication, rather than drawing providers away from patients. This prompts increasing efforts developing connected health care technologies linking mobile data to electronic health record systems guided by user-centered design and redesign principles. However, lack of scientific, objective, and systematic usability evaluation put connected health care technologies at risk for low adoption and eventual system failure.
With lessons learned in electronic health record implementation across the United States, evidence-based, objective, and systematic usability evaluation is key to successful connected health care technologies. A review in health care information technology usability studies also pointed out that a theoretical framework to guide usability evaluation is essential [4]. Therefore, we propose to adapt an existing unified framework -TURF- for electronic health record usability evaluation to guide the design, redesign, and usability evaluation of connected health care technologies linking mobile data to electronic health record systems or other provider-facing Web-based evaluation tools. In summary, this paper will present the adapted TURF framework as a coherent, unified framework to guide objective and systematic usability evaluation [5].
Methods
TURF, standing for “task, user, representation, and function,” is a unified framework of EHR usability, that is “(1) a theory for describing, explaining, and predicting usability differences; (2) a method for defining, evaluating, and measuring usability objectively; (3) a process for designing built-in good usability; and (4) once fully developed, a potential principle for developing EHR usability guidelines and standards.”5 TURF defines usability as “how useful, usable, and satisfying a system is for the intended users to accomplish goals in the work domain by performing certain sequences of tasks” [5].
TURF involves three dimensions: useful, usable, and satisfying; and four key components: task, user, representation, and function. Each dimension and component is described with theoretical underpinnings along with examples of how usability can be measured. The relationships between each dimension and components are depicted in Figure 1.
The original TURF framework for Electronic Health Record Usability (from Zhang, 2011)[5].
ResultsAdapted TURF for Connected Health Technologies
Under adapted TURF, usability for connected health care technologies involving mobile data is defined as useful, usable, and satisfying for intended connected interface users to accomplish goals in the work domain by performing certain sequences of tasks, as well as useful, usable, and satisfying for mobile users feeding data into the connected health care technologies.
User satisfaction is often measured by survey questions evaluating users’ perceptions or ratings on a scale. It is an important step, but it often gives individuals wrong impression that usability is subjective, unreliable, and useless for system design and redesign. TURF has both subjective and objective measures of usability. Both measures of usability will have to be conducted to give a complete picture of usability.
We presented each dimension and measure of usability under adapted TURF framework in Table 1.
Usability dimensions and measures of usability under adapted TURF for connected health care technologies.
Dimensions
Descriptions
Representative measures
Useful
A connected health care technology is useful if it supports the work domain where the users accomplish goals for their work, independent of how the system is implemented.
Across-model domain function saturation: % of domain functions in the system vs all domain functions in the work domainWithin-model Domain Function Saturation: & of domain functions overall all functions
Usable
A connected health care technology is usable if it is easy to learn, easy to use, and error-tolerant on the connected interface, and with some features relying on mobile technology users to feed data/information into the connected interface.
Learnability; Efficiency; Error Prevention and Recovery
Satisfying
A connected health care technology is satisfying to use if the users have good subjective impression of how useful, usable, and likable the system is for both mobile technology users and connected interface users.
Ratings through survey, interviews, and other instruments
User Analysis
User analysis for “satisfying” dimension will need to include both users using and mobile health users who are feeding data into the system. User analysis is the first step of usability evaluation; it involves steps of identifying the types of users and characteristics of users that are using the connected interfaces (often being key health care professionals) as well as mobile technology users (often being patients or supporting personnel).
Functional Analysis
Function analysis for “useful” dimension will need to consider functions or data wanted by the providers, functions actually used in real activities, functions or data available from mobile devices and with agreement from patients, functions or data available in interfaces within connected health care technologies
Representational Analysis
Representation analysis for “usable” dimension need to consider correct representation of data from mobile devices in connected interface. Heuristic evaluation is often used for representation analysis. It is an easy to learn, easy to use, discounted usability evaluation method that involves expert review of usability violations against established usability principles [5]. Adapted TURF framework proposes to use the 12 principles including consistency, visibility, match, minimalist, memory, feedback, flexibility, message, error, closure, undo, language, control, and document [6]. Particular attention is to be paid to the consistency and match between mobile information and connected interface.
Task Analysis
Task analysis for “usable” dimension will highlight learnability, efficiency (time on task, steps on task, task success, mental effort), and error prevention and recovery (occurrence rate, error recovery rate). A connected health care technology is considered as easy to learn if the number of trials to reach a certain performance level, number of items that need to memorized, number of sequences of steps that need to be memorized are all minimized.
DiscussionPrincipal Findings
We are the first to propose and present a unified framework to guide the design and redesign of connected health care technologies. Future work should expand the framework in real world settings considering real world interruptions, team dynamics, and multitasking. Both subjective and objective dimensions of usability should be applied during development of connected health care technologies involving mobile data. A software called turf is guided by the original TURF framework to semi-automate the usability evaluation process and make it simple and straightforward for usability testing professionals. We anticipate that this software can also be used to support usability evaluation of connected health care technologies.
Conclusions
An adapted framework is proposed to offer objective, evidence-based, and systematic usability evaluation to guide the design and redesign of interfaces connecting mobile data with electronic health record systems and Web-based evaluation tools.
This project was funded by the Robert Wood Johnson Foundation Nurse Faculty Scholars Program and partly supported by the Grant No. 10510592 for Patient-Centered Cognitive Support under the Strategic Health IT Advanced Research Projects (SHARP) from the Office of the National Coordinator for Health Information Technology. We would like to thank our colleagues Muhammad Valji, Amy Franklin, Min Zhu, and students for their assistance and feedback.
None declared.
KvedarJosephCoyeMolly JoelEverettWendyConnected health: a review of technologies and strategies to improve patient care with telemedicine and telehealthHealth Aff (Millwood)201402332194910.1377/hlthaff.2013.09922449376033/2/194WangJingSereikaSusan MChasensEileen REwingLinda JMatthewsJudith TBurkeLora EEffect of adherence to self-monitoring of diet and physical activity on weight loss in a technology-supported behavioral interventionPatient Prefer Adherence20126221610.2147/PPA.S2888922536058ppa-6-221PMC3333811BurkeLora EWangJingSevickMary AnnSelf-monitoring in weight loss: a systematic review of the literatureJ Am Diet Assoc20110111119210210.1016/j.jada.2010.10.00821185970S0002-8223(10)01644-5PMC3268700YenPo-YinBakkenSuzanneReview of health information technology usability study methodologiesJ Am Med Inform Assoc20121934132210.1136/amiajnl-2010-00002021828224amiajnl-2010-000020PMC3341772ZhangJiajieWaljiMuhammad FTURF: toward a unified framework of EHR usabilityJ Biomed Inform20111244610566710.1016/j.jbi.2011.08.00521867774S1532-0464(11)00132-8ZhangJiajieJohnsonTodd RPatelVimla LPaigeDanielle LKuboseTateUsing usability heuristics to evaluate patient safety of medical devicesJ Biomed Inform2003361-2233014552844S1532046403000601