Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, March 11, 2019 at 4:00 PM to 4:30 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?


Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 30.12.16 in Vol 2, No 1 (2016): December

Preprints (earlier versions) of this paper are available at, first published Jun 20, 2016.

This paper is in the following e-collection/theme issue:


    The Impact of Electronic Administration of the MMSE on Error Rates and Data Quality

    Bracket Global, Wayne, PA, United States

    Corresponding Author:

    Dan DeBonis

    Bracket Global

    575 E Swedesford Rd

    Wayne, PA, 19087

    United States

    Phone: 1 617 285 4342

    Fax:1 617 285 4342



    Background: The Mini-Mental State Examination (MMSE) is a widely used measure of cognition in clinical practice and is a frequently used screening and outcome measure for cognition-enhancing clinical drug trials for Alzheimer’s Disease. Scoring and administration errors on the MMSE in a clinical drug trial can contribute to placebo response and unreliable data. Examples of such errors range from basic addition errors to failure to adhere to structured administration instructions. Prior research from independent clinical review of paper versions of the scale as part of a data quality monitoring program in worldwide studies has demonstrated that MMSE error rates among clinical trial raters tend to be upwards of 30% on initial administration. An enhanced tablet-based electronic (eCOA) version of the MMSE was developed for clinical trials by Bracket and has been utilized in worldwide clinical trials since 2013. The scale is administered in a system compliant with US Food and Drug Administration guidelines on the use of electronic medical records in trials and was designed to be equivalent to the paper MMSE. The eCOA version was augmented with internal logic, automated scoring and addition, standardized instructions, and enforcement of administration conventions. These enhancements were intended to decrease errors and increase the likelihood that the scale would be consistently administered and scored according to standard conventions.

    Objective: Our objectives were to (1) determine if error rates using the eCOA MMSE were reduced when compared to standard paper administration and comparable to eCOA error rates from an analysis published in 2015 and (2) consider the impact of utilizing enhanced eCOA versions of scales like the MMSE in clinical practice.

    Methods: All data from this analysis and comparative analyses were from multinational studies that trained and certified raters on the MMSE and included an in-study data monitoring program of clinical reviews to detect scoring and administration errors. Error rates on the enhanced electronic versions of the scales used in a late-phase Alzheimer’s study were compared to those previously seen in other studies when using paper versions and the enhanced eCOA MMSE. The dataset for this eCOA analysis was limited to English-speaking sites.

    Results: A total of 66/626 (10.5%) of the enhanced eCOA MMSEs reviewed at the inclusion visits contained an error in scoring which required outreach to the rater. This represents a statistically significant reduction (P<.0001) in the error rates compared to paper administration of the MMSE (877/2889, 30.4%) and is consistent with the findings from a prior analysis of an eCOA MMSE dataset from another trial (107/990,10.8%).

    Conclusions: Use of enhanced eCOA for the MMSE in clinical trials has significantly improved error rates and data quality compared to standard paper administration. Enhanced eCOA that is designed to minimize errors and enforce administration standards could be a useful tool to reduce common errors and increase the precision of the MMSE use in clinical practice.

    iproc 2016;2(1):e46



    This poster was presented at the Connected Health Symposium 2016, October 20-21, Boston, MA, United States. The poster is displayed as an image in Figure 1 and as a PDF in Multimedia Appendix 1.

    Figure 1. Poster.
    View this figure

    Multimedia Appendix 1


    PDF File (Adobe PDF File), 854KB

    Edited by T Hale; submitted 20.06.16; peer-reviewed by CHS Scientific Program Committee; accepted 02.08.16; published 30.12.16

    ©Dan DeBonis, Todd Solomon, David Miller. Originally published in Iproceedings (, 30.12.2016.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in Iproceedings, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.