Smart-home consumer technologies have been criticized for failing to disclose their operational performance characteristics to the marketplace. As one result, some users of wearable fitness technologies have reported being frustrated by invalid motivational responses based on fluctuations in accurate performance measurement by certain brands. Gerontechnology researchers have similarly documented the critical importance of valid operations and technical stability as major influences on whether older adults and their caregivers adopt and use new cognitive assistive technologies. We have been iteratively developing the DRESS (Development of a Responsive Emotive Sensing System) system, integrating context aware computing with effective sensor and interactive technologies, to customize coaching persons with dementia to dress independently. Our prior testing focused on components and clothing identification, not the overall system performance. Consequently, we initiated system testing, as part of our alpha version development phase, to assess key metrics and disclose the performance outcomes.

To assess the operational accuracy (validity) and stability (reliability) of the DRESS system alpha prototype model.

We conducted a 110 day device trial run-in study. The system operated 24/7 in a studio-sized testing unit using the local WiFi network. A 69-year-old tester documented any usability issues during this period. Automatic log reports were generated daily by the system and validated and annotated by the project manager. A content analysis of the user and log reports was conducted, and descriptive statistics were used to describe the operational findings.

The system functioned error free for the majority of the trial (75% of days) with stable performance for 95.5% of days. Thirty-seven correctable error events occurred during 28 of the 110 days and resulted in 4 categories of errors: Hardware (0.9%), from a defective IPad charger; Network (3.6%), from host network disconnects/power outage; Usability (4.5%), from the visual displays/buttons on the caregivers’ device being too small in size; and Re-initialization (24.5%), from the operating system/Indigo software updates.

Overall, the system performed very favorably for an alpha prototype. As expected, the initial deployment required an immediate debugging period primarily rectified by software recoding. Notably no fatal or irresolvable errors occurred. The system remained stable except for a disconnect due to a weather-related regional power outage. Lessons learned, such as integrating a remote automatic reboot capability, will be used to further optimize system performance before advancing to an in-home study with persons experiencing dementia.