A process for reviewing mental health apps: Using the One Mind PsyberGuide Credibility Rating System

Martha Neary*, John Bunyi, Kristina Palomares, David C. Mohr, Adam Powell, Josef Ruzek, Leanne M. Williams, Til Wykes, Stephen Matthew Schueller

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Objective: Given the increasing number of publicly available mental health apps, we need independent advice to guide adoption. This paper discusses the challenges and opportunities of current mental health app rating systems and describes the refinement process of one prominent system, the One Mind PsyberGuide Credibility Rating Scale (PGCRS). Methods: PGCRS Version 1 was developed in 2013 and deployed for 7 years, during which time a number of limitations were identified. Version 2 was created through multiple stages, including a review of evaluation guidelines and consumer research, input from scientific experts, testing, and evaluation of face validity. We then re-reviewed 161 mental health apps using the updated rating scale, investigated the reliability and discrepancy of initial scores, and updated ratings on the One Mind PsyberGuide public app guide. Results: Reliabilities across the scale's 9 items ranged from −0.10 to 1.00, demonstrating that some characteristics of apps are more difficult to rate consistently. The average overall score of the 161 reviewed mental health apps was 2.51/5.00 (range 0.33–5.00). Ratings were not strongly correlated with app store star ratings, suggesting that credibility scores provide different information to what is contained in star ratings. Conclusion: PGCRS summarizes and weights available information in 4 domains: intervention specificity, consumer ratings, research, and development. Final scores are created through an iterative process of initial rating and consensus review. The process of updating this rating scale and integrating it into a procedure for evaluating apps demonstrates one method for determining app quality.

Original languageEnglish (US)
JournalDigital Health
Volume7
DOIs
StatePublished - 2021

Keywords

  • digital mental health
  • evaluation
  • mental health
  • mHealth
  • mobile health

ASJC Scopus subject areas

  • Health Policy
  • Health Informatics
  • Computer Science Applications
  • Health Information Management

Fingerprint

Dive into the research topics of 'A process for reviewing mental health apps: Using the One Mind PsyberGuide Credibility Rating System'. Together they form a unique fingerprint.

Cite this