Methodological approaches and author-reported limitations in evaluation studies of digital health technologies (DHT): A scoping review of DHT interventions for cancer, diabetes mellitus, and cardiovascular diseases

Gityamwi, Nyangi ORCID: https://orcid.org/0000-0002-5567-9300, Armes, Jo, Harris, Jenny, Ream, Emma, Green, Richard, Ahankari, Anand ORCID: https://orcid.org/0000-0001-8695-8938, Callwood, Alison, Ip, Athena, Cockle-Hearne, Jane, Grosvenor, Wendy, Lemanska, Agnieszka and Skene, Simon S ORCID: https://orcid.org/0000-0002-7828-3122 (2025) Methodological approaches and author-reported limitations in evaluation studies of digital health technologies (DHT): A scoping review of DHT interventions for cancer, diabetes mellitus, and cardiovascular diseases. PLOS Digital Health, 4 (4). e0000806.

Preview

Published Version Available under License Creative Commons Attribution. Download (1MB) | Preview

Abstract

Digital health technology (DHT) holds the potential to improve health services, and its adoption has proliferated in recent decades owing to technological advancement. Optimal evaluation methodologies appropriate for generating quality evidence on DHT have yet to be established; traditional comparative designs present several limitations. This study aimed to scope the literature to highlight common methodological approaches used and their limitations to inform considerations for designing robust DHT evaluation studies. A scoping review was conducted following the Joanna Briggs Institute (JBI) scoping review guidelines. A systematic search was conducted using the CINAHL (EBSCO), MEDLINE (EBSCOhost), PsycINFO (EBSCO), EMBASE (Elsevier) and Web of Science (Clarivate Analytics) databases using iteratively developed search terms. We selected studies published in English between January 2016 and March 2022 and focussed on primary research evaluating the effectiveness of DHT with technology-user interactive or asynchronous features for adults (≥18 years) with cancer, diabetes or cardiovascular conditions. The final number of articles, after the screening and selection process, comprised 140 records. Data were analysed descriptively (frequency and percentages) and summarised thematically. Results showed most studies (n = 104, 74.3%) employed the standard two-arm parallel RCT design, with usual/standard care as the preferred comparator in nearly half (n = 65, 47.1%) of all included studies. Of the 104 comparative studies reviewed, limitations in recruitment were most frequently reported (n = 70, 37%), followed by limitations in evaluation/measurement techniques (n = 57, 27%), presence of confounding factors (n = 50, 24%) and short duration of studies (n = 24, 11%). The review highlights the need to consider inclusive approaches to recruitment and adoption of the emerging methodological approaches that account for the fast-paced, multi-component and group contamination problem resulting from the unconcealable nature of DHT interventions.