How pragmatic is it? Lessons learned using PRECIS and RE-AIM for determining pragmatic characteristics of research

Bridget Gaglio*, Siobhan M. Phillips, Suzanne Heurtin-Roberts, Michael A. Sanchez, Russell E. Glasgow

*Corresponding author for this work

Research output: Contribution to journalArticle

44 Citations (Scopus)

Abstract

Background: The need for high-quality evidence that is applicable in real-world, routine settings continues to increase. Pragmatic trials are designed to evaluate the effectiveness of interventions in real-world settings, whereas explanatory trials aim to test whether an intervention works under optimal situations. There is a continuum between explanatory and pragmatic trials. Most trials have aspects of both, making it challenging to label and categorize a trial and to evaluate its potential for translation into practice. Methods: We summarize our experience applying the Pragmatic-Explanatory Continuum Indicator Summary (PRECIS) combined with external validity items based on the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework to three studies to provide a more robust and comprehensive assessment of trial characteristics related to translation of research. We summarize lessons learned using domains from the combined frameworks for use in study planning, evaluating specific studies, and reviewing the literature and make recommendations for future use. Results: A variety of coders can be trained to use the PRECIS and RE-AIM domains. These domains can also be used for diverse purposes, content areas, and study types, but are not without challenges. Both PRECIS and RE-AIM domains required modification in two of the three studies to evaluate and rate domains specific to study type. Lessons learned involved: dedicating enough time for training activities related to the domains; use of reviewers with a range of familiarity with specific study protocols; how to best adapt ratings that reflect complex study designs; and differences of opinion regarding the value of creating a composite score for these criteria. Conclusions: Combining both frameworks can specifically help identify where and how a study is and is not pragmatic. Using both PRECIS and RE-AIM allows for standard reporting of key study characteristics related to pragmatism and translation. Such measures should be used more consistently to help plan more pragmatic studies, evaluate progress, increase transparency of reporting, and integrate literature to facilitate translation of research into practice and policy.

Original languageEnglish (US)
Article number96
JournalImplementation Science
Volume9
Issue number1
DOIs
StatePublished - Jan 1 2014

Fingerprint

Pragmatic Clinical Trials
Maintenance
Research
Teaching
Practice (Psychology)

Keywords

  • External validity
  • PRECIS
  • Pragmatic trials
  • RE-AIM
  • Research translation

ASJC Scopus subject areas

  • Health Policy
  • Health Informatics
  • Public Health, Environmental and Occupational Health

Cite this

Gaglio, Bridget ; Phillips, Siobhan M. ; Heurtin-Roberts, Suzanne ; Sanchez, Michael A. ; Glasgow, Russell E. / How pragmatic is it? Lessons learned using PRECIS and RE-AIM for determining pragmatic characteristics of research. In: Implementation Science. 2014 ; Vol. 9, No. 1.
@article{b5834abe3627400e8b12045b8abc4a3a,
title = "How pragmatic is it? Lessons learned using PRECIS and RE-AIM for determining pragmatic characteristics of research",
abstract = "Background: The need for high-quality evidence that is applicable in real-world, routine settings continues to increase. Pragmatic trials are designed to evaluate the effectiveness of interventions in real-world settings, whereas explanatory trials aim to test whether an intervention works under optimal situations. There is a continuum between explanatory and pragmatic trials. Most trials have aspects of both, making it challenging to label and categorize a trial and to evaluate its potential for translation into practice. Methods: We summarize our experience applying the Pragmatic-Explanatory Continuum Indicator Summary (PRECIS) combined with external validity items based on the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework to three studies to provide a more robust and comprehensive assessment of trial characteristics related to translation of research. We summarize lessons learned using domains from the combined frameworks for use in study planning, evaluating specific studies, and reviewing the literature and make recommendations for future use. Results: A variety of coders can be trained to use the PRECIS and RE-AIM domains. These domains can also be used for diverse purposes, content areas, and study types, but are not without challenges. Both PRECIS and RE-AIM domains required modification in two of the three studies to evaluate and rate domains specific to study type. Lessons learned involved: dedicating enough time for training activities related to the domains; use of reviewers with a range of familiarity with specific study protocols; how to best adapt ratings that reflect complex study designs; and differences of opinion regarding the value of creating a composite score for these criteria. Conclusions: Combining both frameworks can specifically help identify where and how a study is and is not pragmatic. Using both PRECIS and RE-AIM allows for standard reporting of key study characteristics related to pragmatism and translation. Such measures should be used more consistently to help plan more pragmatic studies, evaluate progress, increase transparency of reporting, and integrate literature to facilitate translation of research into practice and policy.",
keywords = "External validity, PRECIS, Pragmatic trials, RE-AIM, Research translation",
author = "Bridget Gaglio and Phillips, {Siobhan M.} and Suzanne Heurtin-Roberts and Sanchez, {Michael A.} and Glasgow, {Russell E.}",
year = "2014",
month = "1",
day = "1",
doi = "10.1186/s13012-014-0096-x",
language = "English (US)",
volume = "9",
journal = "Implementation Science",
issn = "1748-5908",
publisher = "BioMed Central",
number = "1",

}

How pragmatic is it? Lessons learned using PRECIS and RE-AIM for determining pragmatic characteristics of research. / Gaglio, Bridget; Phillips, Siobhan M.; Heurtin-Roberts, Suzanne; Sanchez, Michael A.; Glasgow, Russell E.

In: Implementation Science, Vol. 9, No. 1, 96, 01.01.2014.

Research output: Contribution to journalArticle

TY - JOUR

T1 - How pragmatic is it? Lessons learned using PRECIS and RE-AIM for determining pragmatic characteristics of research

AU - Gaglio, Bridget

AU - Phillips, Siobhan M.

AU - Heurtin-Roberts, Suzanne

AU - Sanchez, Michael A.

AU - Glasgow, Russell E.

PY - 2014/1/1

Y1 - 2014/1/1

N2 - Background: The need for high-quality evidence that is applicable in real-world, routine settings continues to increase. Pragmatic trials are designed to evaluate the effectiveness of interventions in real-world settings, whereas explanatory trials aim to test whether an intervention works under optimal situations. There is a continuum between explanatory and pragmatic trials. Most trials have aspects of both, making it challenging to label and categorize a trial and to evaluate its potential for translation into practice. Methods: We summarize our experience applying the Pragmatic-Explanatory Continuum Indicator Summary (PRECIS) combined with external validity items based on the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework to three studies to provide a more robust and comprehensive assessment of trial characteristics related to translation of research. We summarize lessons learned using domains from the combined frameworks for use in study planning, evaluating specific studies, and reviewing the literature and make recommendations for future use. Results: A variety of coders can be trained to use the PRECIS and RE-AIM domains. These domains can also be used for diverse purposes, content areas, and study types, but are not without challenges. Both PRECIS and RE-AIM domains required modification in two of the three studies to evaluate and rate domains specific to study type. Lessons learned involved: dedicating enough time for training activities related to the domains; use of reviewers with a range of familiarity with specific study protocols; how to best adapt ratings that reflect complex study designs; and differences of opinion regarding the value of creating a composite score for these criteria. Conclusions: Combining both frameworks can specifically help identify where and how a study is and is not pragmatic. Using both PRECIS and RE-AIM allows for standard reporting of key study characteristics related to pragmatism and translation. Such measures should be used more consistently to help plan more pragmatic studies, evaluate progress, increase transparency of reporting, and integrate literature to facilitate translation of research into practice and policy.

AB - Background: The need for high-quality evidence that is applicable in real-world, routine settings continues to increase. Pragmatic trials are designed to evaluate the effectiveness of interventions in real-world settings, whereas explanatory trials aim to test whether an intervention works under optimal situations. There is a continuum between explanatory and pragmatic trials. Most trials have aspects of both, making it challenging to label and categorize a trial and to evaluate its potential for translation into practice. Methods: We summarize our experience applying the Pragmatic-Explanatory Continuum Indicator Summary (PRECIS) combined with external validity items based on the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework to three studies to provide a more robust and comprehensive assessment of trial characteristics related to translation of research. We summarize lessons learned using domains from the combined frameworks for use in study planning, evaluating specific studies, and reviewing the literature and make recommendations for future use. Results: A variety of coders can be trained to use the PRECIS and RE-AIM domains. These domains can also be used for diverse purposes, content areas, and study types, but are not without challenges. Both PRECIS and RE-AIM domains required modification in two of the three studies to evaluate and rate domains specific to study type. Lessons learned involved: dedicating enough time for training activities related to the domains; use of reviewers with a range of familiarity with specific study protocols; how to best adapt ratings that reflect complex study designs; and differences of opinion regarding the value of creating a composite score for these criteria. Conclusions: Combining both frameworks can specifically help identify where and how a study is and is not pragmatic. Using both PRECIS and RE-AIM allows for standard reporting of key study characteristics related to pragmatism and translation. Such measures should be used more consistently to help plan more pragmatic studies, evaluate progress, increase transparency of reporting, and integrate literature to facilitate translation of research into practice and policy.

KW - External validity

KW - PRECIS

KW - Pragmatic trials

KW - RE-AIM

KW - Research translation

UR - http://www.scopus.com/inward/record.url?scp=84908464324&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84908464324&partnerID=8YFLogxK

U2 - 10.1186/s13012-014-0096-x

DO - 10.1186/s13012-014-0096-x

M3 - Article

C2 - 25163664

AN - SCOPUS:84908464324

VL - 9

JO - Implementation Science

JF - Implementation Science

SN - 1748-5908

IS - 1

M1 - 96

ER -