Implicit Review Instrument to Evaluate Quality of Care Delivered by Physicians to Children in Emergency Departments

the Pediatric Emergency Care Applied Research Network

doi:10.1111/1475-6773.12800

Implicit Review Instrument to Evaluate Quality of Care Delivered by Physicians to Children in Emergency Departments

the Pediatric Emergency Care Applied Research Network

Pediatrics

Research output: Contribution to journal › Article › peer-review

6 Scopus citations

Abstract

Objective: To evaluate the consistency, reliability, and validity of an implicit review instrument that measures the quality of care provided to children in the emergency department (ED). Data Sources/Study Setting: Medical records of randomly selected children from 12 EDs in the Pediatric Emergency Care Applied Research Network (PECARN). Study Design: Eight pediatric emergency medicine physicians applied the instrument to 620 medical records. Data Collection/Extraction Methods: We determined internal consistency using Cronbach's alpha and inter-rater reliability using the intraclass correlation coefficient (ICC). We evaluated the validity of the instrument by correlating scores with four condition-specific explicit review instruments. Principal Findings: Individual reviewers' Cronbach's alpha had a mean of 0.85 with a range of 0.76–0.97; overall Cronbach's alpha was 0.90. The ICC was 0.49 for the summary score with a range from 0.40 to 0.46. Correlations between the quality of care score and the four condition-specific explicit review scores ranged from 0.24 to 0.38. Conclusions: The quality of care instrument demonstrated good internal consistency, moderate inter-rater reliability, high inter-rater agreement, and evidence supporting validity. The instrument could be useful for systems' assessment and research in evaluating the care delivered to children in the ED.

Original language	English (US)
Pages (from-to)	1316-1334
Number of pages	19
Journal	Health Services Research
Volume	53
Issue number	3
DOIs	https://doi.org/10.1111/1475-6773.12800
State	Published - Jun 2018

Funding

This project was also supported in part by the Health Resources and Services Administration (HRSA), Maternal and Child Health Bureau (MCHB), Emergency Medical Services for Children (EMSC) Network Development Demonstration Program under cooperative agreements U03MC00008, U03MC00001, U03MC00003, U03MC00006, U03MC00007, U03MC22684, and U03MC22685. This information or content and conclusions are those of the author and should not be construed as the official position or policy of, nor should any endorsements be inferred by HRSA, HHS, or the U.S. Government. The authors have no financial relationships relevant to this article to disclose. Disclosure: None. Disclaimer: None. Joint Acknowledgment/Disclosure Statement: We acknowledge the efforts of the following individuals participating in PECARN at the time this study was initiated: PECARN Steering Committee: N. Kuppermann, Chair; E. Alpern, D. Borgialli, J. Callahan, J. Chamberlain, P. Dayan, J. M. Dean, M. Gerardi, M. Gorelick, J. Hoyle, E. Jacobs, D. Jaffe, R. Lichenstein, K. Lillis, P. Mahajan, R. Maio, F. Moler, D. Monroe, R. Ruddy, R. Stanley, M. Tunik, A. Walker. MCHB/EMSC liaisons: D. Kavanaugh, H. Park. Data Coordinating Center (DCC): M. Dean, R. Holubkov, S. Knight, A. Donaldson, S. Zuspan, M. Miskin, J. Wade, A. Jones, M. Fjelstad. Feasibility and Budget Subcommittee (FABS): T. Singh, Chair; A. Drongowski, L. Fukushima, E. Kim, D. Monroe, G. O'Gara, H. Rincon, M. Tunik, S. Zuspan. Grants and Publications Subcommittee (GAPS): M. Gorelick, Chair; E. Alpern, D. Borgialli, K. Brown, L. Babcock, A. Donaldson, G. Foltin, F. Moler, S. Teach. Protocol Concept Review and Development Subcommittee (PCRADS): D. Jaffe, Chair; J. Chamberlain, A. Cooper, P. Dayan, J. M. Dean, R. Holubkov, P. Mahajan, R. Maio, N.C. Mann, K. Shaw, A. Walker. Quality Assurance Subcommittee (QAS): R. Stanley, Chair; P. Ehrlich, R. Enriquez, M. Gerardi, R. Holubkov, E. Jacobs, R. Lichenstein, K. Lillis, B. Millar, R. Ruddy, M. Shults. Safety and Regulatory Affairs Subcommittee (SRAS): W. Schalick, J. Callahan, Co-Chairs; S. Atabaki, J. Burr, K. Call, J. Hoyle, R. Ruddy, J. Suhajda, N. Schamban. Funding Sources: 1R01HS019712 Agency for Healthcare Quality and Research. This project was also supported in part by the Health Resources and Services Administration (HRSA), Maternal and Child Health Bureau (MCHB), Emergency Medical Services for Children (EMSC) Network Development Demonstration Program under cooperative agreements U03MC00008, U03MC00001, U03MC00003, U03MC00006, U03MC00007, U03MC22684, and U03MC22685. This information or content and conclusions are those of the author and should not be construed as the official position or policy of, nor should any endorsements be inferred by HRSA, HHS, or the U.S. Government. The authors have no financial relationships relevant to this article to disclose. Disclosure: None. Disclaimer: None.

Keywords

Pediatrics
emergency department
quality

ASJC Scopus subject areas

Health Policy

Access to Document

10.1111/1475-6773.12800

Cite this

@article{772923a06c944247b855a66efba31deb,

title = "Implicit Review Instrument to Evaluate Quality of Care Delivered by Physicians to Children in Emergency Departments",

abstract = "Objective: To evaluate the consistency, reliability, and validity of an implicit review instrument that measures the quality of care provided to children in the emergency department (ED). Data Sources/Study Setting: Medical records of randomly selected children from 12 EDs in the Pediatric Emergency Care Applied Research Network (PECARN). Study Design: Eight pediatric emergency medicine physicians applied the instrument to 620 medical records. Data Collection/Extraction Methods: We determined internal consistency using Cronbach's alpha and inter-rater reliability using the intraclass correlation coefficient (ICC). We evaluated the validity of the instrument by correlating scores with four condition-specific explicit review instruments. Principal Findings: Individual reviewers' Cronbach's alpha had a mean of 0.85 with a range of 0.76–0.97; overall Cronbach's alpha was 0.90. The ICC was 0.49 for the summary score with a range from 0.40 to 0.46. Correlations between the quality of care score and the four condition-specific explicit review scores ranged from 0.24 to 0.38. Conclusions: The quality of care instrument demonstrated good internal consistency, moderate inter-rater reliability, high inter-rater agreement, and evidence supporting validity. The instrument could be useful for systems' assessment and research in evaluating the care delivered to children in the ED.",

keywords = "Pediatrics, emergency department, quality",

author = "\{the Pediatric Emergency Care Applied Research Network\} and Marcin, \{James P.\} and Romano, \{Patrick S.\} and Madan Dharmar and Chamberlain, \{James M.\} and Nanette Dudley and Macias, \{Charles G.\} and Nigrovic, \{Lise E.\} and Powell, \{Elizabeth C.\} and Rogers, \{Alexander J.\} and Meridith Sonnett and Leah Tzimenatos and Alpern, \{Elizabeth R.\} and Rebecca Andrews-Dickert and Borgialli, \{Dominic A.\} and Erika Sidney and Charlie Casper and Dean, \{Jonathan Michael\} and Nathan Kuppermann and J. Callahan and P. Dayan and M. Gerardi and M. Gorelick and J. Hoyle and E. Jacobs and D. Jaffe and R. Lichenstein and K. Lillis and P. Mahajan and R. Maio and F. Moler and D. Monroe and R. Ruddy and R. Stanley and M. Tunik and A. Walker and D. Kavanaugh and H. Park and R. Holubkov and S. Knight and A. Donaldson and S. Zuspan and M. Miskin and J. Wade and A. Jones and M. Fjelstad and T. Singh and A. Drongowski and L. Fukushima and E. Kim and G. O'Gara",

note = "Publisher Copyright: {\textcopyright} Health Research and Educational Trust",

year = "2018",

month = jun,

doi = "10.1111/1475-6773.12800",

language = "English (US)",

volume = "53",

pages = "1316--1334",

journal = "Health Services Research",

issn = "0017-9124",

publisher = "John Wiley and Sons Inc.",

number = "3",

}

TY - JOUR

T1 - Implicit Review Instrument to Evaluate Quality of Care Delivered by Physicians to Children in Emergency Departments

AU - the Pediatric Emergency Care Applied Research Network

AU - Marcin, James P.

AU - Romano, Patrick S.

AU - Dharmar, Madan

AU - Chamberlain, James M.

AU - Dudley, Nanette

AU - Macias, Charles G.

AU - Nigrovic, Lise E.

AU - Powell, Elizabeth C.

AU - Rogers, Alexander J.

AU - Sonnett, Meridith

AU - Tzimenatos, Leah

AU - Alpern, Elizabeth R.

AU - Andrews-Dickert, Rebecca

AU - Borgialli, Dominic A.

AU - Sidney, Erika

AU - Casper, Charlie

AU - Dean, Jonathan Michael

AU - Kuppermann, Nathan

AU - Callahan, J.

AU - Dayan, P.

AU - Gerardi, M.

AU - Gorelick, M.

AU - Hoyle, J.

AU - Jacobs, E.

AU - Jaffe, D.

AU - Lichenstein, R.

AU - Lillis, K.

AU - Mahajan, P.

AU - Maio, R.

AU - Moler, F.

AU - Monroe, D.

AU - Ruddy, R.

AU - Stanley, R.

AU - Tunik, M.

AU - Walker, A.

AU - Kavanaugh, D.

AU - Park, H.

AU - Holubkov, R.

AU - Knight, S.

AU - Donaldson, A.

AU - Zuspan, S.

AU - Miskin, M.

AU - Wade, J.

AU - Jones, A.

AU - Fjelstad, M.

AU - Singh, T.

AU - Drongowski, A.

AU - Fukushima, L.

AU - Kim, E.

AU - O'Gara, G.

PY - 2018/6

Y1 - 2018/6

N2 - Objective: To evaluate the consistency, reliability, and validity of an implicit review instrument that measures the quality of care provided to children in the emergency department (ED). Data Sources/Study Setting: Medical records of randomly selected children from 12 EDs in the Pediatric Emergency Care Applied Research Network (PECARN). Study Design: Eight pediatric emergency medicine physicians applied the instrument to 620 medical records. Data Collection/Extraction Methods: We determined internal consistency using Cronbach's alpha and inter-rater reliability using the intraclass correlation coefficient (ICC). We evaluated the validity of the instrument by correlating scores with four condition-specific explicit review instruments. Principal Findings: Individual reviewers' Cronbach's alpha had a mean of 0.85 with a range of 0.76–0.97; overall Cronbach's alpha was 0.90. The ICC was 0.49 for the summary score with a range from 0.40 to 0.46. Correlations between the quality of care score and the four condition-specific explicit review scores ranged from 0.24 to 0.38. Conclusions: The quality of care instrument demonstrated good internal consistency, moderate inter-rater reliability, high inter-rater agreement, and evidence supporting validity. The instrument could be useful for systems' assessment and research in evaluating the care delivered to children in the ED.

AB - Objective: To evaluate the consistency, reliability, and validity of an implicit review instrument that measures the quality of care provided to children in the emergency department (ED). Data Sources/Study Setting: Medical records of randomly selected children from 12 EDs in the Pediatric Emergency Care Applied Research Network (PECARN). Study Design: Eight pediatric emergency medicine physicians applied the instrument to 620 medical records. Data Collection/Extraction Methods: We determined internal consistency using Cronbach's alpha and inter-rater reliability using the intraclass correlation coefficient (ICC). We evaluated the validity of the instrument by correlating scores with four condition-specific explicit review instruments. Principal Findings: Individual reviewers' Cronbach's alpha had a mean of 0.85 with a range of 0.76–0.97; overall Cronbach's alpha was 0.90. The ICC was 0.49 for the summary score with a range from 0.40 to 0.46. Correlations between the quality of care score and the four condition-specific explicit review scores ranged from 0.24 to 0.38. Conclusions: The quality of care instrument demonstrated good internal consistency, moderate inter-rater reliability, high inter-rater agreement, and evidence supporting validity. The instrument could be useful for systems' assessment and research in evaluating the care delivered to children in the ED.

KW - Pediatrics

KW - emergency department

KW - quality

UR - http://www.scopus.com/inward/record.url?scp=85034217139&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85034217139&partnerID=8YFLogxK

U2 - 10.1111/1475-6773.12800

DO - 10.1111/1475-6773.12800

M3 - Article

C2 - 29143331

AN - SCOPUS:85034217139

SN - 0017-9124

VL - 53

SP - 1316

EP - 1334

JO - Health Services Research

JF - Health Services Research

IS - 3

ER -

Implicit Review Instrument to Evaluate Quality of Care Delivered by Physicians to Children in Emergency Departments

Abstract

Funding

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this