The Quality Assurance and Quality Control Protocol for Neuropsychological Data Collection and Curation in the Ontario Neurodegenerative Disease Research Initiative (ONDRI) Study

Paula M. McLaughlin; Kelly M. Sunderland; Derek Beaton; Malcolm A. Binns; Donna Kwan; Brian Levine; Joseph B. Orange; Alicia J. Peltsch; Angela C. Roberts; Stephen C. Strother; Angela K. Troyer

doi:10.1177/1073191120913933

The Quality Assurance and Quality Control Protocol for Neuropsychological Data Collection and Curation in the Ontario Neurodegenerative Disease Research Initiative (ONDRI) Study

Paula M. McLaughlin^*, Kelly M. Sunderland, Derek Beaton, Malcolm A. Binns, Donna Kwan, Brian Levine, Joseph B. Orange, Alicia J. Peltsch, Angela C. Roberts, Stephen C. Strother, Angela K. Troyer

^*Corresponding author for this work

Communication Sciences and Disorders

Research output: Contribution to journal › Article › peer-review

25 Scopus citations

Abstract

As large research initiatives designed to generate big data on clinical cohorts become more common, there is an increasing need to establish standard quality assurance (QA; preventing errors) and quality control (QC; identifying and correcting errors) procedures for critical outcome measures. The present article describes the QA and QC approach developed and implemented for the neuropsychology data collected as part of the Ontario Neurodegenerative Disease Research Initiative study. We report on the efficacy of our approach and provide data quality metrics. Our findings demonstrate that even with a comprehensive QA protocol, the proportion of data errors still can be high. Additionally, we show that several widely used neuropsychological measures are particularly susceptible to error. These findings highlight the need for large research programs to put into place active, comprehensive, and separate QA and QC procedures before, during, and after protocol deployment. Detailed recommendations and considerations for future studies are provided.

Original language	English (US)
Pages (from-to)	1267-1286
Number of pages	20
Journal	Assessment
Volume	28
Issue number	5
DOIs	https://doi.org/10.1177/1073191120913933
State	Published - Jul 2021

Funding

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: J. Orange is the Scientific Director of the Canadian Centre for Activity and Aging, and he curently holds grants from the Social Sciences and Humanities Research Council, the Canadian Centre for Aging and Brain Health Innovation, the Ontario Brain Institute and the Canadian Consortium on Neurodegeneration in Aging. S. Strother is the Chief Scientific Officer of ADMdx, Inc., which receives NIH funding, and he currently has research grants from Brain Canada, Canada Foundation for Innovation (CFI), Canadian Institutes of Health Research (CIHR), and the Ontario Brain Institute in Canada. We like to thank Kristen Lutz for her assistance with the electronic case report form design in REDCap, as well as the Indoc research team for their data management support. This work was completed on behalf of the Ontario Neurodegenerative Disease Research Initiative (ONDRI). The authors would like to acknowledge the ONDRI Founding Authors: Robert Bartha, Sandra E. Black, Michael Borrie, Dale Corbett, Elizabeth Finger, Morris Freedman, Barry Greenberg, David A. Grimes, Robert A. Hegele, Chris Hudson, Anthony E. Lang, Mario Masellis, William E. McIlroy, Paula M. McLaughlin, Manuel Montero-Odasso, David G. Munoz, Douglas P. Munoz, J. B. Orange, Michael J. Strong, Stephen C. Strother, Richard H. Swartz, Sean Symons, Maria Carmela Tartaglia, Angela Troyer, and Lorne Zinman. The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was conducted with the support of the Ontario Brain Institute, an independent nonprofit corporation, funded partially by the Ontario government. Matching funds were provided by participant hospital and research institute foundations, including the Baycrest Foundation, Bruy?re Research Institute, Centre for Addiction and Mental Health Foundation, London Health Sciences Foundation, McMaster University Faculty of Health Sciences, Ottawa Brain and Mind Research Institute, Queen?s University Faculty of Health Sciences, Sunnybrook Health Sciences Foundation, the Thunder Bay Regional Health Sciences Centre, the University of Ottawa Faculty of Medicine, and the Windsor/Essex County ALS Association. The Temerty Family Foundation provided the major infrastructure matching funds. D. Beaton and S. Strother are partly supported by a CIHR grant (MOP 201403). S. Strother is partly supported by a CFI grant (#34862). The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was conducted with the support of the Ontario Brain Institute, an independent nonprofit corporation, funded partially by the Ontario government. Matching funds were provided by participant hospital and research institute foundations, including the Baycrest Foundation, Bruyère Research Institute, Centre for Addiction and Mental Health Foundation, London Health Sciences Foundation, McMaster University Faculty of Health Sciences, Ottawa Brain and Mind Research Institute, Queen’s University Faculty of Health Sciences, Sunnybrook Health Sciences Foundation, the Thunder Bay Regional Health Sciences Centre, the University of Ottawa Faculty of Medicine, and the Windsor/Essex County ALS Association. The Temerty Family Foundation provided the major infrastructure matching funds. D. Beaton and S. Strother are partly supported by a CIHR grant (MOP 201403). S. Strother is partly supported by a CFI grant (#34862).

Keywords

data quality
neuropsychology
quality assurance
quality control

ASJC Scopus subject areas

Clinical Psychology
Applied Psychology

Access to Document

10.1177/1073191120913933

Cite this

McLaughlin, P. M., Sunderland, K. M., Beaton, D., Binns, M. A., Kwan, D., Levine, B., Orange, J. B., Peltsch, A. J., Roberts, A. C., Strother, S. C., & Troyer, A. K. (2021). The Quality Assurance and Quality Control Protocol for Neuropsychological Data Collection and Curation in the Ontario Neurodegenerative Disease Research Initiative (ONDRI) Study. Assessment, 28(5), 1267-1286. https://doi.org/10.1177/1073191120913933

@article{e581a10901414a28b24ece69ad0c14ff,

title = "The Quality Assurance and Quality Control Protocol for Neuropsychological Data Collection and Curation in the Ontario Neurodegenerative Disease Research Initiative (ONDRI) Study",

abstract = "As large research initiatives designed to generate big data on clinical cohorts become more common, there is an increasing need to establish standard quality assurance (QA; preventing errors) and quality control (QC; identifying and correcting errors) procedures for critical outcome measures. The present article describes the QA and QC approach developed and implemented for the neuropsychology data collected as part of the Ontario Neurodegenerative Disease Research Initiative study. We report on the efficacy of our approach and provide data quality metrics. Our findings demonstrate that even with a comprehensive QA protocol, the proportion of data errors still can be high. Additionally, we show that several widely used neuropsychological measures are particularly susceptible to error. These findings highlight the need for large research programs to put into place active, comprehensive, and separate QA and QC procedures before, during, and after protocol deployment. Detailed recommendations and considerations for future studies are provided.",

keywords = "data quality, neuropsychology, quality assurance, quality control",

author = "McLaughlin, {Paula M.} and Sunderland, {Kelly M.} and Derek Beaton and Binns, {Malcolm A.} and Donna Kwan and Brian Levine and Orange, {Joseph B.} and Peltsch, {Alicia J.} and Roberts, {Angela C.} and Strother, {Stephen C.} and Troyer, {Angela K.}",

note = "Funding Information: The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: J. Orange is the Scientific Director of the Canadian Centre for Activity and Aging, and he curently holds grants from the Social Sciences and Humanities Research Council, the Canadian Centre for Aging and Brain Health Innovation, the Ontario Brain Institute and the Canadian Consortium on Neurodegeneration in Aging. S. Strother is the Chief Scientific Officer of ADMdx, Inc., which receives NIH funding, and he currently has research grants from Brain Canada, Canada Foundation for Innovation (CFI), Canadian Institutes of Health Research (CIHR), and the Ontario Brain Institute in Canada. Funding Information: We like to thank Kristen Lutz for her assistance with the electronic case report form design in REDCap, as well as the Indoc research team for their data management support. This work was completed on behalf of the Ontario Neurodegenerative Disease Research Initiative (ONDRI). The authors would like to acknowledge the ONDRI Founding Authors: Robert Bartha, Sandra E. Black, Michael Borrie, Dale Corbett, Elizabeth Finger, Morris Freedman, Barry Greenberg, David A. Grimes, Robert A. Hegele, Chris Hudson, Anthony E. Lang, Mario Masellis, William E. McIlroy, Paula M. McLaughlin, Manuel Montero-Odasso, David G. Munoz, Douglas P. Munoz, J. B. Orange, Michael J. Strong, Stephen C. Strother, Richard H. Swartz, Sean Symons, Maria Carmela Tartaglia, Angela Troyer, and Lorne Zinman. The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was conducted with the support of the Ontario Brain Institute, an independent nonprofit corporation, funded partially by the Ontario government. Matching funds were provided by participant hospital and research institute foundations, including the Baycrest Foundation, Bruy?re Research Institute, Centre for Addiction and Mental Health Foundation, London Health Sciences Foundation, McMaster University Faculty of Health Sciences, Ottawa Brain and Mind Research Institute, Queen?s University Faculty of Health Sciences, Sunnybrook Health Sciences Foundation, the Thunder Bay Regional Health Sciences Centre, the University of Ottawa Faculty of Medicine, and the Windsor/Essex County ALS Association. The Temerty Family Foundation provided the major infrastructure matching funds. D. Beaton and S. Strother are partly supported by a CIHR grant (MOP 201403). S. Strother is partly supported by a CFI grant (#34862). Funding Information: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was conducted with the support of the Ontario Brain Institute, an independent nonprofit corporation, funded partially by the Ontario government. Matching funds were provided by participant hospital and research institute foundations, including the Baycrest Foundation, Bruy{\`e}re Research Institute, Centre for Addiction and Mental Health Foundation, London Health Sciences Foundation, McMaster University Faculty of Health Sciences, Ottawa Brain and Mind Research Institute, Queen{\textquoteright}s University Faculty of Health Sciences, Sunnybrook Health Sciences Foundation, the Thunder Bay Regional Health Sciences Centre, the University of Ottawa Faculty of Medicine, and the Windsor/Essex County ALS Association. The Temerty Family Foundation provided the major infrastructure matching funds. D. Beaton and S. Strother are partly supported by a CIHR grant (MOP 201403). S. Strother is partly supported by a CFI grant (#34862). Publisher Copyright: {\textcopyright} The Author(s) 2020.",

year = "2021",

month = jul,

doi = "10.1177/1073191120913933",

language = "English (US)",

volume = "28",

pages = "1267--1286",

journal = "Assessment",

issn = "1073-1911",

publisher = "SAGE Publications Inc.",

number = "5",

}

McLaughlin, PM, Sunderland, KM, Beaton, D, Binns, MA, Kwan, D, Levine, B, Orange, JB, Peltsch, AJ, Roberts, AC, Strother, SC & Troyer, AK 2021, 'The Quality Assurance and Quality Control Protocol for Neuropsychological Data Collection and Curation in the Ontario Neurodegenerative Disease Research Initiative (ONDRI) Study', Assessment, vol. 28, no. 5, pp. 1267-1286. https://doi.org/10.1177/1073191120913933

TY - JOUR

T1 - The Quality Assurance and Quality Control Protocol for Neuropsychological Data Collection and Curation in the Ontario Neurodegenerative Disease Research Initiative (ONDRI) Study

AU - McLaughlin, Paula M.

AU - Sunderland, Kelly M.

AU - Beaton, Derek

AU - Binns, Malcolm A.

AU - Kwan, Donna

AU - Levine, Brian

AU - Orange, Joseph B.

AU - Peltsch, Alicia J.

AU - Roberts, Angela C.

AU - Strother, Stephen C.

AU - Troyer, Angela K.

N1 - Funding Information: The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: J. Orange is the Scientific Director of the Canadian Centre for Activity and Aging, and he curently holds grants from the Social Sciences and Humanities Research Council, the Canadian Centre for Aging and Brain Health Innovation, the Ontario Brain Institute and the Canadian Consortium on Neurodegeneration in Aging. S. Strother is the Chief Scientific Officer of ADMdx, Inc., which receives NIH funding, and he currently has research grants from Brain Canada, Canada Foundation for Innovation (CFI), Canadian Institutes of Health Research (CIHR), and the Ontario Brain Institute in Canada. Funding Information: We like to thank Kristen Lutz for her assistance with the electronic case report form design in REDCap, as well as the Indoc research team for their data management support. This work was completed on behalf of the Ontario Neurodegenerative Disease Research Initiative (ONDRI). The authors would like to acknowledge the ONDRI Founding Authors: Robert Bartha, Sandra E. Black, Michael Borrie, Dale Corbett, Elizabeth Finger, Morris Freedman, Barry Greenberg, David A. Grimes, Robert A. Hegele, Chris Hudson, Anthony E. Lang, Mario Masellis, William E. McIlroy, Paula M. McLaughlin, Manuel Montero-Odasso, David G. Munoz, Douglas P. Munoz, J. B. Orange, Michael J. Strong, Stephen C. Strother, Richard H. Swartz, Sean Symons, Maria Carmela Tartaglia, Angela Troyer, and Lorne Zinman. The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was conducted with the support of the Ontario Brain Institute, an independent nonprofit corporation, funded partially by the Ontario government. Matching funds were provided by participant hospital and research institute foundations, including the Baycrest Foundation, Bruy?re Research Institute, Centre for Addiction and Mental Health Foundation, London Health Sciences Foundation, McMaster University Faculty of Health Sciences, Ottawa Brain and Mind Research Institute, Queen?s University Faculty of Health Sciences, Sunnybrook Health Sciences Foundation, the Thunder Bay Regional Health Sciences Centre, the University of Ottawa Faculty of Medicine, and the Windsor/Essex County ALS Association. The Temerty Family Foundation provided the major infrastructure matching funds. D. Beaton and S. Strother are partly supported by a CIHR grant (MOP 201403). S. Strother is partly supported by a CFI grant (#34862). Funding Information: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was conducted with the support of the Ontario Brain Institute, an independent nonprofit corporation, funded partially by the Ontario government. Matching funds were provided by participant hospital and research institute foundations, including the Baycrest Foundation, Bruyère Research Institute, Centre for Addiction and Mental Health Foundation, London Health Sciences Foundation, McMaster University Faculty of Health Sciences, Ottawa Brain and Mind Research Institute, Queen’s University Faculty of Health Sciences, Sunnybrook Health Sciences Foundation, the Thunder Bay Regional Health Sciences Centre, the University of Ottawa Faculty of Medicine, and the Windsor/Essex County ALS Association. The Temerty Family Foundation provided the major infrastructure matching funds. D. Beaton and S. Strother are partly supported by a CIHR grant (MOP 201403). S. Strother is partly supported by a CFI grant (#34862). Publisher Copyright: © The Author(s) 2020.

PY - 2021/7

Y1 - 2021/7

N2 - As large research initiatives designed to generate big data on clinical cohorts become more common, there is an increasing need to establish standard quality assurance (QA; preventing errors) and quality control (QC; identifying and correcting errors) procedures for critical outcome measures. The present article describes the QA and QC approach developed and implemented for the neuropsychology data collected as part of the Ontario Neurodegenerative Disease Research Initiative study. We report on the efficacy of our approach and provide data quality metrics. Our findings demonstrate that even with a comprehensive QA protocol, the proportion of data errors still can be high. Additionally, we show that several widely used neuropsychological measures are particularly susceptible to error. These findings highlight the need for large research programs to put into place active, comprehensive, and separate QA and QC procedures before, during, and after protocol deployment. Detailed recommendations and considerations for future studies are provided.

AB - As large research initiatives designed to generate big data on clinical cohorts become more common, there is an increasing need to establish standard quality assurance (QA; preventing errors) and quality control (QC; identifying and correcting errors) procedures for critical outcome measures. The present article describes the QA and QC approach developed and implemented for the neuropsychology data collected as part of the Ontario Neurodegenerative Disease Research Initiative study. We report on the efficacy of our approach and provide data quality metrics. Our findings demonstrate that even with a comprehensive QA protocol, the proportion of data errors still can be high. Additionally, we show that several widely used neuropsychological measures are particularly susceptible to error. These findings highlight the need for large research programs to put into place active, comprehensive, and separate QA and QC procedures before, during, and after protocol deployment. Detailed recommendations and considerations for future studies are provided.

KW - data quality

KW - neuropsychology

KW - quality assurance

KW - quality control

UR - http://www.scopus.com/inward/record.url?scp=85084059145&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85084059145&partnerID=8YFLogxK

U2 - 10.1177/1073191120913933

DO - 10.1177/1073191120913933

M3 - Article

C2 - 32321297

AN - SCOPUS:85084059145

SN - 1073-1911

VL - 28

SP - 1267

EP - 1286

JO - Assessment

JF - Assessment

IS - 5

ER -

The Quality Assurance and Quality Control Protocol for Neuropsychological Data Collection and Curation in the Ontario Neurodegenerative Disease Research Initiative (ONDRI) Study

Abstract

Funding

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint