Psychometric properties of the mock interview rating scale for schizophrenia and other serious mental illnesses

Matthew J. Smith; Jane K. Burke-Miller; Lindsay A. Bornheimer; Brittany Ross; Morris D. Bell; Susan R. McGurk; Kim T. Mueser; Adrienne Brown; John Prestipino; Nayab Borghani; Karley Nelson; Tovah Lieberman; Nicole J. Pashka; Lisa A. Razzano; Michael A. Kallen

doi:10.3389/fpsyt.2023.1150307

Psychometric properties of the mock interview rating scale for schizophrenia and other serious mental illnesses

Matthew J. Smith^*, Jane K. Burke-Miller, Lindsay A. Bornheimer, Brittany Ross, Morris D. Bell, Susan R. McGurk, Kim T. Mueser, Adrienne Brown, John Prestipino, Nayab Borghani, Karley Nelson, Tovah Lieberman, Nicole J. Pashka, Lisa A. Razzano, Michael A. Kallen

^*Corresponding author for this work

Medical Social Sciences

Research output: Contribution to journal › Article › peer-review

2 Scopus citations

Abstract

Background: Over the past 10 years, job interview training has emerged as an area of study among adults with schizophrenia and other serious mental illnesses who face significant challenges when navigating job interviews. The field of mental health services research has limited access to assessments of job interview skills with rigorously evaluated psychometric properties. Objective: We sought to evaluate the initial psychometric properties of a measure assessing job interview skills via role-play performance. Methods: As part of a randomized controlled trial, 90 adults with schizophrenia or other serious mental illnesses completed a job interview role-play assessment with eight items (and scored using anchors) called the mock interview rating scale (MIRS). A classical test theory analysis was conducted including confirmatory factor analyses, Rasch model analysis and calibration, and differential item functioning; along with inter-rater, internal consistency, and test-retest reliabilities. Pearson correlations were used to evaluate construct, convergent, divergent, criterion, and predictive validity by correlating the MIRS with demographic, clinical, cognitive, work history measures, and employment outcomes. Results: Our analyses resulted in the removal of a single item (sounding honest) and yielded a unidimensional total score measurement with support for its inter-rater reliability, internal consistency, and test-retest reliability. There was initial support for the construct, convergent, criterion, and predictive validities of the MIRS, as it correlated with measures of social competence, neurocognition, valuing job interview training, and employment outcomes. Meanwhile, the lack of correlations with race, physical health, and substance abuse lent support for divergent validity. Conclusion: This study presents initial evidence that the seven-item version of the MIRS has acceptable psychometric properties supporting its use to assess job interview skills reliably and validly among adults with schizophrenia and other serious mental illnesses. Clinical Trial Registration: NCT03049813.

Original language	English (US)
Article number	1150307
Journal	Frontiers in Psychiatry
Volume	14
DOIs	https://doi.org/10.3389/fpsyt.2023.1150307
State	Published - 2023

Funding

We thank the members of Thresholds who participated in the study as well as acknowledge the more than 50 administrators, research staff, and employment staff at Thresholds in Chicago, IL, who diligently supported various aspects of the study implementation. We also thank the research staff at the University of Michigan Level Up: Employment Skills Simulation Lab (http://leveluplab.org/) for data collection, processing, and management and research staff at the Northwestern University Department of Psychiatry and Behavioral Sciences Mental Health Services and Policy Program for IRB regulatory support. This project was supported by funding from the National Institute of Mental Health (R01MH110524; PI: MS).

Keywords

employment
job interview skills assessment
psychometric properties
schizophrenia
serious mental illness (SMI)

ASJC Scopus subject areas

Psychiatry and Mental health

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.3389/fpsyt.2023.1150307

Cite this

Smith, M. J., Burke-Miller, J. K., Bornheimer, L. A., Ross, B., Bell, M. D., McGurk, S. R., Mueser, K. T., Brown, A., Prestipino, J., Borghani, N., Nelson, K., Lieberman, T., Pashka, N. J., Razzano, L. A., & Kallen, M. A. (2023). Psychometric properties of the mock interview rating scale for schizophrenia and other serious mental illnesses. Frontiers in Psychiatry, 14, Article 1150307. https://doi.org/10.3389/fpsyt.2023.1150307

@article{a3d627fab42e4040be8f55af136a5545,

title = "Psychometric properties of the mock interview rating scale for schizophrenia and other serious mental illnesses",

abstract = "Background: Over the past 10 years, job interview training has emerged as an area of study among adults with schizophrenia and other serious mental illnesses who face significant challenges when navigating job interviews. The field of mental health services research has limited access to assessments of job interview skills with rigorously evaluated psychometric properties. Objective: We sought to evaluate the initial psychometric properties of a measure assessing job interview skills via role-play performance. Methods: As part of a randomized controlled trial, 90 adults with schizophrenia or other serious mental illnesses completed a job interview role-play assessment with eight items (and scored using anchors) called the mock interview rating scale (MIRS). A classical test theory analysis was conducted including confirmatory factor analyses, Rasch model analysis and calibration, and differential item functioning; along with inter-rater, internal consistency, and test-retest reliabilities. Pearson correlations were used to evaluate construct, convergent, divergent, criterion, and predictive validity by correlating the MIRS with demographic, clinical, cognitive, work history measures, and employment outcomes. Results: Our analyses resulted in the removal of a single item (sounding honest) and yielded a unidimensional total score measurement with support for its inter-rater reliability, internal consistency, and test-retest reliability. There was initial support for the construct, convergent, criterion, and predictive validities of the MIRS, as it correlated with measures of social competence, neurocognition, valuing job interview training, and employment outcomes. Meanwhile, the lack of correlations with race, physical health, and substance abuse lent support for divergent validity. Conclusion: This study presents initial evidence that the seven-item version of the MIRS has acceptable psychometric properties supporting its use to assess job interview skills reliably and validly among adults with schizophrenia and other serious mental illnesses. Clinical Trial Registration: NCT03049813.",

keywords = "employment, job interview skills assessment, psychometric properties, schizophrenia, serious mental illness (SMI)",

author = "Smith, {Matthew J.} and Burke-Miller, {Jane K.} and Bornheimer, {Lindsay A.} and Brittany Ross and Bell, {Morris D.} and McGurk, {Susan R.} and Mueser, {Kim T.} and Adrienne Brown and John Prestipino and Nayab Borghani and Karley Nelson and Tovah Lieberman and Pashka, {Nicole J.} and Razzano, {Lisa A.} and Kallen, {Michael A.}",

note = "Funding Information: We thank the members of Thresholds who participated in the study as well as acknowledge the more than 50 administrators, research staff, and employment staff at Thresholds in Chicago, IL, who diligently supported various aspects of the study implementation. We also thank the research staff at the University of Michigan Level Up: Employment Skills Simulation Lab (http://leveluplab.org/) for data collection, processing, and management and research staff at the Northwestern University Department of Psychiatry and Behavioral Sciences Mental Health Services and Policy Program for IRB regulatory support. Funding Information: This project was supported by funding from the National Institute of Mental Health (R01MH110524; PI: MS). Publisher Copyright: Copyright {\textcopyright} 2023 Smith, Burke-Miller, Bornheimer, Ross, Bell, McGurk, Mueser, Brown, Prestipino, Borghani, Nelson, Lieberman, Pashka, Razzano and Kallen.",

year = "2023",

doi = "10.3389/fpsyt.2023.1150307",

language = "English (US)",

volume = "14",

journal = "Frontiers in Psychiatry",

issn = "1664-0640",

publisher = "Frontiers Media SA",

}

Smith, MJ, Burke-Miller, JK, Bornheimer, LA, Ross, B, Bell, MD, McGurk, SR, Mueser, KT, Brown, A, Prestipino, J, Borghani, N, Nelson, K, Lieberman, T, Pashka, NJ, Razzano, LA & Kallen, MA 2023, 'Psychometric properties of the mock interview rating scale for schizophrenia and other serious mental illnesses', Frontiers in Psychiatry, vol. 14, 1150307. https://doi.org/10.3389/fpsyt.2023.1150307

TY - JOUR

T1 - Psychometric properties of the mock interview rating scale for schizophrenia and other serious mental illnesses

AU - Smith, Matthew J.

AU - Burke-Miller, Jane K.

AU - Bornheimer, Lindsay A.

AU - Ross, Brittany

AU - Bell, Morris D.

AU - McGurk, Susan R.

AU - Mueser, Kim T.

AU - Brown, Adrienne

AU - Prestipino, John

AU - Borghani, Nayab

AU - Nelson, Karley

AU - Lieberman, Tovah

AU - Pashka, Nicole J.

AU - Razzano, Lisa A.

AU - Kallen, Michael A.

N1 - Funding Information: We thank the members of Thresholds who participated in the study as well as acknowledge the more than 50 administrators, research staff, and employment staff at Thresholds in Chicago, IL, who diligently supported various aspects of the study implementation. We also thank the research staff at the University of Michigan Level Up: Employment Skills Simulation Lab (http://leveluplab.org/) for data collection, processing, and management and research staff at the Northwestern University Department of Psychiatry and Behavioral Sciences Mental Health Services and Policy Program for IRB regulatory support. Funding Information: This project was supported by funding from the National Institute of Mental Health (R01MH110524; PI: MS). Publisher Copyright: Copyright © 2023 Smith, Burke-Miller, Bornheimer, Ross, Bell, McGurk, Mueser, Brown, Prestipino, Borghani, Nelson, Lieberman, Pashka, Razzano and Kallen.

PY - 2023

Y1 - 2023

N2 - Background: Over the past 10 years, job interview training has emerged as an area of study among adults with schizophrenia and other serious mental illnesses who face significant challenges when navigating job interviews. The field of mental health services research has limited access to assessments of job interview skills with rigorously evaluated psychometric properties. Objective: We sought to evaluate the initial psychometric properties of a measure assessing job interview skills via role-play performance. Methods: As part of a randomized controlled trial, 90 adults with schizophrenia or other serious mental illnesses completed a job interview role-play assessment with eight items (and scored using anchors) called the mock interview rating scale (MIRS). A classical test theory analysis was conducted including confirmatory factor analyses, Rasch model analysis and calibration, and differential item functioning; along with inter-rater, internal consistency, and test-retest reliabilities. Pearson correlations were used to evaluate construct, convergent, divergent, criterion, and predictive validity by correlating the MIRS with demographic, clinical, cognitive, work history measures, and employment outcomes. Results: Our analyses resulted in the removal of a single item (sounding honest) and yielded a unidimensional total score measurement with support for its inter-rater reliability, internal consistency, and test-retest reliability. There was initial support for the construct, convergent, criterion, and predictive validities of the MIRS, as it correlated with measures of social competence, neurocognition, valuing job interview training, and employment outcomes. Meanwhile, the lack of correlations with race, physical health, and substance abuse lent support for divergent validity. Conclusion: This study presents initial evidence that the seven-item version of the MIRS has acceptable psychometric properties supporting its use to assess job interview skills reliably and validly among adults with schizophrenia and other serious mental illnesses. Clinical Trial Registration: NCT03049813.

AB - Background: Over the past 10 years, job interview training has emerged as an area of study among adults with schizophrenia and other serious mental illnesses who face significant challenges when navigating job interviews. The field of mental health services research has limited access to assessments of job interview skills with rigorously evaluated psychometric properties. Objective: We sought to evaluate the initial psychometric properties of a measure assessing job interview skills via role-play performance. Methods: As part of a randomized controlled trial, 90 adults with schizophrenia or other serious mental illnesses completed a job interview role-play assessment with eight items (and scored using anchors) called the mock interview rating scale (MIRS). A classical test theory analysis was conducted including confirmatory factor analyses, Rasch model analysis and calibration, and differential item functioning; along with inter-rater, internal consistency, and test-retest reliabilities. Pearson correlations were used to evaluate construct, convergent, divergent, criterion, and predictive validity by correlating the MIRS with demographic, clinical, cognitive, work history measures, and employment outcomes. Results: Our analyses resulted in the removal of a single item (sounding honest) and yielded a unidimensional total score measurement with support for its inter-rater reliability, internal consistency, and test-retest reliability. There was initial support for the construct, convergent, criterion, and predictive validities of the MIRS, as it correlated with measures of social competence, neurocognition, valuing job interview training, and employment outcomes. Meanwhile, the lack of correlations with race, physical health, and substance abuse lent support for divergent validity. Conclusion: This study presents initial evidence that the seven-item version of the MIRS has acceptable psychometric properties supporting its use to assess job interview skills reliably and validly among adults with schizophrenia and other serious mental illnesses. Clinical Trial Registration: NCT03049813.

KW - employment

KW - job interview skills assessment

KW - psychometric properties

KW - schizophrenia

KW - serious mental illness (SMI)

UR - http://www.scopus.com/inward/record.url?scp=85159048507&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85159048507&partnerID=8YFLogxK

U2 - 10.3389/fpsyt.2023.1150307

DO - 10.3389/fpsyt.2023.1150307

M3 - Article

C2 - 37181877

AN - SCOPUS:85159048507

SN - 1664-0640

VL - 14

JO - Frontiers in Psychiatry

JF - Frontiers in Psychiatry

M1 - 1150307

ER -

Psychometric properties of the mock interview rating scale for schizophrenia and other serious mental illnesses

Abstract

Funding

Keywords

ASJC Scopus subject areas

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this