Data envelopment analysis to evaluate the efficiency of tobacco treatment programs in the NCI Moonshot Cancer Center Cessation Initiative

Kathryn Pluta; Sarah D. Hohl; Heather D’Angelo; Jamie S. Ostroff; Donna Shelley; Yasmin Asvat; Li Shiun Chen; K. Michael Cummings; Neely Dahl; Andrew T. Day; Linda Fleisher; Adam O. Goldstein; Rashelle Hayes; Brian Hitsman; Deborah Hudson Buckles; Andrea C. King; Cho Y. Lam; Katie Lenhoff; Arnold H. Levinson; Mara Minion; Cary Presant; Judith J. Prochaska; Kimberly Shoenbill; Vani Simmons; Kathryn Taylor; Hilary Tindle; Elisa Tong; Justin S. White; Kara P. Wiseman; Graham W. Warren; Timothy B. Baker; Betsy Rolland; Michael C. Fiore; Ramzi G. Salloum

doi:10.1186/s43058-023-00433-3

Data envelopment analysis to evaluate the efficiency of tobacco treatment programs in the NCI Moonshot Cancer Center Cessation Initiative

Kathryn Pluta, Sarah D. Hohl, Heather D’Angelo, Jamie S. Ostroff, Donna Shelley, Yasmin Asvat, Li Shiun Chen, K. Michael Cummings, Neely Dahl, Andrew T. Day, Linda Fleisher, Adam O. Goldstein, Rashelle Hayes, Brian Hitsman, Deborah Hudson Buckles, Andrea C. King, Cho Y. Lam, Katie Lenhoff, Arnold H. Levinson, Mara MinionCary Presant, Judith J. Prochaska, Kimberly Shoenbill, Vani Simmons, Kathryn Taylor, Hilary Tindle, Elisa Tong, Justin S. White, Kara P. Wiseman, Graham W. Warren, Timothy B. Baker, Betsy Rolland, Michael C. Fiore, Ramzi G. Salloum^*

^*Corresponding author for this work

Preventive Medicine

Research output: Contribution to journal › Article › peer-review

3 Scopus citations

Abstract

Background: The Cancer Center Cessation Initiative (C3I) is a National Cancer Institute (NCI) Cancer Moonshot Program that supports NCI-designated cancer centers developing tobacco treatment programs for oncology patients who smoke. C3I-funded centers implement evidence-based programs that offer various smoking cessation treatment components (e.g., counseling, Quitline referrals, access to medications). While evaluation of implementation outcomes in C3I is guided by evaluation of reach and effectiveness (via RE-AIM), little is known about technical efficiency—i.e., how inputs (e.g., program costs, staff time) influence implementation outcomes (e.g., reach, effectiveness). This study demonstrates the application of data envelopment analysis (DEA) as an implementation science tool to evaluate technical efficiency of C3I programs and advance prioritization of implementation resources. Methods: DEA is a linear programming technique widely used in economics and engineering for assessing relative performance of production units. Using data from 16 C3I-funded centers reported in 2020, we applied input-oriented DEA to model technical efficiency (i.e., proportion of observed outcomes to benchmarked outcomes for given input levels). The primary models used the constant returns-to-scale specification and featured cost-per-participant, total full-time equivalent (FTE) effort, and tobacco treatment specialist effort as model inputs and reach and effectiveness (quit rates) as outcomes. Results: In the DEA model featuring cost-per-participant (input) and reach/effectiveness (outcomes), average constant returns-to-scale technical efficiency was 25.66 (SD = 24.56). When stratified by program characteristics, technical efficiency was higher among programs in cohort 1 (M = 29.15, SD = 28.65, n = 11) vs. cohort 2 (M = 17.99, SD = 10.16, n = 5), with point-of-care (M = 33.90, SD = 28.63, n = 9) vs. no point-of-care services (M = 15.59, SD = 14.31, n = 7), larger (M = 33.63, SD = 30.38, n = 8) vs. smaller center size (M = 17.70, SD = 15.00, n = 8), and higher (M = 29.65, SD = 30.99, n = 8) vs. lower smoking prevalence (M = 21.67, SD = 17.21, n = 8). Conclusion: Most C3I programs assessed were technically inefficient relative to the most efficient center benchmark and may be improved by optimizing the use of inputs (e.g., cost-per-participant) relative to program outcomes (e.g., reach, effectiveness). This study demonstrates the appropriateness and feasibility of using DEA to evaluate the relative performance of evidence-based programs.

Original language	English (US)
Article number	50
Journal	Implementation Science Communications
Volume	4
Issue number	1
DOIs	https://doi.org/10.1186/s43058-023-00433-3
State	Published - Dec 2023

Funding

A contract from the 17GZSK0031) supported the implementation of tobacco treatment programs and data reporting at the NCI-designated cancer centers and coordination efforts at the University of Wisconsin—Madison. Ramzi G. Salloum was supported by NCTATS award UL1TR001427. Jamie S. Ostroff was supported by NCI award P30CA008748-52S1. Li-Shiun Chen was supported by NCI awards P30CA091842-19S5, P50CA24443 and the Siteman Investment Program. Cary A. Presant was supported by NCI grants P30CA033572 and P30CA03572-37S5. Neely Dahl and Kara Wiseman were supported by NCI award P30CA044579. Kara Wiseman was supported by the UVA iTHRIV Scholars Program, NCATS UL1T R003015 and KL2TR003016. Katie Lenhoff was supported by NCI award P30CA023108-43. Kimberly Shoenbill was supported by NCI awards P30CA016086-43S1 and P30CA016086-44S5. Hilary A. Tindle was supported by NCI award 3P30CA068485-24S3. Graham Warren was supported by NCI awards P30CA138313-09S4 and A22-0010–01. Judith J. Prochaska was supported by NCI awards P30CA124435-11S1 and P30CA124435-13S2. Timothy Baker was supported by NCI award P01 CA180945. Justin S. White was supported by award P30CA082103-19S2.

Keywords

Cancer
Data envelopment analysis
Efficiency
Implementation costs
Implementation science
Program performance
Smoking cessation
Tobacco treatment

ASJC Scopus subject areas

Health Policy
Health Informatics
Public Health, Environmental and Occupational Health

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1186/s43058-023-00433-3

Cite this

Pluta, K., Hohl, S. D., D’Angelo, H., Ostroff, J. S., Shelley, D., Asvat, Y., Chen, L. S., Cummings, K. M., Dahl, N., Day, A. T., Fleisher, L., Goldstein, A. O., Hayes, R., Hitsman, B., Buckles, D. H., King, A. C., Lam, C. Y., Lenhoff, K., Levinson, A. H., ... Salloum, R. G. (2023). Data envelopment analysis to evaluate the efficiency of tobacco treatment programs in the NCI Moonshot Cancer Center Cessation Initiative. Implementation Science Communications, 4(1), Article 50. https://doi.org/10.1186/s43058-023-00433-3

@article{808edae29766457db6f51ddbc1c05238,

title = "Data envelopment analysis to evaluate the efficiency of tobacco treatment programs in the NCI Moonshot Cancer Center Cessation Initiative",

abstract = "Background: The Cancer Center Cessation Initiative (C3I) is a National Cancer Institute (NCI) Cancer Moonshot Program that supports NCI-designated cancer centers developing tobacco treatment programs for oncology patients who smoke. C3I-funded centers implement evidence-based programs that offer various smoking cessation treatment components (e.g., counseling, Quitline referrals, access to medications). While evaluation of implementation outcomes in C3I is guided by evaluation of reach and effectiveness (via RE-AIM), little is known about technical efficiency—i.e., how inputs (e.g., program costs, staff time) influence implementation outcomes (e.g., reach, effectiveness). This study demonstrates the application of data envelopment analysis (DEA) as an implementation science tool to evaluate technical efficiency of C3I programs and advance prioritization of implementation resources. Methods: DEA is a linear programming technique widely used in economics and engineering for assessing relative performance of production units. Using data from 16 C3I-funded centers reported in 2020, we applied input-oriented DEA to model technical efficiency (i.e., proportion of observed outcomes to benchmarked outcomes for given input levels). The primary models used the constant returns-to-scale specification and featured cost-per-participant, total full-time equivalent (FTE) effort, and tobacco treatment specialist effort as model inputs and reach and effectiveness (quit rates) as outcomes. Results: In the DEA model featuring cost-per-participant (input) and reach/effectiveness (outcomes), average constant returns-to-scale technical efficiency was 25.66 (SD = 24.56). When stratified by program characteristics, technical efficiency was higher among programs in cohort 1 (M = 29.15, SD = 28.65, n = 11) vs. cohort 2 (M = 17.99, SD = 10.16, n = 5), with point-of-care (M = 33.90, SD = 28.63, n = 9) vs. no point-of-care services (M = 15.59, SD = 14.31, n = 7), larger (M = 33.63, SD = 30.38, n = 8) vs. smaller center size (M = 17.70, SD = 15.00, n = 8), and higher (M = 29.65, SD = 30.99, n = 8) vs. lower smoking prevalence (M = 21.67, SD = 17.21, n = 8). Conclusion: Most C3I programs assessed were technically inefficient relative to the most efficient center benchmark and may be improved by optimizing the use of inputs (e.g., cost-per-participant) relative to program outcomes (e.g., reach, effectiveness). This study demonstrates the appropriateness and feasibility of using DEA to evaluate the relative performance of evidence-based programs.",

keywords = "Cancer, Data envelopment analysis, Efficiency, Implementation costs, Implementation science, Program performance, Smoking cessation, Tobacco treatment",

author = "Kathryn Pluta and Hohl, {Sarah D.} and Heather D{\textquoteright}Angelo and Ostroff, {Jamie S.} and Donna Shelley and Yasmin Asvat and Chen, {Li Shiun} and Cummings, {K. Michael} and Neely Dahl and Day, {Andrew T.} and Linda Fleisher and Goldstein, {Adam O.} and Rashelle Hayes and Brian Hitsman and Buckles, {Deborah Hudson} and King, {Andrea C.} and Lam, {Cho Y.} and Katie Lenhoff and Levinson, {Arnold H.} and Mara Minion and Cary Presant and Prochaska, {Judith J.} and Kimberly Shoenbill and Vani Simmons and Kathryn Taylor and Hilary Tindle and Elisa Tong and White, {Justin S.} and Wiseman, {Kara P.} and Warren, {Graham W.} and Baker, {Timothy B.} and Betsy Rolland and Fiore, {Michael C.} and Salloum, {Ramzi G.}",

note = "Publisher Copyright: {\textcopyright} 2023, The Author(s).",

year = "2023",

month = dec,

doi = "10.1186/s43058-023-00433-3",

language = "English (US)",

volume = "4",

journal = "Implementation Science Communications",

issn = "2662-2211",

publisher = "BioMed Central Ltd.",

number = "1",

}

Pluta, K, Hohl, SD, D’Angelo, H, Ostroff, JS, Shelley, D, Asvat, Y, Chen, LS, Cummings, KM, Dahl, N, Day, AT, Fleisher, L, Goldstein, AO, Hayes, R, Hitsman, B, Buckles, DH, King, AC, Lam, CY, Lenhoff, K, Levinson, AH, Minion, M, Presant, C, Prochaska, JJ, Shoenbill, K, Simmons, V, Taylor, K, Tindle, H, Tong, E, White, JS, Wiseman, KP, Warren, GW, Baker, TB, Rolland, B, Fiore, MC & Salloum, RG 2023, 'Data envelopment analysis to evaluate the efficiency of tobacco treatment programs in the NCI Moonshot Cancer Center Cessation Initiative', Implementation Science Communications, vol. 4, no. 1, 50. https://doi.org/10.1186/s43058-023-00433-3

TY - JOUR

T1 - Data envelopment analysis to evaluate the efficiency of tobacco treatment programs in the NCI Moonshot Cancer Center Cessation Initiative

AU - Pluta, Kathryn

AU - Hohl, Sarah D.

AU - D’Angelo, Heather

AU - Ostroff, Jamie S.

AU - Shelley, Donna

AU - Asvat, Yasmin

AU - Chen, Li Shiun

AU - Cummings, K. Michael

AU - Dahl, Neely

AU - Day, Andrew T.

AU - Fleisher, Linda

AU - Goldstein, Adam O.

AU - Hayes, Rashelle

AU - Hitsman, Brian

AU - Buckles, Deborah Hudson

AU - King, Andrea C.

AU - Lam, Cho Y.

AU - Lenhoff, Katie

AU - Levinson, Arnold H.

AU - Minion, Mara

AU - Presant, Cary

AU - Prochaska, Judith J.

AU - Shoenbill, Kimberly

AU - Simmons, Vani

AU - Taylor, Kathryn

AU - Tindle, Hilary

AU - Tong, Elisa

AU - White, Justin S.

AU - Wiseman, Kara P.

AU - Warren, Graham W.

AU - Baker, Timothy B.

AU - Rolland, Betsy

AU - Fiore, Michael C.

AU - Salloum, Ramzi G.

PY - 2023/12

Y1 - 2023/12

N2 - Background: The Cancer Center Cessation Initiative (C3I) is a National Cancer Institute (NCI) Cancer Moonshot Program that supports NCI-designated cancer centers developing tobacco treatment programs for oncology patients who smoke. C3I-funded centers implement evidence-based programs that offer various smoking cessation treatment components (e.g., counseling, Quitline referrals, access to medications). While evaluation of implementation outcomes in C3I is guided by evaluation of reach and effectiveness (via RE-AIM), little is known about technical efficiency—i.e., how inputs (e.g., program costs, staff time) influence implementation outcomes (e.g., reach, effectiveness). This study demonstrates the application of data envelopment analysis (DEA) as an implementation science tool to evaluate technical efficiency of C3I programs and advance prioritization of implementation resources. Methods: DEA is a linear programming technique widely used in economics and engineering for assessing relative performance of production units. Using data from 16 C3I-funded centers reported in 2020, we applied input-oriented DEA to model technical efficiency (i.e., proportion of observed outcomes to benchmarked outcomes for given input levels). The primary models used the constant returns-to-scale specification and featured cost-per-participant, total full-time equivalent (FTE) effort, and tobacco treatment specialist effort as model inputs and reach and effectiveness (quit rates) as outcomes. Results: In the DEA model featuring cost-per-participant (input) and reach/effectiveness (outcomes), average constant returns-to-scale technical efficiency was 25.66 (SD = 24.56). When stratified by program characteristics, technical efficiency was higher among programs in cohort 1 (M = 29.15, SD = 28.65, n = 11) vs. cohort 2 (M = 17.99, SD = 10.16, n = 5), with point-of-care (M = 33.90, SD = 28.63, n = 9) vs. no point-of-care services (M = 15.59, SD = 14.31, n = 7), larger (M = 33.63, SD = 30.38, n = 8) vs. smaller center size (M = 17.70, SD = 15.00, n = 8), and higher (M = 29.65, SD = 30.99, n = 8) vs. lower smoking prevalence (M = 21.67, SD = 17.21, n = 8). Conclusion: Most C3I programs assessed were technically inefficient relative to the most efficient center benchmark and may be improved by optimizing the use of inputs (e.g., cost-per-participant) relative to program outcomes (e.g., reach, effectiveness). This study demonstrates the appropriateness and feasibility of using DEA to evaluate the relative performance of evidence-based programs.

AB - Background: The Cancer Center Cessation Initiative (C3I) is a National Cancer Institute (NCI) Cancer Moonshot Program that supports NCI-designated cancer centers developing tobacco treatment programs for oncology patients who smoke. C3I-funded centers implement evidence-based programs that offer various smoking cessation treatment components (e.g., counseling, Quitline referrals, access to medications). While evaluation of implementation outcomes in C3I is guided by evaluation of reach and effectiveness (via RE-AIM), little is known about technical efficiency—i.e., how inputs (e.g., program costs, staff time) influence implementation outcomes (e.g., reach, effectiveness). This study demonstrates the application of data envelopment analysis (DEA) as an implementation science tool to evaluate technical efficiency of C3I programs and advance prioritization of implementation resources. Methods: DEA is a linear programming technique widely used in economics and engineering for assessing relative performance of production units. Using data from 16 C3I-funded centers reported in 2020, we applied input-oriented DEA to model technical efficiency (i.e., proportion of observed outcomes to benchmarked outcomes for given input levels). The primary models used the constant returns-to-scale specification and featured cost-per-participant, total full-time equivalent (FTE) effort, and tobacco treatment specialist effort as model inputs and reach and effectiveness (quit rates) as outcomes. Results: In the DEA model featuring cost-per-participant (input) and reach/effectiveness (outcomes), average constant returns-to-scale technical efficiency was 25.66 (SD = 24.56). When stratified by program characteristics, technical efficiency was higher among programs in cohort 1 (M = 29.15, SD = 28.65, n = 11) vs. cohort 2 (M = 17.99, SD = 10.16, n = 5), with point-of-care (M = 33.90, SD = 28.63, n = 9) vs. no point-of-care services (M = 15.59, SD = 14.31, n = 7), larger (M = 33.63, SD = 30.38, n = 8) vs. smaller center size (M = 17.70, SD = 15.00, n = 8), and higher (M = 29.65, SD = 30.99, n = 8) vs. lower smoking prevalence (M = 21.67, SD = 17.21, n = 8). Conclusion: Most C3I programs assessed were technically inefficient relative to the most efficient center benchmark and may be improved by optimizing the use of inputs (e.g., cost-per-participant) relative to program outcomes (e.g., reach, effectiveness). This study demonstrates the appropriateness and feasibility of using DEA to evaluate the relative performance of evidence-based programs.

KW - Cancer

KW - Data envelopment analysis

KW - Efficiency

KW - Implementation costs

KW - Implementation science

KW - Program performance

KW - Smoking cessation

KW - Tobacco treatment

UR - http://www.scopus.com/inward/record.url?scp=85165299810&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85165299810&partnerID=8YFLogxK

U2 - 10.1186/s43058-023-00433-3

DO - 10.1186/s43058-023-00433-3

M3 - Article

C2 - 37170381

AN - SCOPUS:85165299810

SN - 2662-2211

VL - 4

JO - Implementation Science Communications

JF - Implementation Science Communications

IS - 1

M1 - 50

ER -

Data envelopment analysis to evaluate the efficiency of tobacco treatment programs in the NCI Moonshot Cancer Center Cessation Initiative

Abstract

Funding

Keywords

ASJC Scopus subject areas

UN SDGs

Access to Document

Other files and links

Fingerprint