Evaluating Community-Based Translational Interventions Using Historical Controls: Propensity Score vs. Disease Risk Score Approach

the Special Diabetes Program for Indians Diabetes Prevention Demonstration Project; Grant programs participating in the Special Diabetes Program for Indians Diabetes Prevention Program are as follows: Confederated Tribes of the Chehalis Reservation, Cherokee Nation, Cheyenne River Sioux Tribe, the Chickasaw Nation, Coeur d’Alene Tribe, Colorado River Indian Tribes, Colville Confederated Tribes, Cow Creek Band of Umpqua Tribe, Klamath Tribes, and Coquille Tribe, Fond du Lac Reservation, Gila River Health Care, Haskell Health Center, Ho-Chunk Nation, Indian Health Board of Minneapolis, Indian Health Center of Santa Clara Valley, Native American Rehabilitation Association of the NW, Hunter Health, Kenaitze Indian Tribe IRA, Lawton IHS Service Unit, Menominee Indian Tribe of Wisconsin, Mississippi Band of Choctaw Indians, Norton Sound Health Corporation, Pine Ridge IHS Service Unit, Pueblo of San Felipe, Quinault Indian Nation, Rapid City IHS Diabetes Program, Red Lake Comprehensive Health Services, Rocky Boy Health Board, Seneca Nation of Indians, Sonoma County Indian Health Project, South E...

doi:10.1007/s11121-019-0980-3

Evaluating Community-Based Translational Interventions Using Historical Controls: Propensity Score vs. Disease Risk Score Approach

the Special Diabetes Program for Indians Diabetes Prevention Demonstration Project, Grant programs participating in the Special Diabetes Program for Indians Diabetes Prevention Program are as follows: Confederated Tribes of the Chehalis Reservation, Cherokee Nation, Cheyenne River Sioux Tribe, the Chickasaw Nation, Coeur d’Alene Tribe, Colorado River Indian Tribes, Colville Confederated Tribes, Cow Creek Band of Umpqua Tribe, Klamath Tribes, and Coquille Tribe, Fond du Lac Reservation, Gila River Health Care, Haskell Health Center, Ho-Chunk Nation, Indian Health Board of Minneapolis, Indian Health Center of Santa Clara Valley, Native American Rehabilitation Association of the NW, Hunter Health, Kenaitze Indian Tribe IRA, Lawton IHS Service Unit, Menominee Indian Tribe of Wisconsin, Mississippi Band of Choctaw Indians, Norton Sound Health Corporation, Pine Ridge IHS Service Unit, Pueblo of San Felipe, Quinault Indian Nation, Rapid City IHS Diabetes Program, Red Lake Comprehensive Health Services, Rocky Boy Health Board, Seneca Nation of Indians, Sonoma County Indian Health Project, South E...

Preventive Medicine

Research output: Contribution to journal › Article › peer-review

1 Scopus citations

Abstract

Many community-based translations of evidence-based interventions are designed as one-arm studies due to ethical and other considerations. Evaluating the impacts of such programs is challenging. Here, we examine the effectiveness of the lifestyle intervention implemented by the Special Diabetes Program for Indians Diabetes Prevention (SDPI-DP) demonstration project, a translational lifestyle intervention among American Indian and Alaska Native communities. Data from the landmark Diabetes Prevention Program placebo group was used as a historical control. We compared the use of propensity score (PS) and disease risk score (DRS) matching to adjust for potential confounder imbalance between groups. The unadjusted hazard ratio (HR) for diabetes risk was 0.35 for SDPI-DP lifestyle intervention vs. control. However, when relevant diabetes risk factors were considered, the adjusted HR estimates were attenuated toward 1, ranging from 0.56 (95% CI 0.44–0.71) to 0.69 (95% CI 0.56–0.96). The differences in estimated HRs using the PS and DRS approaches were relatively small but DRS matching resulted in more participants being matched and smaller standard errors of effect estimates. Carefully employed, publicly available randomized clinical trial data can be used as a historical control to evaluate the intervention effectiveness of one-arm community translational initiatives. It is critical to use a proper statistical method to balance the distributions of potential confounders between comparison groups in this kind of evaluations.

Original language	English (US)
Pages (from-to)	598-608
Number of pages	11
Journal	Prevention Science
Volume	20
Issue number	4
DOIs	https://doi.org/10.1007/s11121-019-0980-3
State	Published - May 15 2019

Funding

The authors would like to express our gratitude to the Indian Health Service (IHS) as well as tribal and urban Indian health programs and participants involved in the SDPI-DP. The findings and conclusions in this article are those of the authors and do not necessarily represent the official position of the IHS. The DPP was conducted by the DPP Investigators and supported by the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK). This manuscript was not prepared in collaboration with the Investigators of the DPP study and does not necessarily reflect the opinions or views of the DPP study or the NIDDK. The authors would also like to thank Dr. James Hill for his valuable scientific suggestions and comments. Grant programs participating in the Special Diabetes Program for Indians Diabetes Prevention Program are as follows: Confederated Tribes of the Chehalis Reservation, Cherokee Nation, Cheyenne River Sioux Tribe, the Chickasaw Nation, Coeur d?Alene Tribe, Colorado River Indian Tribes, Colville Confederated Tribes, Cow Creek Band of Umpqua Tribe, Klamath Tribes, and Coquille Tribe, Fond du Lac Reservation, Gila River Health Care, Haskell Health Center, Ho-Chunk Nation, Indian Health Board of Minneapolis, Indian Health Center of Santa Clara Valley, Native American Rehabilitation Association of the NW, Hunter Health, Kenaitze Indian Tribe IRA, Lawton IHS Service Unit, Menominee Indian Tribe of Wisconsin, Mississippi Band of Choctaw Indians, Norton Sound Health Corporation, Pine Ridge IHS Service Unit, Pueblo of San Felipe, Quinault Indian Nation, Rapid City IHS Diabetes Program, Red Lake Comprehensive Health Services, Rocky Boy Health Board, Seneca Nation of Indians, Sonoma County Indian Health Project, South East Alaska Regional Health Consortium, Southcentral Foundation, Trenton Indian Service Area, Tuba City Regional Health Care Corporation, United American Indian Involvement, Inc., United Indian Health Services, Inc., Warm Springs Health & Wellness Center, Winnebago Tribe of Nebraska, Zuni Pueblo. Funding Funding for the SDPI-DP project was provided by the Indian Health Service (HHSI242200400049C, S.M. Manson). Manuscript preparation was supported in part by American Diabetes Association (ADA #7-12-CT-36, L. Jiang), and National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) (1P30DK092923, S.M. Manson, and R21DK108187, L. Jiang).

Keywords

Comparative effectiveness evaluation
Disease risk score
Historical controls
Prognostic score
Propensity score
Single-arm intervention

ASJC Scopus subject areas

Public Health, Environmental and Occupational Health

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1007/s11121-019-0980-3

Cite this

the Special Diabetes Program for Indians Diabetes Prevention Demonstration Project, & Grant programs participating in the Special Diabetes Program for Indians Diabetes Prevention Program are as follows: Confederated Tribes of the Chehalis Reservation, Cherokee Nation, Cheyenne River Sioux Tribe, the Chickasaw Nation, Coeur d’Alene Tribe, Colorado River Indian Tribes, Colville Confederated Tribes, Cow Creek Band of Umpqua Tribe, Klamath Tribes, and Coquille Tribe, Fond du Lac Reservation, Gila River Health Care, Haskell Health Center, Ho-Chunk Nation, Indian Health Board of Minneapolis, Indian Health Center of Santa Clara Valley, Native American Rehabilitation Association of the NW, Hunter Health, Kenaitze Indian Tribe IRA, Lawton IHS Service Unit, Menominee Indian Tribe of Wisconsin, Mississippi Band of Choctaw Indians, Norton Sound Health Corporation, Pine Ridge IHS Service Unit, Pueblo of San Felipe, Quinault Indian Nation, Rapid City IHS Diabetes Program, Red Lake Comprehensive Health Services, Rocky Boy Health Board, Seneca Nation of Indians, Sonoma County Indian Health Project, South E... (2019). Evaluating Community-Based Translational Interventions Using Historical Controls: Propensity Score vs. Disease Risk Score Approach. Prevention Science, 20(4), 598-608. https://doi.org/10.1007/s11121-019-0980-3

the Special Diabetes Program for Indians Diabetes Prevention Demonstration Project ; Grant programs participating in the Special Diabetes Program for Indians Diabetes Prevention Program are as follows: Confederated Tribes of the Chehalis Reservation, Cherokee Nation, Cheyenne River Sioux Tribe, the Chickasaw Nation, Coeur d’Alene Tribe, Colorado River Indian Tribes, Colville Confederated Tribes, Cow Creek Band of Umpqua Tribe, Klamath Tribes, and Coquille Tribe, Fond du Lac Reservation, Gila River Health Care, Haskell Health Center, Ho-Chunk Nation, Indian Health Board of Minneapolis, Indian Health Center of Santa Clara Valley, Native American Rehabilitation Association of the NW, Hunter Health, Kenaitze Indian Tribe IRA, Lawton IHS Service Unit, Menominee Indian Tribe of Wisconsin, Mississippi Band of Choctaw Indians, Norton Sound Health Corporation, Pine Ridge IHS Service Unit, Pueblo of San Felipe, Quinault Indian Nation, Rapid City IHS Diabetes Program, Red Lake Comprehensive Health Services, Rocky Boy Health Board, Seneca Nation of Indians, Sonoma County Indian Health Project, South E... / Evaluating Community-Based Translational Interventions Using Historical Controls : Propensity Score vs. Disease Risk Score Approach. In: Prevention Science. 2019 ; Vol. 20, No. 4. pp. 598-608.

@article{55f5ba763b4a441c86fe0533a3be01ff,

title = "Evaluating Community-Based Translational Interventions Using Historical Controls: Propensity Score vs. Disease Risk Score Approach",

abstract = "Many community-based translations of evidence-based interventions are designed as one-arm studies due to ethical and other considerations. Evaluating the impacts of such programs is challenging. Here, we examine the effectiveness of the lifestyle intervention implemented by the Special Diabetes Program for Indians Diabetes Prevention (SDPI-DP) demonstration project, a translational lifestyle intervention among American Indian and Alaska Native communities. Data from the landmark Diabetes Prevention Program placebo group was used as a historical control. We compared the use of propensity score (PS) and disease risk score (DRS) matching to adjust for potential confounder imbalance between groups. The unadjusted hazard ratio (HR) for diabetes risk was 0.35 for SDPI-DP lifestyle intervention vs. control. However, when relevant diabetes risk factors were considered, the adjusted HR estimates were attenuated toward 1, ranging from 0.56 (95% CI 0.44–0.71) to 0.69 (95% CI 0.56–0.96). The differences in estimated HRs using the PS and DRS approaches were relatively small but DRS matching resulted in more participants being matched and smaller standard errors of effect estimates. Carefully employed, publicly available randomized clinical trial data can be used as a historical control to evaluate the intervention effectiveness of one-arm community translational initiatives. It is critical to use a proper statistical method to balance the distributions of potential confounders between comparison groups in this kind of evaluations.",

keywords = "Comparative effectiveness evaluation, Disease risk score, Historical controls, Prognostic score, Propensity score, Single-arm intervention",

author = "{the Special Diabetes Program for Indians Diabetes Prevention Demonstration Project} and {Grant programs participating in the Special Diabetes Program for Indians Diabetes Prevention Program are as follows: Confederated Tribes of the Chehalis Reservation, Cherokee Nation, Cheyenne River Sioux Tribe, the Chickasaw Nation, Coeur d{\textquoteright}Alene Tribe, Colorado River Indian Tribes, Colville Confederated Tribes, Cow Creek Band of Umpqua Tribe, Klamath Tribes, and Coquille Tribe, Fond du Lac Reservation, Gila River Health Care, Haskell Health Center, Ho-Chunk Nation, Indian Health Board of Minneapolis, Indian Health Center of Santa Clara Valley, Native American Rehabilitation Association of the NW, Hunter Health, Kenaitze Indian Tribe IRA, Lawton IHS Service Unit, Menominee Indian Tribe of Wisconsin, Mississippi Band of Choctaw Indians, Norton Sound Health Corporation, Pine Ridge IHS Service Unit, Pueblo of San Felipe, Quinault Indian Nation, Rapid City IHS Diabetes Program, Red Lake Comprehensive Health Services, Rocky Boy Health Board, Seneca Nation of Indians, Sonoma County Indian Health Project, South E...} and Luohua Jiang and Shuai Chen and Janette Beals and Juned Siddique and Hamman, {Richard F.} and Ann Bullock and Manson, {Spero M.}",

note = "Funding Information: Acknowledgments The authors would like to express our gratitude to the Indian Health Service (IHS) as well as tribal and urban Indian health programs and participants involved in the SDPI-DP. The findings and conclusions in this article are those of the authors and do not necessarily represent the official position of the IHS. The DPP was conducted by the DPP Investigators and supported by the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK). This manuscript was not prepared in collaboration with the Investigators of the DPP study and does not necessarily reflect the opinions or views of the DPP study or the NIDDK. The authors would also like to thank Dr. James Hill for his valuable scientific suggestions and comments. Funding Information: Funding Funding for the SDPI-DP project was provided by the Indian Health Service (HHSI242200400049C, S.M. Manson). Manuscript preparation was supported in part by American Diabetes Association (ADA #7-12-CT-36, L. Jiang), and National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) (1P30DK092923, S.M. Manson, and R21DK108187, L. Jiang).",

year = "2019",

month = may,

day = "15",

doi = "10.1007/s11121-019-0980-3",

language = "English (US)",

volume = "20",

pages = "598--608",

journal = "Prevention Science",

issn = "1389-4986",

publisher = "Springer New York",

number = "4",

}

the Special Diabetes Program for Indians Diabetes Prevention Demonstration Project & Grant programs participating in the Special Diabetes Program for Indians Diabetes Prevention Program are as follows: Confederated Tribes of the Chehalis Reservation, Cherokee Nation, Cheyenne River Sioux Tribe, the Chickasaw Nation, Coeur d’Alene Tribe, Colorado River Indian Tribes, Colville Confederated Tribes, Cow Creek Band of Umpqua Tribe, Klamath Tribes, and Coquille Tribe, Fond du Lac Reservation, Gila River Health Care, Haskell Health Center, Ho-Chunk Nation, Indian Health Board of Minneapolis, Indian Health Center of Santa Clara Valley, Native American Rehabilitation Association of the NW, Hunter Health, Kenaitze Indian Tribe IRA, Lawton IHS Service Unit, Menominee Indian Tribe of Wisconsin, Mississippi Band of Choctaw Indians, Norton Sound Health Corporation, Pine Ridge IHS Service Unit, Pueblo of San Felipe, Quinault Indian Nation, Rapid City IHS Diabetes Program, Red Lake Comprehensive Health Services, Rocky Boy Health Board, Seneca Nation of Indians, Sonoma County Indian Health Project, South E... 2019, 'Evaluating Community-Based Translational Interventions Using Historical Controls: Propensity Score vs. Disease Risk Score Approach', Prevention Science, vol. 20, no. 4, pp. 598-608. https://doi.org/10.1007/s11121-019-0980-3

Evaluating Community-Based Translational Interventions Using Historical Controls: Propensity Score vs. Disease Risk Score Approach. / the Special Diabetes Program for Indians Diabetes Prevention Demonstration Project; Grant programs participating in the Special Diabetes Program for Indians Diabetes Prevention Program are as follows: Confederated Tribes of the Chehalis Reservation, Cherokee Nation, Cheyenne River Sioux Tribe, the Chickasaw Nation, Coeur d’Alene Tribe, Colorado River Indian Tribes, Colville Confederated Tribes, Cow Creek Band of Umpqua Tribe, Klamath Tribes, and Coquille Tribe, Fond du Lac Reservation, Gila River Health Care, Haskell Health Center, Ho-Chunk Nation, Indian Health Board of Minneapolis, Indian Health Center of Santa Clara Valley, Native American Rehabilitation Association of the NW, Hunter Health, Kenaitze Indian Tribe IRA, Lawton IHS Service Unit, Menominee Indian Tribe of Wisconsin, Mississippi Band of Choctaw Indians, Norton Sound Health Corporation, Pine Ridge IHS Service Unit, Pueblo of San Felipe, Quinault Indian Nation, Rapid City IHS Diabetes Program, Red Lake Comprehensive Health Services, Rocky Boy Health Board, Seneca Nation of Indians, Sonoma County Indian Health Project, South E...
In: Prevention Science, Vol. 20, No. 4, 15.05.2019, p. 598-608.

Research output: Contribution to journal › Article › peer-review

TY - JOUR

T1 - Evaluating Community-Based Translational Interventions Using Historical Controls

T2 - Propensity Score vs. Disease Risk Score Approach

AU - the Special Diabetes Program for Indians Diabetes Prevention Demonstration Project

AU - Grant programs participating in the Special Diabetes Program for Indians Diabetes Prevention Program are as follows: Confederated Tribes of the Chehalis Reservation, Cherokee Nation, Cheyenne River Sioux Tribe, the Chickasaw Nation, Coeur d’Alene Tribe, C

AU - Jiang, Luohua

AU - Chen, Shuai

AU - Beals, Janette

AU - Siddique, Juned

AU - Hamman, Richard F.

AU - Bullock, Ann

AU - Manson, Spero M.

N1 - Funding Information: Acknowledgments The authors would like to express our gratitude to the Indian Health Service (IHS) as well as tribal and urban Indian health programs and participants involved in the SDPI-DP. The findings and conclusions in this article are those of the authors and do not necessarily represent the official position of the IHS. The DPP was conducted by the DPP Investigators and supported by the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK). This manuscript was not prepared in collaboration with the Investigators of the DPP study and does not necessarily reflect the opinions or views of the DPP study or the NIDDK. The authors would also like to thank Dr. James Hill for his valuable scientific suggestions and comments. Funding Information: Funding Funding for the SDPI-DP project was provided by the Indian Health Service (HHSI242200400049C, S.M. Manson). Manuscript preparation was supported in part by American Diabetes Association (ADA #7-12-CT-36, L. Jiang), and National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) (1P30DK092923, S.M. Manson, and R21DK108187, L. Jiang).

PY - 2019/5/15

Y1 - 2019/5/15

N2 - Many community-based translations of evidence-based interventions are designed as one-arm studies due to ethical and other considerations. Evaluating the impacts of such programs is challenging. Here, we examine the effectiveness of the lifestyle intervention implemented by the Special Diabetes Program for Indians Diabetes Prevention (SDPI-DP) demonstration project, a translational lifestyle intervention among American Indian and Alaska Native communities. Data from the landmark Diabetes Prevention Program placebo group was used as a historical control. We compared the use of propensity score (PS) and disease risk score (DRS) matching to adjust for potential confounder imbalance between groups. The unadjusted hazard ratio (HR) for diabetes risk was 0.35 for SDPI-DP lifestyle intervention vs. control. However, when relevant diabetes risk factors were considered, the adjusted HR estimates were attenuated toward 1, ranging from 0.56 (95% CI 0.44–0.71) to 0.69 (95% CI 0.56–0.96). The differences in estimated HRs using the PS and DRS approaches were relatively small but DRS matching resulted in more participants being matched and smaller standard errors of effect estimates. Carefully employed, publicly available randomized clinical trial data can be used as a historical control to evaluate the intervention effectiveness of one-arm community translational initiatives. It is critical to use a proper statistical method to balance the distributions of potential confounders between comparison groups in this kind of evaluations.

AB - Many community-based translations of evidence-based interventions are designed as one-arm studies due to ethical and other considerations. Evaluating the impacts of such programs is challenging. Here, we examine the effectiveness of the lifestyle intervention implemented by the Special Diabetes Program for Indians Diabetes Prevention (SDPI-DP) demonstration project, a translational lifestyle intervention among American Indian and Alaska Native communities. Data from the landmark Diabetes Prevention Program placebo group was used as a historical control. We compared the use of propensity score (PS) and disease risk score (DRS) matching to adjust for potential confounder imbalance between groups. The unadjusted hazard ratio (HR) for diabetes risk was 0.35 for SDPI-DP lifestyle intervention vs. control. However, when relevant diabetes risk factors were considered, the adjusted HR estimates were attenuated toward 1, ranging from 0.56 (95% CI 0.44–0.71) to 0.69 (95% CI 0.56–0.96). The differences in estimated HRs using the PS and DRS approaches were relatively small but DRS matching resulted in more participants being matched and smaller standard errors of effect estimates. Carefully employed, publicly available randomized clinical trial data can be used as a historical control to evaluate the intervention effectiveness of one-arm community translational initiatives. It is critical to use a proper statistical method to balance the distributions of potential confounders between comparison groups in this kind of evaluations.

KW - Comparative effectiveness evaluation

KW - Disease risk score

KW - Historical controls

KW - Prognostic score

KW - Propensity score

KW - Single-arm intervention

UR - http://www.scopus.com/inward/record.url?scp=85061491236&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85061491236&partnerID=8YFLogxK

U2 - 10.1007/s11121-019-0980-3

DO - 10.1007/s11121-019-0980-3

M3 - Article

C2 - 30747394

AN - SCOPUS:85061491236

SN - 1389-4986

VL - 20

SP - 598

EP - 608

JO - Prevention Science

JF - Prevention Science

IS - 4

ER -

the Special Diabetes Program for Indians Diabetes Prevention Demonstration Project, Grant programs participating in the Special Diabetes Program for Indians Diabetes Prevention Program are as follows: Confederated Tribes of the Chehalis Reservation, Cherokee Nation, Cheyenne River Sioux Tribe, the Chickasaw Nation, Coeur d’Alene Tribe, Colorado River Indian Tribes, Colville Confederated Tribes, Cow Creek Band of Umpqua Tribe, Klamath Tribes, and Coquille Tribe, Fond du Lac Reservation, Gila River Health Care, Haskell Health Center, Ho-Chunk Nation, Indian Health Board of Minneapolis, Indian Health Center of Santa Clara Valley, Native American Rehabilitation Association of the NW, Hunter Health, Kenaitze Indian Tribe IRA, Lawton IHS Service Unit, Menominee Indian Tribe of Wisconsin, Mississippi Band of Choctaw Indians, Norton Sound Health Corporation, Pine Ridge IHS Service Unit, Pueblo of San Felipe, Quinault Indian Nation, Rapid City IHS Diabetes Program, Red Lake Comprehensive Health Services, Rocky Boy Health Board, Seneca Nation of Indians, Sonoma County Indian Health Project, South E... Evaluating Community-Based Translational Interventions Using Historical Controls: Propensity Score vs. Disease Risk Score Approach. Prevention Science. 2019 May 15;20(4):598-608. doi: 10.1007/s11121-019-0980-3

Evaluating Community-Based Translational Interventions Using Historical Controls: Propensity Score vs. Disease Risk Score Approach

Abstract

Funding

Keywords

ASJC Scopus subject areas

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this