Evaluation of Chicago’s Community HIV Services Portfolio

I. Theoretical Approach: The Evaluation Center team will use a Systems-Informed Empowerment Evaluation (SIEE) framework for the multi-site evaluation. SIEE first utilizes complex, multi-level methodology (e.g., network analysis, implementation science, agent-based modeling) to measure the tangible impacts that a community’s social and structural environment has on individuals impacted by HIV. Given the unique social and structural factors of the City of Chicago, this approach will allow for a more rigorous interpretation of evaluation findings and increase the usability of findings moving forward by placing observations and program impact in context. This framework will be particularly crucial as we promote replication of effective evidence informed interventions (EIIs) and models of care to improve the City of Chicago’s response to the HIV epidemic through community development. The second key component of the SIEE framework is building capacity at agencies to be able to strategically plan, implement, and evaluate programming using the theory of empowerment. Capacity building is accomplished through a number of important activities including technical assistance provision, stakeholder engagement in the evaluation process, and high-impact collaborative dissemination activities. Our team has extensive experience using this approach to conduct multi-site evaluations of HIV care and prevention demonstration projects within the City of Chicago, as evidenced within our last four years of funding from the Chicago Department of Public Health to oversee the implementation and evaluation of diverse HIV prevention projects. Therefore, we plan to use lessons learned from these experiences to inform this initiative. II. Evaluation Methodology: The Evaluation Center team will work with sites to iteratively design a multi-level mixed methods evaluation of their community development programming. Based on the needs of the site and the nature of their programming, the Evaluation Center team will work to identify evaluative components that are most relevant to their specific process in order to achieve comprehensive and objective evaluation while simultaneously encouraging program success. The following are examples of what these evaluation activities could entail, but are not an exhaustive list: a. Single-Site Evaluation i. Needs Assessment – Given the particular needs of each organization’s programming implementation plan, the Evaluation Center is able to provide guidance and assistance in conducting a comprehensive needs assessment to appropriately identify high-impact areas of need in underserved communities (e.g., Black men who have sex with men [BMSM], transgender women). This could include a combination of quantitative surveys, qualitative focus groups with members of the community, and community engagement studios with individuals currently engaged in care. Further, these data sources will be integrated with available surveillance data to ensure the greatest amount of information available to inform programming development and implementation. ii. Process – Process evaluation will be conducted to ensure fidelity in the implementation of project activities, and the overall acceptability of the programming. This could include the creation of fidelity assessment tools, interviews with program clients, and interviews with those implementing the program. iii. Outcome – While the primary method of outcome evaluation utilized will depend on the unique programming implemented at each site, the Evaluation Center will work closely with project stak