Year 4 Center for the Evaluation of the Chicago HIV Services Portfolio

I. Theoretical Approach: The Evaluation Center team will use a Systems-Informed Empowerment Evaluation (SIEE) framework for the multi-site evaluation. SIEE first utilizes complex, multi-level methodology (e.g., network analysis, implementation science, agent-based modeling) to measure the tangible impacts that a community’s social and structural environment has on individuals impacted by HIV. Given the unique social and structural factors of the City of Chicago, this approach will allow for a more rigorous interpretation of evaluation findings and increase the usability of findings moving forward by placing observations and program impact in context. This framework will be particularly crucial as we work to evaluate and support implementation of the programming at delegate agencies selected as part of this task order’s companion project. The second key component of the SIEE framework is building capacity at agencies to be able to strategically evaluate their own programming using the theory of empowerment. Capacity building is accomplished through a number of important activities including technical assistance provision, stakeholder engagement in the evaluation process, and high-impact collaborative dissemination activities. Our team has extensive experience using this approach to conduct multi-site evaluations of HIV care and prevention demonstration projects within the City of Chicago, as evidenced within our last four years of funding from the Chicago Department of Public Health to oversee the implementation and evaluation of diverse HIV prevention projects. As this project will in many ways extend and build on our prior partnerships, we plan to use lessons learned from these experiences to inform this initiative. II. Evaluation Methodology: The Evaluation Center team will work with sites to iteratively design a multi-level mixed methods evaluation of their HIV services programming. Based on the needs of the site and the nature of their programming, the Evaluation Center team will identify evaluative components that are most relevant to their specific process in order to achieve comprehensive and objective evaluation while simultaneously encouraging program success. The following are examples of what these evaluation activities could entail, but are not an exhaustive list: a. Single-Site Evaluation i. Needs Assessment – Given the particular needs of each organization’s programming implementation plan, the Evaluation Center is able to provide guidance and assistance in conducting a comprehensive needs assessment to appropriately identify and target high-impact areas of need in underserved communities (e.g., Black men who have sex with men [BMSM], transgender women). This could include a combination of quantitative surveys, qualitative focus groups with members of the community, and community engagement studios with individuals currently engaged in care. In addition, this will include an initial in-person meeting with delegate agency representatives to assess capacity for implementation and evaluation, followed by at least bi-monthly check-ins to assess changes in needs over time. These data sources will be integrated with available surveillance data to ensure the greatest amount of information available to inform programming implementation and evaluation. ii. Process – Process evaluation will be conducted to ensure fidelity in the implementation of project activities, and the overall acceptability of the programming. This could include the creation of fidelity assessment tools, interviews with program clients, and interviews