The popularity of hackathons has increased as technology pervades more facets of our lives. Originally designed for programmers, hackathons are now being appropriated by new stakeholders across diverse sectors. Yet with this evolution in hackathons, we no longer adequately understand what is produced and, thereby, the value of these events. We conducted an interview study with 22 stakeholders - participants, representatives of nonprofit organizations, and organizers - of the CHI4Good Day of Service to understand what is produced through philanthropic hackathons. Whereas traditional hackathons are oriented around the production of code or prototypes, our analysis of interview data suggests that the production work of philanthropic hackathons also includes technical capacity and expertise, expanded social networks, an exposure to design process, affective experiences, and an opportunity for participants to shape their identities against a cross-sectoral, interdisciplinary backdrop. We conclude by reflecting on implications for the CHI community in carrying out philanthropic events styled after hackathons.