ProvBuild: Improving Data Scientist Efficiency with Provenance (An Extended Abstract)

Jingmei Hu, Jiwon Joung, Maia Jacobs, Krzysztof Z. Gajos, Margo I. Seltzer

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Data scientists frequently analyze data by writing scripts. We conducted a contextual inquiry with interdisciplinary researchers, which revealed that parameter tuning is a highly iterative process and that debugging is time-consuming. As analysis scripts evolve and become more complex, analysts have difficulty conceptualizing their workflow. In particular, after editing a script, it becomes difficult to determine precisely which code blocks depend on the edit. Consequently, scientists frequently re-run entire scripts instead of re-running only the necessary parts. We present ProvBuild, a data analysis environment that uses change impact analysis [1] to improve the iterative debugging process in script-based workflow pipelines. ProvBuild is a tool that leverages language-level provenance [2] to streamline the debugging process by reducing programmer cognitive load and decreasing subsequent runtimes, leading to an overall reduction in elapsed debugging time. ProvBuild uses provenance to track dependencies in a script. When an analyst debugs a script, ProvBuild generates a simplified script that contains only the information necessary to debug a particular problem. We demonstrate that debugging the simplified script lowers a programmer's cognitive load and permits faster re-execution when testing changes. The combination of reduced cognitive load and shorter runtime reduces the time necessary to debug a script. We quantitatively and qualitatively show that even though ProvBuild introduces overhead during a script's first execution, it is a more efficient way for users to debug and tune complex workflows. ProvBuild demonstrates a novel use of language-level provenance, in which it is used to proactively improve programmer productively rather than merely providing a way to retroactively gain insight into a body of code. To the best of our knowledge, ProvBuild is a novel application of change impact analysis and it is the first debugging tool to leverage language-level provenance to reduce cognitive load and execution time.

Original languageEnglish (US)
Title of host publicationProceedings - 2020 ACM/IEEE 42nd International Conference on Software Engineering
Subtitle of host publicationCompanion, ICSE-Companion 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages266-267
Number of pages2
ISBN (Electronic)9781450371223
DOIs
StatePublished - Oct 2020
Externally publishedYes
Event42nd ACM/IEEE International Conference on Software Engineering: Companion, ICSE-Companion 2020 - Seoul, Korea, Republic of
Duration: Jun 27 2020Jul 19 2020

Publication series

NameProceedings - 2020 ACM/IEEE 42nd International Conference on Software Engineering: Companion, ICSE-Companion 2020

Conference

Conference42nd ACM/IEEE International Conference on Software Engineering: Companion, ICSE-Companion 2020
CountryKorea, Republic of
CitySeoul
Period6/27/207/19/20

ASJC Scopus subject areas

  • Artificial Intelligence
  • Software
  • Safety, Risk, Reliability and Quality

Fingerprint Dive into the research topics of 'ProvBuild: Improving Data Scientist Efficiency with Provenance (An Extended Abstract)'. Together they form a unique fingerprint.

Cite this