The BigDAWG polystore system and architecture

Vijay Gadepally*, Peinan Chen, Jennie Duggan, Aaron Elmore, Brandon Haynes, Jeremy Kepner, Samuel Madden, Tim Mattson, Michael Stonebraker

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

83 Scopus citations

Abstract

Organizations are often faced with the challenge of providing data management solutions for large, heterogenous datasets that may have different underlying data and programming models. For example, a medical dataset may have unstructured text, relational data, time series waveforms and imagery. Trying to fit such datasets in a single data management system can have adverse performance and efficiency effects. As a part of the Intel Science and Technology Center on Big Data, we are developing a polystore system designed for such problems. BigDAWG (short for the Big Data Analytics Working Group) is a polystore system designed to work on complex problems that naturally span across different processing or storage engines. BigDAWG provides an architecture that supports diverse database systems working with different data models, support for the competing notions of location transparency and semantic completeness via islands and a middleware that provides a uniform multi-island interface. Initial results from a prototype of the BigDAWG system applied to a medical dataset validate polystore concepts. In this article, we will describe polystore databases, the current BigDAWG architecture and its application on the MIMIC II medical dataset, initial performance results and our future development plans.

Original languageEnglish (US)
Title of host publication2016 IEEE High Performance Extreme Computing Conference, HPEC 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781509035250
DOIs
StatePublished - Nov 28 2016
Event2016 IEEE High Performance Extreme Computing Conference, HPEC 2016 - Waltham, United States
Duration: Sep 13 2016Sep 15 2016

Publication series

Name2016 IEEE High Performance Extreme Computing Conference, HPEC 2016

Other

Other2016 IEEE High Performance Extreme Computing Conference, HPEC 2016
Country/TerritoryUnited States
CityWaltham
Period9/13/169/15/16

ASJC Scopus subject areas

  • Computer Science (miscellaneous)
  • Hardware and Architecture
  • Computational Mathematics

Fingerprint

Dive into the research topics of 'The BigDAWG polystore system and architecture'. Together they form a unique fingerprint.

Cite this