A case study of tuning MapReduce for efficient Bioinformatics in the cloud

Lizhen Shi*, Zhong Wang, Weikuan Yu, Xiandong Meng

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

11 Scopus citations


The combination of the Hadoop MapReduce programming model and cloud computing allows biological scientists to analyze next-generation sequencing (NGS) data in a timely and cost-effective manner. Cloud computing platforms remove the burden of IT facility procurement and management from end users and provide ease of access to Hadoop clusters. However, biological scientists are still expected to choose appropriate Hadoop parameters for running their jobs. More importantly, the available Hadoop tuning guidelines are either obsolete or too general to capture the particular characteristics of bioinformatics applications. In this study, we aim to minimize the cloud computing cost spent on bioinformatics data analysis by optimizing the extracted significant Hadoop parameters. When using MapReduce-based bioinformatics tools in the cloud, the default settings often lead to resource underutilization and wasteful expenses. We choose k-mer counting, a representative application used in a large number of NGS data analysis tools, as our study case. Experimental results show that, with the fine-tuned parameters, we achieve a total of 4× speedup compared with the original performance (using the default settings). This paper presents an exemplary case for tuning MapReduce-based bioinformatics applications in the cloud, and documents the key parameters that could lead to significant performance benefits.

Original languageEnglish (US)
Pages (from-to)83-95
Number of pages13
JournalParallel Computing
StatePublished - Jan 1 2017


  • Hadoop
  • K-mer counting
  • NGS
  • Parameter optimization
  • YARN

ASJC Scopus subject areas

  • Software
  • Theoretical Computer Science
  • Hardware and Architecture
  • Computer Networks and Communications
  • Computer Graphics and Computer-Aided Design
  • Artificial Intelligence


Dive into the research topics of 'A case study of tuning MapReduce for efficient Bioinformatics in the cloud'. Together they form a unique fingerprint.

Cite this