Assessing technical performance in differential gene expression experiments with external spike-in RNA control ratio mixtures

Sarah A. Munro; Steven P. Lund; P. Scott Pine; Hans Binder; Djork Arné Clevert; Ana Conesa; Joaquin Dopazo; Mario Fasold; Sepp Hochreiter; Huixiao Hong; Nadereh Jafari; David P. Kreil; Paweł P. Łabaj; Sheng Li; Yang Liao; Simon M. Lin; Joseph Meehan; Christopher E. Mason; Javier Santoyo-Lopez; Robert A. Setterquist; Leming Shi; Wei Shi; Gordon K. Smyth; Nancy Stralis-Pavese; Zhenqiang Su; Weida Tong; Charles Wang; Jian Wang; Joshua Xu; Zhan Ye; Yong Yang; Ying Yu; Marc Salit

doi:10.1038/ncomms6125

Assessing technical performance in differential gene expression experiments with external spike-in RNA control ratio mixtures

Sarah A. Munro^*, Steven P. Lund, P. Scott Pine, Hans Binder, Djork Arné Clevert, Ana Conesa, Joaquin Dopazo, Mario Fasold, Sepp Hochreiter, Huixiao Hong, Nadereh Jafari, David P. Kreil, Paweł P. Łabaj, Sheng Li, Yang Liao, Simon M. Lin, Joseph Meehan, Christopher E. Mason, Javier Santoyo-Lopez, Robert A. SetterquistLeming Shi, Wei Shi, Gordon K. Smyth, Nancy Stralis-Pavese, Zhenqiang Su, Weida Tong, Charles Wang, Jian Wang, Joshua Xu, Zhan Ye, Yong Yang, Ying Yu, Marc Salit

^*Corresponding author for this work

Center for Genetic Medicine

Research output: Contribution to journal › Article › peer-review

99 Scopus citations

Abstract

There is a critical need for standard approaches to assess, report and compare the technical performance of genome-scale differential gene expression experiments. Here we assess technical performance with a proposed standard 'dashboard' of metrics derived from analysis of external spike-in RNA control ratio mixtures. These control ratio mixtures with defined abundance ratios enable assessment of diagnostic performance of differentially expressed transcript lists, limit of detection of ratio (LODR) estimates and expression ratio variability and measurement bias. The performance metrics suite is applicable to analysis of a typical experiment, and here we also apply these metrics to evaluate technical performance among laboratories. An interlaboratory study using identical samples shared among 12 laboratories with three different measurement processes demonstrates generally consistent diagnostic power across 11 laboratories. Ratio measurement variability and bias are also comparable among laboratories for the same measurement process. We observe different biases for measurement processes using different mRNA-enrichment protocols.

Original language	English (US)
Article number	5125
Journal	Nature communications
Volume	5
DOIs	https://doi.org/10.1038/ncomms6125
State	Published - 2014

Funding

We thank David L. Duewer and Jerod Parsons for review of the manuscript, Cecelie Boysen for discussion of results and all other members of the SEQC consortium who supported this work. P.P.Ł., N.S.-P. and D.P.K. acknowledge the support from the Vienna Scientific Cluster (VSC), the Vienna Science and Technology Fund (WWTF), Baxter AG, Austrian Research Centres (ARC) Seibersdorf and the Austrian Centre of Biopharmaceutical Technology (ACBT). J.D. was supported by grant BIO2011–27069 from the Spanish Ministry of Economy and Competitiveness (MINECO). L.S. and Y.Yu acknowledge support from China’s National Supercomputing Center in Guangzhou.

ASJC Scopus subject areas

General Chemistry
General Biochemistry, Genetics and Molecular Biology
General Physics and Astronomy

Access to Document

10.1038/ncomms6125

Cite this

Munro, S. A., Lund, S. P., Pine, P. S., Binder, H., Clevert, D. A., Conesa, A., Dopazo, J., Fasold, M., Hochreiter, S., Hong, H., Jafari, N., Kreil, D. P., Łabaj, P. P., Li, S., Liao, Y., Lin, S. M., Meehan, J., Mason, C. E., Santoyo-Lopez, J., ... Salit, M. (2014). Assessing technical performance in differential gene expression experiments with external spike-in RNA control ratio mixtures. Nature communications, 5, Article 5125. https://doi.org/10.1038/ncomms6125

@article{6b8657e8a80c460fb235d6fa2cb977cb,

title = "Assessing technical performance in differential gene expression experiments with external spike-in RNA control ratio mixtures",

abstract = "There is a critical need for standard approaches to assess, report and compare the technical performance of genome-scale differential gene expression experiments. Here we assess technical performance with a proposed standard 'dashboard' of metrics derived from analysis of external spike-in RNA control ratio mixtures. These control ratio mixtures with defined abundance ratios enable assessment of diagnostic performance of differentially expressed transcript lists, limit of detection of ratio (LODR) estimates and expression ratio variability and measurement bias. The performance metrics suite is applicable to analysis of a typical experiment, and here we also apply these metrics to evaluate technical performance among laboratories. An interlaboratory study using identical samples shared among 12 laboratories with three different measurement processes demonstrates generally consistent diagnostic power across 11 laboratories. Ratio measurement variability and bias are also comparable among laboratories for the same measurement process. We observe different biases for measurement processes using different mRNA-enrichment protocols.",

author = "Munro, {Sarah A.} and Lund, {Steven P.} and Pine, {P. Scott} and Hans Binder and Clevert, {Djork Arn{\'e}} and Ana Conesa and Joaquin Dopazo and Mario Fasold and Sepp Hochreiter and Huixiao Hong and Nadereh Jafari and Kreil, {David P.} and {\L}abaj, {Pawe{\l} P.} and Sheng Li and Yang Liao and Lin, {Simon M.} and Joseph Meehan and Mason, {Christopher E.} and Javier Santoyo-Lopez and Setterquist, {Robert A.} and Leming Shi and Wei Shi and Smyth, {Gordon K.} and Nancy Stralis-Pavese and Zhenqiang Su and Weida Tong and Charles Wang and Jian Wang and Joshua Xu and Zhan Ye and Yong Yang and Ying Yu and Marc Salit",

year = "2014",

doi = "10.1038/ncomms6125",

language = "English (US)",

volume = "5",

journal = "Nature communications",

issn = "2041-1723",

publisher = "Nature Research",

}

Munro, SA, Lund, SP, Pine, PS, Binder, H, Clevert, DA, Conesa, A, Dopazo, J, Fasold, M, Hochreiter, S, Hong, H, Jafari, N, Kreil, DP, Łabaj, PP, Li, S, Liao, Y, Lin, SM, Meehan, J, Mason, CE, Santoyo-Lopez, J, Setterquist, RA, Shi, L, Shi, W, Smyth, GK, Stralis-Pavese, N, Su, Z, Tong, W, Wang, C, Wang, J, Xu, J, Ye, Z, Yang, Y, Yu, Y & Salit, M 2014, 'Assessing technical performance in differential gene expression experiments with external spike-in RNA control ratio mixtures', Nature communications, vol. 5, 5125. https://doi.org/10.1038/ncomms6125

TY - JOUR

T1 - Assessing technical performance in differential gene expression experiments with external spike-in RNA control ratio mixtures

AU - Munro, Sarah A.

AU - Lund, Steven P.

AU - Pine, P. Scott

AU - Binder, Hans

AU - Clevert, Djork Arné

AU - Conesa, Ana

AU - Dopazo, Joaquin

AU - Fasold, Mario

AU - Hochreiter, Sepp

AU - Hong, Huixiao

AU - Jafari, Nadereh

AU - Kreil, David P.

AU - Łabaj, Paweł P.

AU - Li, Sheng

AU - Liao, Yang

AU - Lin, Simon M.

AU - Meehan, Joseph

AU - Mason, Christopher E.

AU - Santoyo-Lopez, Javier

AU - Setterquist, Robert A.

AU - Shi, Leming

AU - Shi, Wei

AU - Smyth, Gordon K.

AU - Stralis-Pavese, Nancy

AU - Su, Zhenqiang

AU - Tong, Weida

AU - Wang, Charles

AU - Wang, Jian

AU - Xu, Joshua

AU - Ye, Zhan

AU - Yang, Yong

AU - Yu, Ying

AU - Salit, Marc

PY - 2014

Y1 - 2014

N2 - There is a critical need for standard approaches to assess, report and compare the technical performance of genome-scale differential gene expression experiments. Here we assess technical performance with a proposed standard 'dashboard' of metrics derived from analysis of external spike-in RNA control ratio mixtures. These control ratio mixtures with defined abundance ratios enable assessment of diagnostic performance of differentially expressed transcript lists, limit of detection of ratio (LODR) estimates and expression ratio variability and measurement bias. The performance metrics suite is applicable to analysis of a typical experiment, and here we also apply these metrics to evaluate technical performance among laboratories. An interlaboratory study using identical samples shared among 12 laboratories with three different measurement processes demonstrates generally consistent diagnostic power across 11 laboratories. Ratio measurement variability and bias are also comparable among laboratories for the same measurement process. We observe different biases for measurement processes using different mRNA-enrichment protocols.

AB - There is a critical need for standard approaches to assess, report and compare the technical performance of genome-scale differential gene expression experiments. Here we assess technical performance with a proposed standard 'dashboard' of metrics derived from analysis of external spike-in RNA control ratio mixtures. These control ratio mixtures with defined abundance ratios enable assessment of diagnostic performance of differentially expressed transcript lists, limit of detection of ratio (LODR) estimates and expression ratio variability and measurement bias. The performance metrics suite is applicable to analysis of a typical experiment, and here we also apply these metrics to evaluate technical performance among laboratories. An interlaboratory study using identical samples shared among 12 laboratories with three different measurement processes demonstrates generally consistent diagnostic power across 11 laboratories. Ratio measurement variability and bias are also comparable among laboratories for the same measurement process. We observe different biases for measurement processes using different mRNA-enrichment protocols.

UR - http://www.scopus.com/inward/record.url?scp=84923358731&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84923358731&partnerID=8YFLogxK

U2 - 10.1038/ncomms6125

DO - 10.1038/ncomms6125

M3 - Article

C2 - 25254650

AN - SCOPUS:84923358731

SN - 2041-1723

VL - 5

JO - Nature communications

JF - Nature communications

M1 - 5125

ER -

Assessing technical performance in differential gene expression experiments with external spike-in RNA control ratio mixtures

Abstract

Funding

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this