An extreme-value approach for testing the equality of large U-statistic based correlation matrices

Cheng Zhou, Fang Han, Xin Sheng Zhang, Han Liu

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

There has been an increasing interest in testing the equality of large Pearson’s correlation matrices. However, in many applications it is more important to test the equality of large rank-based correlation matrices since they are more robust to outliers and nonlinearity. Unlike the Pearson’s case, testing the equality of large rank-based statistics has not been well explored and requires us to develop new methods and theory. In this paper, we provide a framework for testing the equality of two large U-statistic based correlation matrices, which include the rank-based correlation matrices as special cases. Our approach exploits extreme value statistics and the Jackknife estimator for uncertainty assessment and is valid under a fully nonparametric model. Theoretically, we develop a theory for testing the equality of U-statistic based correlation matrices. We then apply this theory to study the problem of testing large Kendall’s tau correlation matrices and demonstrate its optimality. For proving this optimality, a novel construction of least favorable distributions is developed for the correlation matrix comparison.

Original languageEnglish (US)
Pages (from-to)1472-1503
Number of pages32
JournalBernoulli
Volume25
Issue number2
DOIs
StatePublished - May 1 2019

Fingerprint

U-statistics
Correlation Matrix
Extreme Values
Equality
Testing
Optimality
Extreme Value Statistics
Kendall's tau
Pearson Correlation
Jackknife
Nonparametric Model
Outlier
Nonlinearity
Valid
Statistics
Estimator
Uncertainty
Demonstrate

Keywords

  • Extreme value type I distribution
  • Hypothesis testing
  • Jackknife variance estimator
  • Kendall’s tau
  • U-statistics

ASJC Scopus subject areas

  • Statistics and Probability

Cite this

Zhou, Cheng ; Han, Fang ; Zhang, Xin Sheng ; Liu, Han. / An extreme-value approach for testing the equality of large U-statistic based correlation matrices. In: Bernoulli. 2019 ; Vol. 25, No. 2. pp. 1472-1503.
@article{04ca9e6256154b9cac7a00e760626ec8,
title = "An extreme-value approach for testing the equality of large U-statistic based correlation matrices",
abstract = "There has been an increasing interest in testing the equality of large Pearson’s correlation matrices. However, in many applications it is more important to test the equality of large rank-based correlation matrices since they are more robust to outliers and nonlinearity. Unlike the Pearson’s case, testing the equality of large rank-based statistics has not been well explored and requires us to develop new methods and theory. In this paper, we provide a framework for testing the equality of two large U-statistic based correlation matrices, which include the rank-based correlation matrices as special cases. Our approach exploits extreme value statistics and the Jackknife estimator for uncertainty assessment and is valid under a fully nonparametric model. Theoretically, we develop a theory for testing the equality of U-statistic based correlation matrices. We then apply this theory to study the problem of testing large Kendall’s tau correlation matrices and demonstrate its optimality. For proving this optimality, a novel construction of least favorable distributions is developed for the correlation matrix comparison.",
keywords = "Extreme value type I distribution, Hypothesis testing, Jackknife variance estimator, Kendall’s tau, U-statistics",
author = "Cheng Zhou and Fang Han and Zhang, {Xin Sheng} and Han Liu",
year = "2019",
month = "5",
day = "1",
doi = "10.3150/18-BEJ1027",
language = "English (US)",
volume = "25",
pages = "1472--1503",
journal = "Bernoulli",
issn = "1350-7265",
publisher = "International Statistical Institute",
number = "2",

}

An extreme-value approach for testing the equality of large U-statistic based correlation matrices. / Zhou, Cheng; Han, Fang; Zhang, Xin Sheng; Liu, Han.

In: Bernoulli, Vol. 25, No. 2, 01.05.2019, p. 1472-1503.

Research output: Contribution to journalArticle

TY - JOUR

T1 - An extreme-value approach for testing the equality of large U-statistic based correlation matrices

AU - Zhou, Cheng

AU - Han, Fang

AU - Zhang, Xin Sheng

AU - Liu, Han

PY - 2019/5/1

Y1 - 2019/5/1

N2 - There has been an increasing interest in testing the equality of large Pearson’s correlation matrices. However, in many applications it is more important to test the equality of large rank-based correlation matrices since they are more robust to outliers and nonlinearity. Unlike the Pearson’s case, testing the equality of large rank-based statistics has not been well explored and requires us to develop new methods and theory. In this paper, we provide a framework for testing the equality of two large U-statistic based correlation matrices, which include the rank-based correlation matrices as special cases. Our approach exploits extreme value statistics and the Jackknife estimator for uncertainty assessment and is valid under a fully nonparametric model. Theoretically, we develop a theory for testing the equality of U-statistic based correlation matrices. We then apply this theory to study the problem of testing large Kendall’s tau correlation matrices and demonstrate its optimality. For proving this optimality, a novel construction of least favorable distributions is developed for the correlation matrix comparison.

AB - There has been an increasing interest in testing the equality of large Pearson’s correlation matrices. However, in many applications it is more important to test the equality of large rank-based correlation matrices since they are more robust to outliers and nonlinearity. Unlike the Pearson’s case, testing the equality of large rank-based statistics has not been well explored and requires us to develop new methods and theory. In this paper, we provide a framework for testing the equality of two large U-statistic based correlation matrices, which include the rank-based correlation matrices as special cases. Our approach exploits extreme value statistics and the Jackknife estimator for uncertainty assessment and is valid under a fully nonparametric model. Theoretically, we develop a theory for testing the equality of U-statistic based correlation matrices. We then apply this theory to study the problem of testing large Kendall’s tau correlation matrices and demonstrate its optimality. For proving this optimality, a novel construction of least favorable distributions is developed for the correlation matrix comparison.

KW - Extreme value type I distribution

KW - Hypothesis testing

KW - Jackknife variance estimator

KW - Kendall’s tau

KW - U-statistics

UR - http://www.scopus.com/inward/record.url?scp=85064054301&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85064054301&partnerID=8YFLogxK

U2 - 10.3150/18-BEJ1027

DO - 10.3150/18-BEJ1027

M3 - Article

VL - 25

SP - 1472

EP - 1503

JO - Bernoulli

JF - Bernoulli

SN - 1350-7265

IS - 2

ER -