An Aligned Rank Transform Procedure for Multifactor Contrast Tests

Lisa A. Elkin, Matthew Kay, James J. Higgins, Jacob O. Wobbrock

Research output: Chapter in Book/Report/Conference proceedingConference contribution

17 Scopus citations

Abstract

Data from multifactor HCI experiments often violates the assumptions of parametric tests (i.e., nonconforming data). The Aligned Rank Transform (ART) has become a popular nonparametric analysis in HCI that can find main and interaction effects in nonconforming data, but leads to incorrect results when used to conduct post hoc contrast tests. We created a new algorithm called ART-C for conducting contrast tests within the ART paradigm and validated it on 72,000 synthetic data sets. Our results indicate that ART-C does not inflate Type I error rates, unlike contrasts based on ART, and that ART-C has more statistical power than a t-test, Mann-Whitney U test, Wilcoxon signed-rank test, and ART. We also extended an open-source tool called ARTool with our ART-C algorithm for both Windows and R. Our validation had some limitations (e.g., only six distribution types, no mixed factorial designs, no random slopes), and data drawn from Cauchy distributions should not be analyzed with ART-C.

Original languageEnglish (US)
Title of host publicationUIST 2021 - Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology
PublisherAssociation for Computing Machinery, Inc
Pages754-768
Number of pages15
ISBN (Electronic)9781450386357
DOIs
StatePublished - Oct 10 2021
Event34th Annual ACM Symposium on User Interface Software and Technology, UIST 2021 - Virtual, Online, United States
Duration: Oct 10 2021Oct 14 2021

Publication series

NameUIST 2021 - Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology

Conference

Conference34th Annual ACM Symposium on User Interface Software and Technology, UIST 2021
Country/TerritoryUnited States
CityVirtual, Online
Period10/10/2110/14/21

Keywords

  • aligned rank transform.
  • data analysis
  • experiments
  • nonparametric statistics
  • quantitative methods
  • Statistical methods

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Software

Fingerprint

Dive into the research topics of 'An Aligned Rank Transform Procedure for Multifactor Contrast Tests'. Together they form a unique fingerprint.

Cite this