Evaluating visualization sets: Trade-offs between local effectiveness and global consistency

Zening Qu, Jessica Hullman

Research output: Chapter in Book/Report/Conference proceedingConference contribution

11 Scopus citations


Evaluation criteria like expressiveness and effectiveness favor optimal use of space and visual encoding channels in a single visualization. However, individually optimized views may be inconsistent with one another when presented as a set in recommender systems and narrative visualizations. For example, two visualizations might use very similar color palettes for different data fields, or they might render the same field but in different scales. These inconsistencies in visualization sets can cause interpretation errors and increase the cognitive load on viewers trying to analyze a set of visualizations. We propose two high-level principles for evaluating visualization set consistency: (1) the same fields should be presented in the same way, (2) different fields should be presented differently. These two principles are operationalized as a set of constraints for common visual encoding channels (x, y, color, size, and shape) to enable automated visualization set evaluation. To balance global (visualization set) consistency and local (single visualization) effectiveness, trade-offs in space and visual encodings have to be made. We devise an effectiveness preservation score to guide the selection of which conflicts to surface and potentially revise for sets of quantitative and ordinal encodings and a palette resource allocation mechanism for nominal encodings.

Original languageEnglish (US)
Title of host publicationProceedings - Beyond Time and Errors
Subtitle of host publicationNovel Evaluation Methods for Visualization, BELIV 2016 - 6th Bi-Annual Workshop
EditorsMichael Sedlmair, Heidi Lam, Petra Isenberg, Tobias Isenberg, Narges Mahyar
PublisherAssociation for Computing Machinery
Number of pages9
ISBN (Electronic)9781450348188
StatePublished - Oct 24 2016
Event6th Workshop Beyond Time and Errors on Novel Evaluation Methods for Visualization, BELIV 2016 - Baltimore, United States
Duration: Oct 24 2016 → …

Publication series

NameACM International Conference Proceeding Series


Other6th Workshop Beyond Time and Errors on Novel Evaluation Methods for Visualization, BELIV 2016
Country/TerritoryUnited States
Period10/24/16 → …


  • Consistency effectiveness trade-offs
  • Visualization set

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction
  • Computer Vision and Pattern Recognition
  • Computer Networks and Communications


Dive into the research topics of 'Evaluating visualization sets: Trade-offs between local effectiveness and global consistency'. Together they form a unique fingerprint.

Cite this