Challenges in Evaluating Interactive Visual Machine Learning Systems

N. Boukhelifa; A. Bezerianos; R. Chang; C. Collins; S. Drucker; A. Endert; J. Hullman; C. North; M. Sedlmair

doi:10.1109/MCG.2020.3017064

Challenges in Evaluating Interactive Visual Machine Learning Systems

N. Boukhelifa, A. Bezerianos, R. Chang, C. Collins, S. Drucker, A. Endert, J. Hullman, C. North, M. Sedlmair

Computer Science

Research output: Contribution to journal › Article › peer-review

Abstract

In interactive visual machine learning (IVML), humans and machine learning algorithms collaborate to achieve tasks mediated by interactive visual interfaces. This human-in-the-loop approach to machine learning brings forth not only numerous intelligibility, trust, and usability issues, but also many open questions with respect to the evaluation of the IVML system, both as separate components, and as a holistic entity that includes both human and machine intelligence. This article describes the challenges and research gaps identified in an IEEE VIS workshop on the evaluation of IVML systems.

Original language	English (US)
Article number	9238590
Pages (from-to)	88-96
Number of pages	9
Journal	IEEE Computer Graphics and Applications
Volume	40
Issue number	6
DOIs	https://doi.org/10.1109/MCG.2020.3017064
State	Published - Nov 1 2020

ASJC Scopus subject areas

Software
Computer Graphics and Computer-Aided Design

Access to Document

10.1109/MCG.2020.3017064

Fingerprint Dive into the research topics of 'Challenges in Evaluating Interactive Visual Machine Learning Systems'. Together they form a unique fingerprint.

Cite this

@article{8472f553bfb346fa9852cd3b99665743,

title = "Challenges in Evaluating Interactive Visual Machine Learning Systems",

abstract = "In interactive visual machine learning (IVML), humans and machine learning algorithms collaborate to achieve tasks mediated by interactive visual interfaces. This human-in-the-loop approach to machine learning brings forth not only numerous intelligibility, trust, and usability issues, but also many open questions with respect to the evaluation of the IVML system, both as separate components, and as a holistic entity that includes both human and machine intelligence. This article describes the challenges and research gaps identified in an IEEE VIS workshop on the evaluation of IVML systems. ",

author = "N. Boukhelifa and A. Bezerianos and R. Chang and C. Collins and S. Drucker and A. Endert and J. Hullman and C. North and M. Sedlmair",

year = "2020",

month = nov,

day = "1",

doi = "10.1109/MCG.2020.3017064",

language = "English (US)",

volume = "40",

pages = "88--96",

journal = "IEEE Computer Graphics and Applications",

issn = "0272-1716",

publisher = "IEEE Computer Society",

number = "6",

}

TY - JOUR

T1 - Challenges in Evaluating Interactive Visual Machine Learning Systems

AU - Boukhelifa, N.

AU - Bezerianos, A.

AU - Chang, R.

AU - Collins, C.

AU - Drucker, S.

AU - Endert, A.

AU - Hullman, J.

AU - North, C.

AU - Sedlmair, M.

PY - 2020/11/1

Y1 - 2020/11/1

N2 - In interactive visual machine learning (IVML), humans and machine learning algorithms collaborate to achieve tasks mediated by interactive visual interfaces. This human-in-the-loop approach to machine learning brings forth not only numerous intelligibility, trust, and usability issues, but also many open questions with respect to the evaluation of the IVML system, both as separate components, and as a holistic entity that includes both human and machine intelligence. This article describes the challenges and research gaps identified in an IEEE VIS workshop on the evaluation of IVML systems.

AB - In interactive visual machine learning (IVML), humans and machine learning algorithms collaborate to achieve tasks mediated by interactive visual interfaces. This human-in-the-loop approach to machine learning brings forth not only numerous intelligibility, trust, and usability issues, but also many open questions with respect to the evaluation of the IVML system, both as separate components, and as a holistic entity that includes both human and machine intelligence. This article describes the challenges and research gaps identified in an IEEE VIS workshop on the evaluation of IVML systems.

UR - http://www.scopus.com/inward/record.url?scp=85094685122&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85094685122&partnerID=8YFLogxK

U2 - 10.1109/MCG.2020.3017064

DO - 10.1109/MCG.2020.3017064

M3 - Article

C2 - 33095702

AN - SCOPUS:85094685122

VL - 40

SP - 88

EP - 96

JO - IEEE Computer Graphics and Applications

JF - IEEE Computer Graphics and Applications

SN - 0272-1716

IS - 6

M1 - 9238590

ER -