Knowledge Tracing to Model Learning in Online Citizen Science Projects

Kevin Crowston; Shane L. Larson; Neda Rohani; Joshua R. Smith; Laura Trouille; Michael Zevin; Carsten Osterlund; Tae Kyoung Lee; Corey Jackson; Mahboobeh Harandi; Sarah Allen; Sara Bahaadini; Scott Coughlin; Aggelos K. Katsaggelos

doi:10.1109/TLT.2019.2936480

Knowledge Tracing to Model Learning in Online Citizen Science Projects

Kevin Crowston^*, Shane L. Larson, Neda Rohani, Joshua R. Smith, Laura Trouille, Michael Zevin, Carsten Osterlund, Tae Kyoung Lee, Corey Jackson, Mahboobeh Harandi, Sarah Allen, Sara Bahaadini, Scott Coughlin, Aggelos K. Katsaggelos

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

13 Scopus citations

Abstract

We present the design of a citizen science system that uses machine learning to guide the presentation of image classification tasks to newcomers to help them more quickly learn how to do the task while still contributing to the work of the project. A Bayesian model for tracking volunteer learning for training with tasks with uncertain outcomes is presented and fit to data from 12,986 volunteer contributors. The model can be used both to estimate the ability of volunteers and to decide the classification of an image. A simulation of the model applied to volunteer promotion and image retirement suggests that the model requires fewer classifications than the current system.

Original language	English (US)
Article number	8812979
Pages (from-to)	123-134
Number of pages	12
Journal	IEEE Transactions on Learning Technologies
Volume	13
Issue number	1
DOIs	https://doi.org/10.1109/TLT.2019.2936480
State	Published - Jan 1 2020

Keywords

Citizen science
machine learning
training

ASJC Scopus subject areas

Education
General Engineering
Computer Science Applications

Access to Document

10.1109/TLT.2019.2936480

Cite this

Crowston, K., Larson, S. L., Rohani, N., Smith, J. R., Trouille, L., Zevin, M., Osterlund, C., Lee, T. K., Jackson, C., Harandi, M., Allen, S., Bahaadini, S., Coughlin, S., & Katsaggelos, A. K. (2020). Knowledge Tracing to Model Learning in Online Citizen Science Projects. IEEE Transactions on Learning Technologies, 13(1), 123-134. Article 8812979. https://doi.org/10.1109/TLT.2019.2936480

@article{79f2ff14db9c4bf18ea1225f3219720c,

title = "Knowledge Tracing to Model Learning in Online Citizen Science Projects",

abstract = "We present the design of a citizen science system that uses machine learning to guide the presentation of image classification tasks to newcomers to help them more quickly learn how to do the task while still contributing to the work of the project. A Bayesian model for tracking volunteer learning for training with tasks with uncertain outcomes is presented and fit to data from 12,986 volunteer contributors. The model can be used both to estimate the ability of volunteers and to decide the classification of an image. A simulation of the model applied to volunteer promotion and image retirement suggests that the model requires fewer classifications than the current system.",

keywords = "Citizen science, machine learning, training",

author = "Kevin Crowston and Larson, {Shane L.} and Neda Rohani and Smith, {Joshua R.} and Laura Trouille and Michael Zevin and Carsten Osterlund and Lee, {Tae Kyoung} and Corey Jackson and Mahboobeh Harandi and Sarah Allen and Sara Bahaadini and Scott Coughlin and Katsaggelos, {Aggelos K.}",

note = "Publisher Copyright: {\textcopyright} 2008-2011 IEEE.",

year = "2020",

month = jan,

day = "1",

doi = "10.1109/TLT.2019.2936480",

language = "English (US)",

volume = "13",

pages = "123--134",

journal = "IEEE Transactions on Learning Technologies",

issn = "1939-1382",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "1",

}

Crowston, K, Larson, SL, Rohani, N, Smith, JR, Trouille, L, Zevin, M, Osterlund, C, Lee, TK, Jackson, C, Harandi, M, Allen, S, Bahaadini, S, Coughlin, S & Katsaggelos, AK 2020, 'Knowledge Tracing to Model Learning in Online Citizen Science Projects', IEEE Transactions on Learning Technologies, vol. 13, no. 1, 8812979, pp. 123-134. https://doi.org/10.1109/TLT.2019.2936480

TY - JOUR

T1 - Knowledge Tracing to Model Learning in Online Citizen Science Projects

AU - Crowston, Kevin

AU - Larson, Shane L.

AU - Rohani, Neda

AU - Smith, Joshua R.

AU - Trouille, Laura

AU - Zevin, Michael

AU - Osterlund, Carsten

AU - Lee, Tae Kyoung

AU - Jackson, Corey

AU - Harandi, Mahboobeh

AU - Allen, Sarah

AU - Bahaadini, Sara

AU - Coughlin, Scott

AU - Katsaggelos, Aggelos K.

PY - 2020/1/1

Y1 - 2020/1/1

N2 - We present the design of a citizen science system that uses machine learning to guide the presentation of image classification tasks to newcomers to help them more quickly learn how to do the task while still contributing to the work of the project. A Bayesian model for tracking volunteer learning for training with tasks with uncertain outcomes is presented and fit to data from 12,986 volunteer contributors. The model can be used both to estimate the ability of volunteers and to decide the classification of an image. A simulation of the model applied to volunteer promotion and image retirement suggests that the model requires fewer classifications than the current system.

AB - We present the design of a citizen science system that uses machine learning to guide the presentation of image classification tasks to newcomers to help them more quickly learn how to do the task while still contributing to the work of the project. A Bayesian model for tracking volunteer learning for training with tasks with uncertain outcomes is presented and fit to data from 12,986 volunteer contributors. The model can be used both to estimate the ability of volunteers and to decide the classification of an image. A simulation of the model applied to volunteer promotion and image retirement suggests that the model requires fewer classifications than the current system.

KW - Citizen science

KW - machine learning

KW - training

UR - http://www.scopus.com/inward/record.url?scp=85071543202&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85071543202&partnerID=8YFLogxK

U2 - 10.1109/TLT.2019.2936480

DO - 10.1109/TLT.2019.2936480

M3 - Article

AN - SCOPUS:85071543202

SN - 1939-1382

VL - 13

SP - 123

EP - 134

JO - IEEE Transactions on Learning Technologies

JF - IEEE Transactions on Learning Technologies

IS - 1

M1 - 8812979

ER -

Knowledge Tracing to Model Learning in Online Citizen Science Projects

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this