Practical Methods for Semi-automated Peer Grading in a Classroom Setting

Zheng Yuan, Doug Downey

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Peer grading, in which students grade each other's work, can provide an educational opportunity for students and reduce grading effort for instructors. A variety of methods have been proposed for synthesizing peer-assigned grades into accurate submission grades. However, when the assumptions behind these methods are not met, they may underperform a simple baseline of averaging the peer grades. We introduce SABTXT, which improves over previous work through two mechanisms. First, SABTXT uses a limited amount of historical instructor ground truth to model and correct for each peer's grading bias. Secondly, SABTXT models the thoroughness of a peer review based on its textual content, and puts more weight on the more thorough peer reviews when computing submission grades. In our experiments with over ten thousand peer reviews collected over four courses, we show that SABTXT outperforms existing approaches on our collected data, and achieves a mean squared error that is 6% lower than the strongest baseline on average.

Original languageEnglish (US)
Title of host publicationUMAP 2020 - Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization
PublisherAssociation for Computing Machinery, Inc
Pages363-367
Number of pages5
ISBN (Electronic)9781450368612
DOIs
StatePublished - Jul 7 2020
Event28th ACM International Conference on User Modeling, Adaptation, and Personalization, UMAP 2020 - Genoa, Italy
Duration: Jul 14 2020Jul 17 2020

Publication series

NameUMAP 2020 - Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization

Conference

Conference28th ACM International Conference on User Modeling, Adaptation, and Personalization, UMAP 2020
CountryItaly
CityGenoa
Period7/14/207/17/20

Keywords

  • automated grading
  • educational technology
  • peer review

ASJC Scopus subject areas

  • Software

Fingerprint Dive into the research topics of 'Practical Methods for Semi-automated Peer Grading in a Classroom Setting'. Together they form a unique fingerprint.

Cite this