The Wikipedia adventure: Field evaluation of an interactive tutorial for new users

Sneha Narayan; Jake Orlowitz; Jonathan Morgan; Benjamin Mako Hill; Aaron Shaw

doi:10.1145/2998181.2998307

The Wikipedia adventure: Field evaluation of an interactive tutorial for new users

Sneha Narayan, Jake Orlowitz, Jonathan Morgan, Benjamin Mako Hill, Aaron Shaw

Communication Studies

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

20 Scopus citations

Abstract

Integrating new users into a community with complex norms presents a challenge for peer production projects like Wikipedia. We present The Wikipedia Adventure (TWA): an interactive tutorial that offers a structured and gamified introduction to Wikipedia. In addition to describing the design of the system, we present two empirical evaluations. First, we report on a survey of users, who responded very positively to the tutorial. Second, we report results from a largescale invitation-based field experiment that tests whether using TWA increased newcomers' subsequent contributions to Wikipedia. We find no effect of either using the tutorial or of being invited to do so over a period of 180 days. We conclude that TWA produces a positive socialization experience for those who choose to use it, but that it does not alter patterns of newcomer activity. We reflect on the implications of these mixed results for the evaluation of similar social computing systems.

Original language	English (US)
Title of host publication	CSCW 2017 - Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing
Publisher	Association for Computing Machinery
Pages	1785-1799
Number of pages	15
ISBN (Electronic)	9781450343350
DOIs	https://doi.org/10.1145/2998181.2998307
State	Published - Feb 25 2017
Event	2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, CSCW 2017 - Portland, United States Duration: Feb 25 2017 → Mar 1 2017

Publication series

Name	Proceedings of the ACM Conference on Computer Supported Cooperative Work, CSCW

Other

Other	2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, CSCW 2017
Country/Territory	United States
City	Portland
Period	2/25/17 → 3/1/17

Keywords

Gamification
Newcomer socialization
Online communities
Peer production
Systems design
Systems evaluation
Wikipedia

ASJC Scopus subject areas

Software
Computer Networks and Communications
Human-Computer Interaction

Access to Document

10.1145/2998181.2998307

Cite this

Narayan, S., Orlowitz, J., Morgan, J., Hill, B. M., & Shaw, A. (2017). The Wikipedia adventure: Field evaluation of an interactive tutorial for new users. In CSCW 2017 - Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (pp. 1785-1799). (Proceedings of the ACM Conference on Computer Supported Cooperative Work, CSCW). Association for Computing Machinery. https://doi.org/10.1145/2998181.2998307

Narayan, Sneha ; Orlowitz, Jake ; Morgan, Jonathan et al. / The Wikipedia adventure : Field evaluation of an interactive tutorial for new users. CSCW 2017 - Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. Association for Computing Machinery, 2017. pp. 1785-1799 (Proceedings of the ACM Conference on Computer Supported Cooperative Work, CSCW).

@inproceedings{92904123a93c4ca9a572efe0bc4a2a09,

title = "The Wikipedia adventure: Field evaluation of an interactive tutorial for new users",

abstract = "Integrating new users into a community with complex norms presents a challenge for peer production projects like Wikipedia. We present The Wikipedia Adventure (TWA): an interactive tutorial that offers a structured and gamified introduction to Wikipedia. In addition to describing the design of the system, we present two empirical evaluations. First, we report on a survey of users, who responded very positively to the tutorial. Second, we report results from a largescale invitation-based field experiment that tests whether using TWA increased newcomers' subsequent contributions to Wikipedia. We find no effect of either using the tutorial or of being invited to do so over a period of 180 days. We conclude that TWA produces a positive socialization experience for those who choose to use it, but that it does not alter patterns of newcomer activity. We reflect on the implications of these mixed results for the evaluation of similar social computing systems.",

keywords = "Gamification, Newcomer socialization, Online communities, Peer production, Systems design, Systems evaluation, Wikipedia",

author = "Sneha Narayan and Jake Orlowitz and Jonathan Morgan and Hill, {Benjamin Mako} and Aaron Shaw",

note = "Copyright: Copyright 2017 Elsevier B.V., All rights reserved.; 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, CSCW 2017 ; Conference date: 25-02-2017 Through 01-03-2017",

year = "2017",

month = feb,

day = "25",

doi = "10.1145/2998181.2998307",

language = "English (US)",

series = "Proceedings of the ACM Conference on Computer Supported Cooperative Work, CSCW",

publisher = "Association for Computing Machinery",

pages = "1785--1799",

booktitle = "CSCW 2017 - Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing",

}

Narayan, S, Orlowitz, J, Morgan, J, Hill, BM & Shaw, A 2017, The Wikipedia adventure: Field evaluation of an interactive tutorial for new users. in CSCW 2017 - Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. Proceedings of the ACM Conference on Computer Supported Cooperative Work, CSCW, Association for Computing Machinery, pp. 1785-1799, 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, CSCW 2017, Portland, United States, 2/25/17. https://doi.org/10.1145/2998181.2998307

The Wikipedia adventure: Field evaluation of an interactive tutorial for new users. / Narayan, Sneha; Orlowitz, Jake; Morgan, Jonathan et al.
CSCW 2017 - Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. Association for Computing Machinery, 2017. p. 1785-1799 (Proceedings of the ACM Conference on Computer Supported Cooperative Work, CSCW).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - The Wikipedia adventure

T2 - 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, CSCW 2017

AU - Narayan, Sneha

AU - Orlowitz, Jake

AU - Morgan, Jonathan

AU - Hill, Benjamin Mako

AU - Shaw, Aaron

PY - 2017/2/25

Y1 - 2017/2/25

N2 - Integrating new users into a community with complex norms presents a challenge for peer production projects like Wikipedia. We present The Wikipedia Adventure (TWA): an interactive tutorial that offers a structured and gamified introduction to Wikipedia. In addition to describing the design of the system, we present two empirical evaluations. First, we report on a survey of users, who responded very positively to the tutorial. Second, we report results from a largescale invitation-based field experiment that tests whether using TWA increased newcomers' subsequent contributions to Wikipedia. We find no effect of either using the tutorial or of being invited to do so over a period of 180 days. We conclude that TWA produces a positive socialization experience for those who choose to use it, but that it does not alter patterns of newcomer activity. We reflect on the implications of these mixed results for the evaluation of similar social computing systems.

AB - Integrating new users into a community with complex norms presents a challenge for peer production projects like Wikipedia. We present The Wikipedia Adventure (TWA): an interactive tutorial that offers a structured and gamified introduction to Wikipedia. In addition to describing the design of the system, we present two empirical evaluations. First, we report on a survey of users, who responded very positively to the tutorial. Second, we report results from a largescale invitation-based field experiment that tests whether using TWA increased newcomers' subsequent contributions to Wikipedia. We find no effect of either using the tutorial or of being invited to do so over a period of 180 days. We conclude that TWA produces a positive socialization experience for those who choose to use it, but that it does not alter patterns of newcomer activity. We reflect on the implications of these mixed results for the evaluation of similar social computing systems.

KW - Gamification

KW - Newcomer socialization

KW - Online communities

KW - Peer production

KW - Systems design

KW - Systems evaluation

KW - Wikipedia

UR - http://www.scopus.com/inward/record.url?scp=85014780973&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85014780973&partnerID=8YFLogxK

U2 - 10.1145/2998181.2998307

DO - 10.1145/2998181.2998307

M3 - Conference contribution

AN - SCOPUS:85014780973

T3 - Proceedings of the ACM Conference on Computer Supported Cooperative Work, CSCW

SP - 1785

EP - 1799

BT - CSCW 2017 - Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing

PB - Association for Computing Machinery

Y2 - 25 February 2017 through 1 March 2017

ER -

Narayan S, Orlowitz J, Morgan J, Hill BM, Shaw A. The Wikipedia adventure: Field evaluation of an interactive tutorial for new users. In CSCW 2017 - Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. Association for Computing Machinery. 2017. p. 1785-1799. (Proceedings of the ACM Conference on Computer Supported Cooperative Work, CSCW). doi: 10.1145/2998181.2998307

The Wikipedia adventure: Field evaluation of an interactive tutorial for new users

Abstract

Publication series

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Replication Data for: The Wikipedia Adventure: Field Evaluation of an Interactive Tutorial for New Users

Cite this

The Wikipedia adventure: Field evaluation of an interactive tutorial for new users

Abstract

Publication series

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Datasets

Replication Data for: The Wikipedia Adventure: Field Evaluation of an Interactive Tutorial for New Users

Cite this