Social network, Web forum, or task market? Comparing different crowd genres for design feedback exchange

Yu Chun Grace Yen; Steven P. Dow; Elizabeth Gerber; Brian P. Bailey

doi:10.1145/2901790.2901820

Social network, Web forum, or task market? Comparing different crowd genres for design feedback exchange

Yu Chun Grace Yen, Steven P. Dow, Elizabeth Gerber, Brian P. Bailey

Mechanical Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

33 Scopus citations

Abstract

Increasingly, designers seek feedback on their designs from crowd platforms such as social networks, Web forums, and paid task markets which demand different amounts of social capital, financial resources, and time. Yet it is unknown how the choice of crowd platform affects feedback generation. We conducted an online study where designers created initial designs and revised the designs based on crowd feedback. We measured the quantity, quality, and content of the feedback received at two iterations and from crowds driven by social status, enjoyment, and financial gain. Our results show, for example, that task markets yield more suggestions, online forums provide more process feedback, and social networks give the most suggestions without payment. We contribute an emergent framework for crowd feedback selection, opportunities for enhancing feedback services, and an experimental platform that researchers can adapt to reduce the burden of conducting online studies of design feedback.

Original language	English (US)
Title of host publication	DIS 2016 - Proceedings of the 2016 ACM Conference on Designing Interactive Systems
Subtitle of host publication	Fuse
Publisher	Association for Computing Machinery, Inc
Pages	773-784
Number of pages	12
ISBN (Electronic)	9781450340311
DOIs	https://doi.org/10.1145/2901790.2901820
State	Published - Jun 4 2016
Event	11th ACM SIGCHI Conference on Designing Interactive Systems, DIS 2016 - Brisbane, Australia Duration: Jun 4 2016 → Jun 8 2016

Publication series

Name	DIS 2016 - Proceedings of the 2016 ACM Conference on Designing Interactive Systems: Fuse

Other

Other	11th ACM SIGCHI Conference on Designing Interactive Systems, DIS 2016
Country/Territory	Australia
City	Brisbane
Period	6/4/16 → 6/8/16

Funding

This work was supported in part by the National Science Foundation under awards 1462693, 1530818, and 1213091.

Keywords

Creativity
Critique
Crowdsourcing
Design methods
Feedback
Online crowds

ASJC Scopus subject areas

Computer Graphics and Computer-Aided Design
Human-Computer Interaction
Software

Access to Document

10.1145/2901790.2901820

Cite this

Yen, Y. C. G., Dow, S. P., Gerber, E., & Bailey, B. P. (2016). Social network, Web forum, or task market? Comparing different crowd genres for design feedback exchange. In DIS 2016 - Proceedings of the 2016 ACM Conference on Designing Interactive Systems: Fuse (pp. 773-784). (DIS 2016 - Proceedings of the 2016 ACM Conference on Designing Interactive Systems: Fuse). Association for Computing Machinery, Inc. https://doi.org/10.1145/2901790.2901820

Yen, Yu Chun Grace ; Dow, Steven P. ; Gerber, Elizabeth et al. / Social network, Web forum, or task market? Comparing different crowd genres for design feedback exchange. DIS 2016 - Proceedings of the 2016 ACM Conference on Designing Interactive Systems: Fuse. Association for Computing Machinery, Inc, 2016. pp. 773-784 (DIS 2016 - Proceedings of the 2016 ACM Conference on Designing Interactive Systems: Fuse).

@inproceedings{d9e53d909be144809f00ccb4dd8d6979,

title = "Social network, Web forum, or task market? Comparing different crowd genres for design feedback exchange",

abstract = "Increasingly, designers seek feedback on their designs from crowd platforms such as social networks, Web forums, and paid task markets which demand different amounts of social capital, financial resources, and time. Yet it is unknown how the choice of crowd platform affects feedback generation. We conducted an online study where designers created initial designs and revised the designs based on crowd feedback. We measured the quantity, quality, and content of the feedback received at two iterations and from crowds driven by social status, enjoyment, and financial gain. Our results show, for example, that task markets yield more suggestions, online forums provide more process feedback, and social networks give the most suggestions without payment. We contribute an emergent framework for crowd feedback selection, opportunities for enhancing feedback services, and an experimental platform that researchers can adapt to reduce the burden of conducting online studies of design feedback.",

keywords = "Creativity, Critique, Crowdsourcing, Design methods, Feedback, Online crowds",

author = "Yen, {Yu Chun Grace} and Dow, {Steven P.} and Elizabeth Gerber and Bailey, {Brian P.}",

note = "Publisher Copyright: {\textcopyright} 2016 ACM.; 11th ACM SIGCHI Conference on Designing Interactive Systems, DIS 2016 ; Conference date: 04-06-2016 Through 08-06-2016",

year = "2016",

month = jun,

day = "4",

doi = "10.1145/2901790.2901820",

language = "English (US)",

series = "DIS 2016 - Proceedings of the 2016 ACM Conference on Designing Interactive Systems: Fuse",

publisher = "Association for Computing Machinery, Inc",

pages = "773--784",

booktitle = "DIS 2016 - Proceedings of the 2016 ACM Conference on Designing Interactive Systems",

}

Yen, YCG, Dow, SP, Gerber, E & Bailey, BP 2016, Social network, Web forum, or task market? Comparing different crowd genres for design feedback exchange. in DIS 2016 - Proceedings of the 2016 ACM Conference on Designing Interactive Systems: Fuse. DIS 2016 - Proceedings of the 2016 ACM Conference on Designing Interactive Systems: Fuse, Association for Computing Machinery, Inc, pp. 773-784, 11th ACM SIGCHI Conference on Designing Interactive Systems, DIS 2016, Brisbane, Australia, 6/4/16. https://doi.org/10.1145/2901790.2901820

Social network, Web forum, or task market? Comparing different crowd genres for design feedback exchange. / Yen, Yu Chun Grace; Dow, Steven P.; Gerber, Elizabeth et al.
DIS 2016 - Proceedings of the 2016 ACM Conference on Designing Interactive Systems: Fuse. Association for Computing Machinery, Inc, 2016. p. 773-784 (DIS 2016 - Proceedings of the 2016 ACM Conference on Designing Interactive Systems: Fuse).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Social network, Web forum, or task market? Comparing different crowd genres for design feedback exchange

AU - Yen, Yu Chun Grace

AU - Dow, Steven P.

AU - Gerber, Elizabeth

AU - Bailey, Brian P.

PY - 2016/6/4

Y1 - 2016/6/4

N2 - Increasingly, designers seek feedback on their designs from crowd platforms such as social networks, Web forums, and paid task markets which demand different amounts of social capital, financial resources, and time. Yet it is unknown how the choice of crowd platform affects feedback generation. We conducted an online study where designers created initial designs and revised the designs based on crowd feedback. We measured the quantity, quality, and content of the feedback received at two iterations and from crowds driven by social status, enjoyment, and financial gain. Our results show, for example, that task markets yield more suggestions, online forums provide more process feedback, and social networks give the most suggestions without payment. We contribute an emergent framework for crowd feedback selection, opportunities for enhancing feedback services, and an experimental platform that researchers can adapt to reduce the burden of conducting online studies of design feedback.

AB - Increasingly, designers seek feedback on their designs from crowd platforms such as social networks, Web forums, and paid task markets which demand different amounts of social capital, financial resources, and time. Yet it is unknown how the choice of crowd platform affects feedback generation. We conducted an online study where designers created initial designs and revised the designs based on crowd feedback. We measured the quantity, quality, and content of the feedback received at two iterations and from crowds driven by social status, enjoyment, and financial gain. Our results show, for example, that task markets yield more suggestions, online forums provide more process feedback, and social networks give the most suggestions without payment. We contribute an emergent framework for crowd feedback selection, opportunities for enhancing feedback services, and an experimental platform that researchers can adapt to reduce the burden of conducting online studies of design feedback.

KW - Creativity

KW - Critique

KW - Crowdsourcing

KW - Design methods

KW - Feedback

KW - Online crowds

UR - http://www.scopus.com/inward/record.url?scp=84978634576&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84978634576&partnerID=8YFLogxK

U2 - 10.1145/2901790.2901820

DO - 10.1145/2901790.2901820

M3 - Conference contribution

AN - SCOPUS:84978634576

T3 - DIS 2016 - Proceedings of the 2016 ACM Conference on Designing Interactive Systems: Fuse

SP - 773

EP - 784

BT - DIS 2016 - Proceedings of the 2016 ACM Conference on Designing Interactive Systems

PB - Association for Computing Machinery, Inc

T2 - 11th ACM SIGCHI Conference on Designing Interactive Systems, DIS 2016

Y2 - 4 June 2016 through 8 June 2016

ER -

Yen YCG, Dow SP, Gerber E, Bailey BP. Social network, Web forum, or task market? Comparing different crowd genres for design feedback exchange. In DIS 2016 - Proceedings of the 2016 ACM Conference on Designing Interactive Systems: Fuse. Association for Computing Machinery, Inc. 2016. p. 773-784. (DIS 2016 - Proceedings of the 2016 ACM Conference on Designing Interactive Systems: Fuse). doi: 10.1145/2901790.2901820

Social network, Web forum, or task market? Comparing different crowd genres for design feedback exchange

Abstract

Publication series

Other

Funding

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this