TY - JOUR
T1 - Qrowdsmith: Enhancing paid microtask crowdsourcing with gamification and furtherance incentives
AU - Simperl, Elena
AU - Maddalena, Eddy
AU - Ibáñez, Luis-Daniel
AU - Reeves, Neal
N1 - Funding Information:
This work was partially supported by the European Union’s Horizon 2020 research and innovation programmes Qrowd and Action, under grant agreements No. 732194 and No. 824603; and Cleopatra, under the Marie Skłodowska-Curie grant agreement No. 812997.
Publisher Copyright:
© 2023 Copyright held by the owner/author(s). Publication rights licensed to ACM.
PY - 2023/9/30
Y1 - 2023/9/30
N2 - Microtask crowdsourcing platforms are social intelligence systems in which volunteers, called crowdworkers, complete small, repetitive tasks in return for a small fee. Beyond payments, task requesters are considering non-monetary incentives such as points, badges, and other gamified elements to increase performance and improve crowdworker experience. In this article, we present Qrowdsmith, a platform for gamifying microtask crowdsourcing. To design the system, we explore empirically a range of gamified and financial incentives and analyse their impact on how efficient, effective, and reliable the results are. To maintain participation over time and save costs, we propose furtherance incentives, which are offered to crowdworkers to encourage additional contributions in addition to the fee agreed upfront. In a series of controlled experiments, we find that while gamification can work as furtherance incentives, it impacts negatively on crowdworkers' performance, both in terms of the quantity and quality of work, as compared to a baseline where they can continue to contribute voluntarily. Gamified incentives are also less effective than paid bonus equivalents. Our results contribute to the understanding of how best to encourage engagement in microtask crowdsourcing activities and design better crowd intelligence systems.
AB - Microtask crowdsourcing platforms are social intelligence systems in which volunteers, called crowdworkers, complete small, repetitive tasks in return for a small fee. Beyond payments, task requesters are considering non-monetary incentives such as points, badges, and other gamified elements to increase performance and improve crowdworker experience. In this article, we present Qrowdsmith, a platform for gamifying microtask crowdsourcing. To design the system, we explore empirically a range of gamified and financial incentives and analyse their impact on how efficient, effective, and reliable the results are. To maintain participation over time and save costs, we propose furtherance incentives, which are offered to crowdworkers to encourage additional contributions in addition to the fee agreed upfront. In a series of controlled experiments, we find that while gamification can work as furtherance incentives, it impacts negatively on crowdworkers' performance, both in terms of the quantity and quality of work, as compared to a baseline where they can continue to contribute voluntarily. Gamified incentives are also less effective than paid bonus equivalents. Our results contribute to the understanding of how best to encourage engagement in microtask crowdsourcing activities and design better crowd intelligence systems.
UR - http://www.scopus.com/inward/record.url?scp=85174965421&partnerID=8YFLogxK
U2 - 10.1145/3604940
DO - 10.1145/3604940
M3 - Article
SN - 2157-6904
VL - 14
JO - ACM TRANSACTIONS ON INTELLIGENT SYSTEMS AND TECHNOLOGY
JF - ACM TRANSACTIONS ON INTELLIGENT SYSTEMS AND TECHNOLOGY
IS - 5
M1 - 86
ER -