TY - JOUR

T1 - Competition and moral behavior

T2 - A meta-analysis of forty-five crowd-sourced experimental designs

AU - Huber, Christoph

AU - Dreber, Anna

AU - Huber, Jürgen

AU - Johannesson, Magnus

AU - Kirchler, Michael

AU - Weitzel, Utz

AU - Abellán, Miguel

AU - Adayeva, Xeniya

AU - Ay, Fehime Ceren

AU - Barron, Kai

AU - Berry, Zachariah

AU - Bönte, Werner

AU - Brütt, Katharina

AU - Bulutay, Muhammed

AU - Campos-Mercade, Pol

AU - Cardella, Eric

AU - Claassen, Maria Almudena

AU - Cornelissen, Gert

AU - Dawson, Ian G.J.

AU - Delnoij, Joyce

AU - Demiral, Elif E.

AU - Dimant, Eugen

AU - Doerflinger, Johannes Theodor

AU - Dold, Malte

AU - Emery, Cécile

AU - Fiala, Lenka

AU - Fiedler, Susann

AU - Freddi, Eleonora

AU - Fries, Tilman

AU - Gasiorowska, Agata

AU - Glogowsky, Ulrich

AU - Gorny, Paul M.

AU - Gretton, Jeremy David

AU - Grohmann, Antonia

AU - Hafenbrädl, Sebastian

AU - Handgraaf, Michel

AU - Hanoch, Yaniv

AU - Hart, Einav

AU - Hennig, Max

AU - Hudja, Stanton

AU - Hütter, Mandy

AU - Hyndman, Kyle

AU - Ioannidis, Konstantinos

AU - Isler, Ozan

AU - Jeworrek, Sabrina

AU - Jolles, Daniel

AU - Juanchich, Marie

AU - Pratap, Raghabendra K.C.

AU - Khadjavi, Menusch

AU - Schneider, Florian

AU - Author collaboration many designs

N1 - Funding Information: ACKNOWLEDGMENTS.For financial support,we thank the Austrian National Bank (grant 17788 to M. Kirchler),Austrian Science Fund (grants SFB F6307 to A.D.; SFB F6309 to J.H.; and SFB F6310 to M. Kirchler), Jan Wallander and Tom Hedelius Foundation (grant P21-0091 to A.D.),Knut and Alice Wallenberg Foundation (grant KAW 2018.0134 to A.D.), Marianne and Marcus Wallenberg Foundation (grant KAW2019.0434;toA.D.),RadboudUniversityNijmegen(grant2701437toU.W.), and Riksbankens Jubileumsfond (grant P21-0168 to M. Johannesson). Publisher Copyright: Copyright © 2023 the Author(s).

PY - 2023/6

Y1 - 2023/6

N2 - Does competition affect moral behavior? This fundamental question has been debated among leading scholars for centuries, and more recently, it has been tested in experimental studies yielding a body of rather inconclusive empirical evidence. A potential source of ambivalent empirical results on the same hypothesis is design heterogeneity—variation in true effect sizes across various reasonable experimental research protocols. To provide further evidence on whether competition affects moral behavior and to examine whether the generalizability of a single experimental study is jeopardized by design heterogeneity, we invited independent research teams to contribute experimental designs to a crowd-sourced project. In a large-scale online data collection, 18,123 experimental participants were randomly allocated to 45 randomly selected experimental designs out of 95 submitted designs. We find a small adverse effect of competition on moral behavior in a meta-analysis of the pooled data. The crowd-sourced design of our study allows for a clean identification and estimation of the variation in effect sizes above and beyond what could be expected due to sampling variance. We find substantial design heterogeneity—estimated to be about 1.6 times as large as the average standard error of effect size estimates of the 45 research designs—indicating that the informativeness and generalizability of results based on a single experimental design are limited. Drawing strong conclusions about the underlying hypotheses in the presence of substantive design heterogeneity requires moving toward much larger data collections on various experimental designs testing the same hypothesis.

AB - Does competition affect moral behavior? This fundamental question has been debated among leading scholars for centuries, and more recently, it has been tested in experimental studies yielding a body of rather inconclusive empirical evidence. A potential source of ambivalent empirical results on the same hypothesis is design heterogeneity—variation in true effect sizes across various reasonable experimental research protocols. To provide further evidence on whether competition affects moral behavior and to examine whether the generalizability of a single experimental study is jeopardized by design heterogeneity, we invited independent research teams to contribute experimental designs to a crowd-sourced project. In a large-scale online data collection, 18,123 experimental participants were randomly allocated to 45 randomly selected experimental designs out of 95 submitted designs. We find a small adverse effect of competition on moral behavior in a meta-analysis of the pooled data. The crowd-sourced design of our study allows for a clean identification and estimation of the variation in effect sizes above and beyond what could be expected due to sampling variance. We find substantial design heterogeneity—estimated to be about 1.6 times as large as the average standard error of effect size estimates of the 45 research designs—indicating that the informativeness and generalizability of results based on a single experimental design are limited. Drawing strong conclusions about the underlying hypotheses in the presence of substantive design heterogeneity requires moving toward much larger data collections on various experimental designs testing the same hypothesis.

KW - competition

KW - experimental design

KW - generalizability

KW - metascience

KW - moral behavior

U2 - 10.1073/pnas.2215572120

DO - 10.1073/pnas.2215572120

M3 - Journal article

C2 - 37252958

AN - SCOPUS:85160653952

VL - 120

JO - Proceedings of the National Academy of Sciences of the United States of America

JF - Proceedings of the National Academy of Sciences of the United States of America

SN - 0027-8424

IS - 23

M1 - e2215572120

ER -