Non-Standard Errors

Fincap Team; Albert J. Menkveld; Anna Dreber; Felix Holzmeister; Juergen Huber; Magnus Johanneson; Michael Kirchler; Michael Razen; Utz Weitzel; David Abad; Menachem (meni) Abudy; Tobias Adrian; Yacine Ait-Sahalia; Olivier Akmansoy; Jamie Alcock; Vitali Alexeev; Arash Aloosh; Livia Amato; Diego Amaya; James J. Angel; Amadeus Bach; Edwin Baidoo; Gaetan Bakalli; Andrea Barbon; Oksana Bashchenko; Parampreet Christopher Bindra; Geir Hoidal Bjonnes; Jeffrey R. Black; Bernard S. Black; Santiago Bohorquez; Oleg Bondarenko; Charles S. Bos; Ciril Bosch-Rosa; Elie Bouri; Christian T. Brownlees; Anna Calamia; Viet Nga Cao; Gunther Capelle-Blancard; Laura Capera; Massimiliano Caporin; Allen Carrion; Tolga Caskurlu; Oleg Deev; Thomas Gehrig; Simon Hartmann; Nikolaus Hautsch; Gabriel Kaiser; Thomas Lindner; Giorgia Simion; Stefan Voigt; Patrick Weiss

doi:10.1111/jofi.13337

Non-Standard Errors

Fincap Team, Albert J. Menkveld, Anna Dreber, Felix Holzmeister, Juergen Huber, Magnus Johanneson, Michael Kirchler, Michael Razen, Utz Weitzel, David Abad, Menachem (meni) Abudy, Tobias Adrian, Yacine Ait-Sahalia, Olivier Akmansoy, Jamie Alcock, Vitali Alexeev, Arash Aloosh, Livia Amato, Diego Amaya, James J. AngelAmadeus Bach, Edwin Baidoo, Gaetan Bakalli, Andrea Barbon, Oksana Bashchenko, Parampreet Christopher Bindra, Geir Hoidal Bjonnes, Jeffrey R. Black, Bernard S. Black, Santiago Bohorquez, Oleg Bondarenko, Charles S. Bos, Ciril Bosch-Rosa, Elie Bouri, Christian T. Brownlees, Anna Calamia, Viet Nga Cao, Gunther Capelle-Blancard, Laura Capera, Massimiliano Caporin, Allen Carrion, Tolga Caskurlu, Oleg Deev, Thomas Gehrig, Simon Hartmann, Nikolaus Hautsch, Gabriel Kaiser, Thomas Lindner, Giorgia Simion, Stefan Voigt, Patrick Weiss

Publication: Scientific journal › Journal article › peer-review

Abstract

In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty: Non-standard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for better reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.

Original language	English
Pages (from-to)	2339-2390
Journal	Journal of Finance
Volume	79
Issue number	3
Early online date	2024
DOIs	https://doi.org/10.1111/jofi.13337
Publication status	Published - Jun 2024

Keywords

non-standard errors
liquidity
multi-analyst approach

Access to Document

10.1111/jofi.13337Licence: CC BY

Check availability

University Library Catalogue

Cite this

@article{ec693c6390b24275a86121f78108890b,

title = "Non-Standard Errors",

abstract = "In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty: Non-standard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for better reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants. ",

keywords = "non-standard errors, liquidity, multi-analyst approach",

author = "{Fincap Team} and Menkveld, {Albert J.} and Anna Dreber and Felix Holzmeister and Juergen Huber and Magnus Johanneson and Michael Kirchler and Michael Razen and Utz Weitzel and David Abad and Abudy, {Menachem (meni)} and Tobias Adrian and Yacine Ait-Sahalia and Olivier Akmansoy and Jamie Alcock and Vitali Alexeev and Arash Aloosh and Livia Amato and Diego Amaya and Angel, {James J.} and Amadeus Bach and Edwin Baidoo and Gaetan Bakalli and Andrea Barbon and Oksana Bashchenko and Bindra, {Parampreet Christopher} and Bjonnes, {Geir Hoidal} and Black, {Jeffrey R.} and Black, {Bernard S.} and Santiago Bohorquez and Oleg Bondarenko and Bos, {Charles S.} and Ciril Bosch-Rosa and Elie Bouri and Brownlees, {Christian T.} and Anna Calamia and Cao, {Viet Nga} and Gunther Capelle-Blancard and Laura Capera and Massimiliano Caporin and Allen Carrion and Tolga Caskurlu and Oleg Deev and Thomas Gehrig and Simon Hartmann and Nikolaus Hautsch and Gabriel Kaiser and Thomas Lindner and Giorgia Simion and Stefan Voigt and Patrick Weiss",

year = "2024",

month = jun,

doi = "10.1111/jofi.13337",

language = "English",

volume = "79",

pages = "2339--2390",

journal = "Journal of Finance",

issn = "0022-1082",

publisher = "Wiley-Blackwell Publishing Ltd",

number = "3",

}

Fincap Team, Menkveld, AJ, Dreber, A, Holzmeister, F, Huber, J, Johanneson, M, Kirchler, M, Razen, M, Weitzel, U, Abad, D, Abudy, M, Adrian, T, Ait-Sahalia, Y, Akmansoy, O, Alcock, J, Alexeev, V, Aloosh, A, Amato, L, Amaya, D, Angel, JJ, Bach, A, Baidoo, E, Bakalli, G, Barbon, A, Bashchenko, O, Bindra, PC, Bjonnes, GH, Black, JR, Black, BS, Bohorquez, S, Bondarenko, O, Bos, CS, Bosch-Rosa, C, Bouri, E, Brownlees, CT, Calamia, A, Cao, VN, Capelle-Blancard, G, Capera, L, Caporin, M, Carrion, A, Caskurlu, T, Deev, O, Gehrig, T, Hartmann, S , Hautsch, N, Kaiser, G, Lindner, T , Simion, G , Voigt, S & Weiss, P 2024, 'Non-Standard Errors', Journal of Finance, vol. 79, no. 3, pp. 2339-2390. https://doi.org/10.1111/jofi.13337

TY - JOUR

T1 - Non-Standard Errors

AU - Fincap Team

AU - Menkveld, Albert J.

AU - Dreber, Anna

AU - Holzmeister, Felix

AU - Huber, Juergen

AU - Johanneson, Magnus

AU - Kirchler, Michael

AU - Razen, Michael

AU - Weitzel, Utz

AU - Abad, David

AU - Abudy, Menachem (meni)

AU - Adrian, Tobias

AU - Ait-Sahalia, Yacine

AU - Akmansoy, Olivier

AU - Alcock, Jamie

AU - Alexeev, Vitali

AU - Aloosh, Arash

AU - Amato, Livia

AU - Amaya, Diego

AU - Angel, James J.

AU - Bach, Amadeus

AU - Baidoo, Edwin

AU - Bakalli, Gaetan

AU - Barbon, Andrea

AU - Bashchenko, Oksana

AU - Bindra, Parampreet Christopher

AU - Bjonnes, Geir Hoidal

AU - Black, Jeffrey R.

AU - Black, Bernard S.

AU - Bohorquez, Santiago

AU - Bondarenko, Oleg

AU - Bos, Charles S.

AU - Bosch-Rosa, Ciril

AU - Bouri, Elie

AU - Brownlees, Christian T.

AU - Calamia, Anna

AU - Cao, Viet Nga

AU - Capelle-Blancard, Gunther

AU - Capera, Laura

AU - Caporin, Massimiliano

AU - Carrion, Allen

AU - Caskurlu, Tolga

AU - Deev, Oleg

AU - Gehrig, Thomas

AU - Hartmann, Simon

AU - Hautsch, Nikolaus

AU - Kaiser, Gabriel

AU - Lindner, Thomas

AU - Simion, Giorgia

AU - Voigt, Stefan

AU - Weiss, Patrick

PY - 2024/6

Y1 - 2024/6

N2 - In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty: Non-standard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for better reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.

AB - In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence-generating process (EGP). We claim that EGP variation across researchers adds uncertainty: Non-standard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for better reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.

KW - non-standard errors

KW - liquidity

KW - multi-analyst approach

U2 - 10.1111/jofi.13337

DO - 10.1111/jofi.13337

M3 - Journal article

SN - 0022-1082

VL - 79

SP - 2339

EP - 2390

JO - Journal of Finance

JF - Journal of Finance

IS - 3

ER -