Next Article in Journal
The Connection between Imported Inputs and Exports: The Importance of Strategic Interdependence
Previous Article in Journal
The Impact of Organizer Market Structure on Participant Entry Behavior in a Multi-Tournament Environment
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Editorial

Pre-Registration in Experimental Economics: An Editorial Policy

1
Behavioral Game Theory, ETH Zurich, 8092 Zurich, Switzerland
2
Markets and Norms, University of Zurich, 8050 Zurich, Switzerland
Games 2023, 14(1), 5; https://doi.org/10.3390/g14010005
Submission received: 14 December 2022 / Accepted: 15 December 2022 / Published: 4 January 2023
(This article belongs to the Section Learning and Evolution in Games)

1. Motivation

First and foremost, pre-registration is not the all-in-one solution for experimental economics. Pre-registration alone cannot overcome all methodological crises, such as the ongoing replicability crisis, reproducibility crisis, publication bias, etc. Even if strict pre-registration requirements were established with immediate effect across all journals publishing experimental economics papers, scientists could and likely would still abandon projects without clear results, run pilots, and carry out many other practices that would contribute to some of the aforementioned phenomena. Ultimately, there is no substitute for good taste, scientific integrity and honesty, and scientific misconduct will not be stopped, perhaps not even reduced, by any pre-registration policy.
Nevertheless, there is great benefit in pre-registration if the aim of a study is clearly set out to be that of falsifying a concrete theory or prediction, or of reproducing and replicating a prior study. That reason that this is the case succinctly relates to two types of empirical research that Karl Popper describes in his study on falsification and scientific method [1,2]. One of the two uses of empirics is as a basis to formulate new theories, i.e., for hypothesis generation. Such research may qualitatively describe data or make use of formal statistics, including the language of confidence and significance, yet clearly such research is exploratory and usually requires “playing around” with data as a theory is formulated in co-evolution with identifying patterns in the data. Perhaps the most important example of such research is Gregor Mendel’s experimentation with peas [3], the pioneer of modern genetics. In experimental economics, one example of a seminal hypothesis-generating experiment that was used for theory proposition is Ernst Fehr and Klaus Schmidt’s experimentation underlying “A theory of fairness, competition, and cooperation” [4]. That paper reproduces the falsification of narrow material self-interest of seminal earlier experiments, such as those from the Ultimatum and Dictator Games by Kahnemann, Knetsch and Thaler [5], and by Forsythe, Horowitz, Savin and Sefton [6]. It then proceeds to propose a theory that fits their own data, and qualitatively matches data from the aforementioned earlier studies.
Empirical observations, such as Mendel’s or Fehr-Schmidt’s, permit novel theory formulations without requiring high levels of power, and pilot studies may be run before scientists settle on the right experimental framework for their objectives. Falsification is not the goal of such studies since, in any case, novel theories are eventually formulated to match the patterns from data. According to Karl Popper, what must come next in the scientific process is that scientists, ideally not those who might have vested interests in the survival of their own theories, should reproduce and replicate original data analyses and then derive “risky” predictions from these theories, run controlled experiments to test them, and ideally falsify them. Any failure to falsify a theory, especially based on risky and novel predictions, corroborates the original theory, and we can think of this as increasing the “worth” of the scope for future falsification attempts.
Pre-registration is ideally suited for a hypothesis-testing kind of experiment (Table 1). Its aim is to falsify existing theories (as opposed to the hypothesis-generating experimentation that is used to formulate or illustrate a new theory). In the history of science, Albert Einstein’s thought experiments hold a special place of this kind of approach, as they are essentially pre-registrations of experiments to be conducted later. In economics, Vernon Smith’s counterexample-type experiments are an example. These experiments show that markets may converge to Walrasian competitive equilibria, which famously falsify the predominant view at the time that complete information and perfect knowledge are prerequisites for market efficiency [7].
With this Popperian distinction in mind between hypothesis-generating and hypothesis-testing experimentation, we can better understand one of the key problems with the scientific practice of experimental economics, which is a lack of appropriate language and norms for presenting results that clearly demarcate between the two approaches. In reality, many experimental papers in economics contain both kinds of analyses without a clear differentiation. We must agree on a language for presenting results that renders hypotheses-generating or a hypotheses-testing analyses unambiguous, the latter ideally being pre-registered, although the same does not apply to the former. Perhaps a certain language should be reserved for pre-registered analyses. Thus, pre-registration would be a tool, not a dogma.

2. Editorial Policy

For experimental papers to be submitted to Section Learning and Evolution in Games, the following editorial policy for experimental papers is implemented as of 1 January 2023:
  • By submitting an experimental paper to the section, authors make a “declaration of honor” that all data and analysis files will be made available along with publication of the article and that all other relevant data, including all data from pilots, is appropriately described in the article. If authors wish not to make their data and/or analysis files available, or if authors prefer not to disclose all data/pilots, a separate statement explaining the reasons needs to be submitted and will be appended to the article for review and publication purposes.
  • Submissions should label themselves as either hypothesis-generating or as hypothesis-testing. Authors are encouraged to pre-register all hypothesis-testing projects. All pre-registered analysis should be reported in the main sections of the paper. Other analyses should be labelled as exploratory.
  • Pre-registered experimental projects can be submitted instead of full papers and will be peer-reviewed with the possibility of provisional acceptance, without the inclusion of results from the pre-registered analysis, both before and after data collection.

Funding

This research is funded by the SNF under the Eccellenza Grant “Markets and Norms”.

Data Availability Statement

This article contains no data.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Popper, K. Logik der Forschung; Mohr Siebeck: Tübingen, Germany, 1934. [Google Scholar]
  2. Popper, K. Conjectures and Refutations: The Growth of Scientific Knowledge; Routledge: London, UK, 1963. [Google Scholar]
  3. Mendel, J.G. Versuche über Pflanzenhybriden. In Verhandlungen des Naturforschenden Vereines in Brünn; Im Verlage des Vereines: Brünn, Czech Republic, 1865; Volume IV, pp. 3–47. [Google Scholar]
  4. Fehr, E.; Schmidt, K.M. A theory of fairness, competition, and Cooperation. Q. J. Econ. 1999, 114, 817–868. [Google Scholar] [CrossRef]
  5. Kahneman, D.; Knetsch, J.L.; Thaler, R.H. Fairness and the assumptions of economics. J. Bus. 1986, 59, S285–S300. [Google Scholar] [CrossRef]
  6. Forsythe, R.; Horowitz, J.L.; Savin, N.E.; Sefton, M. Fairness in simple bargaining experiments. Games Econ. Behav. 1994, 6, 347–369. [Google Scholar] [CrossRef]
  7. Smith, V.L. An experimental study of competitive market behavior. J. Political Econ. 1962, 70, 111–137. [Google Scholar] [CrossRef] [Green Version]
Table 1. Analysis of hypothesis-testing.
Table 1. Analysis of hypothesis-testing.
Analysis: Exploration/Auxiliary Experiment/Illustration/Fitting New TheoryReplication/Reproduction/Falsification/Testing Existing Theory
Pre-registration required? NoYes
Provisional acceptance of pre-registered experiments possible? NoYes
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Nax, H.H. Pre-Registration in Experimental Economics: An Editorial Policy. Games 2023, 14, 5. https://doi.org/10.3390/g14010005

AMA Style

Nax HH. Pre-Registration in Experimental Economics: An Editorial Policy. Games. 2023; 14(1):5. https://doi.org/10.3390/g14010005

Chicago/Turabian Style

Nax, Heinrich H. 2023. "Pre-Registration in Experimental Economics: An Editorial Policy" Games 14, no. 1: 5. https://doi.org/10.3390/g14010005

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop