## Notes to Reproducibility of Scientific Results

1.
Here, the inverse probability fallacy occurs when someone mistakes
the probability of proposition *p* given proposition *q* for
the probability of proposition *q* given proposition *p*.
Consider the difference between the probability that someone is a
human given that they are a female human (a probability of 100%)
versus the probability that someone is a female human given that they
are a human (a probability of around 50%). In this sense, someone
would commit the fallacy if they believed that the *p* value
specifies that probability that the null hypothesis is true given the
results instead of it specifying the probability of obtaining the
results if the null hypothesis were true. Consequently, if someone
believes that the *p* value specifies the probability of the null
hypothesis given the evidence, then why would they worry about the
statistical power—the *a priori* probability of correctly
rejecting the null hypothesis and detecting an effect when it is
really present?

2. Like Feest (2016), other philosophers of science may recognize a resemblance between the experimenter’s regress and the Duhem-Quine problem: both problems concern the limitations of evidence insofar as it putatively underdetermines which propositions to accept.

3.
Technically, the term \(p(T \mid F_N)\) should instead be \(p(T \mid
F_1 \amp F_2 \amp \ldots \amp F_N)\) since \(p(T \mid F_N)\) is the
probability of *T* conditional on only the last replication
failure \(F_N\) and not the other failures (unless, of course, by
\(p(T \mid F_N)\), they really mean some other probability function
\(p\prime\) which has conditionalized on the other failures and would
thus be a different probability function than the one used for \(
p(T))\). The other replications failures would ideally feature in the
antecedent conditions of particular other conditional probabilities
too. Regardless, we have reproduced Earp and Trafimow’s
presentation of the Bayesian framework here despite these technical
details.

4.
Technically, this is only given an additional principle not discussed
explicitly by Earp and Trafimow. This is the principle of
conditionalisation whereby one’s so-called *posterior*
degree of belief in a proposition *P* *after* the receipt
of some evidence *E* should be equal to their *prior*
degree of belief in *P* given *E* *before* the
receipt of evidence *E*. Earp and Trafimow only explicitly
discuss the relations between degrees of belief at a given time
(synchronic relations), rather than the relations between degrees of
belief at two different points in time (diachronic relations).

5. For example, critics have referred
to replication and open science advocates as “replication
police” and “shameless little bullies”
(Gilbert 2014 online post)
and accused “angry mobs” of conducting an
“inquisition”
(Bohannon 2014 online post).
High profile replication studies, such as that of Amy Cuddie’s
power poses, have been used in attempts to establish bullying charges
(Dominus 2017). In a publicly released draft of a guest column for the
Association of Psychological Science (APS) *Observer*, past APS
president Susan Fiske referred to those involved in replication and
meta-science work as “methodological terrorists”
(Fiske 2016 draft).
She was especially scathing of “public shaming and
blaming” by “online vigilantes” and
“destructo-critics”. In the final published version of
this column (Fiske 2016), she retracted some of these phrases, but the
draft had already sparked many social media responses from open
science advocates who largely view the tone debate as a distraction
from their key issues (e.g.,
Chambers 2017 online post, and Sutton 2018).