Appendix 1

Historical Origins and Interpretations of Probabilistic Inductive Logic

Perhaps the oldest and best understood way of representing inductive support is in terms of probability and the equivalent notion odds. Mathematicians have studied probability for over 350 years, but the concept is certainly much older. In recent times a number of other representations of partial belief and uncertain inference have emerged. Some of these approaches have found useful application in computer based artificial intelligence systems that perform inductive inferences in expert domains such as medical diagnosis. Nevertheless, probabilistic representations have predominated in such application domains. So, in this article we focus exclusively on probabilistic representations of inductive support. A brief comparative description of some of the most prominent alternative representations of uncertainty and support-strength can be found in the supplement Some Prominent Approaches to the Representation of Uncertain Inference.

The mathematical study of probability originated with Blaise Pascal and Pierre de Fermat in the mid-17^th century. From that time through the early 19^th century, as the mathematical theory continued to develop, probability theory was primarily applied to the assessment of risk in games of chance and to drawing simple statistical inferences about characteristics of large populations—e.g., to compute appropriate life insurance premiums based on mortality rates. In the early 19^th century Pierre de Laplace made further theoretical advances and showed how to apply probabilistic reasoning to a much wider range of scientific and practical problems. Since that time probability has become an indispensable tool in the sciences, business, and many other areas of modern life.

Throughout the development of probability theory various researchers appear to have thought of it as a kind of logic. But the first extended treatment of probability as an explicit part of logic was George Boole’s The Laws of Thought (1854). John Venn followed two decades later with an alternative empirical frequentist account of probability in The Logic of Chance (1876). Not long after that the whole discipline of logic was transformed by new developments in deductive logic.

In the late 19^th and early 20^th century Frege, followed by Russell and Whitehead, showed how deductive logic may be represented in the kind of rigorous formal system we now call quantified predicate logic. For the first time logicians had a fully formal deductive logic powerful enough to represent all valid deductive arguments that arise in mathematics and the sciences. In this logic the validity of deductive arguments depends only on the logical structure of the sentences involved. This development in deductive logic spurred some logicians to attempt to apply a similar approach to inductive reasoning. The idea was to extend the deductive entailment relation to a notion of probabilistic entailment for cases where premises provide less than conclusive support for conclusions. These partial entailments are expressed in terms of conditional probabilities, probabilities of the form \(P[C \pmid D] = r\) (read “the probability of C given D is r”), where P is a probability function, C is a conclusion sentence, D is a conjunction of premise sentences, and r is the probabilistic degree of support that premises D provide for conclusion C. Attempts to develop such a logic vary somewhat with regard to the ways in which they attempt to emulate the paradigm of formal deductive logic.

Some inductive logicians have tried to follow the deductive paradigm by attempting to specify inductive support probabilities solely in terms of the syntactic structures of premise and conclusion sentences. In deductive logic the syntactic structure of the sentences involved completely determines whether premises logically entail a conclusion. So these inductive logicians have attempted to follow suit. In such a system each sentence confers a syntactically specified degree of support on each of the other sentences of the language. Thus, the inductive probabilities in such a system are logical in the sense that they depend on syntactic structure alone. This kind of conception was articulated to some extent by John Maynard Keynes in his Treatise on Probability (1921). Rudolf Carnap pursued this idea with greater rigor in his Logical Foundations of Probability (1950) and in several subsequent works (e.g., Carnap 1952). (For details of Carnap’s approach see the section on logical probability in the entry on interpretations of the probability calculus, in this Encyclopedia.)

In the inductive logics of Keynes and Carnap, Bayes’ theorem, a straightforward theorem of probability theory, plays a central role in expressing how evidence comes to bear on hypotheses. Bayes’ theorem expresses how the probability of a hypothesis h on the evidence e, \(P[h \pmid e]\), depends on the probability that e should occur if h is true, \(P[e \pmid h]\), and on the probability of hypothesis h prior to taking the evidence into account, \(P[h]\) (called the prior probability of h). So, such approaches might well be called Bayesian logicist inductive logics. Carnap proposed a way to assess the values of both prior probabilities and likelihoods in terms of logical form alone, but his approach only applies to extremely simple languages. Other prominent Bayesian logicist approaches to the development of a probabilistic inductive logic depend on logical considerations that are more subtle than mere logical form. These include the works of Jeffreys (1939), Jaynes (1968), and Rosenkrantz (1981). Jaynes (1968), for instance, articulates a way to assign prior probabilities based on a notion of entropy, which is a measure of information content.

It is now widely held that the core idea of this syntactic approach to Bayesian logicism is fatally flawed—that syntactic logical structure cannot be the sole determiner of the degree to which premises inductively support conclusions. A crucial facet of the problem faced by syntactic Bayesian logicism involves how the logic is supposed to apply in scientific contexts where the conclusion sentence is some scientific hypothesis or theory, and the premises are evidence claims. The difficulty is that in any probabilistic logic that satisfies the usual axioms for probabilities, the inductive support for a hypothesis must depend in part on its prior probability. This prior probability represents (arguably) how plausible the hypothesis is taken to be on the basis of considerations other than the observational and experimental evidence (e.g., perhaps due to various plausibility arguments). A syntactic Bayesian logicist must tell us how to assign values to these pre-evidential prior probabilities of hypotheses in a way that relies only on the syntactic logical structure of the hypothesis, perhaps based on some measure of syntactic simplicity. There are severe problems with getting this idea to work. Various kinds of examples seem to show that such an approach must assign intuitively quite unreasonable prior probabilities to hypotheses in specific cases. Furthermore, for this idea to apply to the evidential support of real scientific theories, scientists would have to formalize theories in a way that makes their relevant syntactic structures apparent, and then evaluate theories solely on that syntactic basis (together with their syntactic relationships to evidence statements). Are we to evaluate alternative theories of gravitation, and alternative quantum theories, this way? This seems an extremely dubious approach to the evaluation of real scientific hypotheses and theories. Thus, it seems that logical structure alone may not suffice for the inductive evaluation of scientific hypotheses.

At about the time that the Bayesian logicist idea was developing, an alternative conception of probabilistic inductive reasoning was also emerging. This approach is now generally referred to as the Bayesian subjectivist or personalist approach to inductive reasoning (see, e.g., Ramsey 1926; De Finetti 1937; Savage 1954; Edwards, Lindman, & Savage 1963; Jeffrey 1983, 1992; Howson & Urbach 1993; Joyce 1999). This approach treats inductive probability as a measure of an agent’s degree-of-belief that a hypothesis is true, given the truth of the evidence. This approach was originally developed as part of a larger normative theory of belief and action called Bayesian decision theory. The principal idea is that the strength of an agent’s desires for various possible outcomes should combine with her belief-strengths regarding claims about the world to produce optimally rational decisions. Bayesian subjectivists provide a logic of decision that captures this idea, and they attempt to justify this logic by showing that in principle it leads to optimal decisions about which of various risky alternatives should be pursued. On the Bayesian subjectivist or personalist account of inductive probability, inductive probability functions represent the subjective (or personal) belief-strengths of ideally rational agents, the kind of belief strengths that figure into rational decision making. (See the section on subjective probability in the entry on interpretations of the probability calculus, in this Encyclopedia.)

Elements of a logicist conception of inductive logic live on today as part of the general approach called Bayesian inductive logic. However, among philosophers and statisticians the term ‘Bayesian’ is now most closely associated with the subjectivist or personalist account of belief and decision. And the term ‘Bayesian inductive logic’ has come to carry the connotation of a logic that involves purely subjective probabilities. This usage is misleading since, for inductive logics, the Bayesian/non-Bayesian distinction should really turn on whether the logic gives Bayes’ theorem a prominent role, or the approach largely eschews the use of Bayes’ theorem in inductive inferences, as do the classical frequentist approaches to statistical inference. Any inductive logic that employs the same probability functions to represent both the probabilities of evidence claims due to hypotheses and the probabilities of hypotheses due to those evidence claims must be a Bayesian inductive logic in this broader sense; because Bayes’ theorem follows directly from the axioms that each probability function must satisfy, and Bayes’ theorem expresses a necessary connection between the probabilities of evidence claims due to hypotheses and the probabilities of hypotheses due to those evidence claims.

In this article the probabilistic inductive logic we examine is a Bayesian inductive logic in this broader sense. This logic does not presuppose the subjectivist Bayesian theory of belief and decision, and avoids the objectionable features of the syntactic version of Bayesian logicism. There are good reasons to distinguish inductive probabilities from degree-of-belief probabilities (e.g. the so-called problem of old evidence) and from purely syntactic logical probabilities. So, the probabilistic logic articulated in this article is presented in a way that depends on neither of these conceptions of what the probability functions are. However, this version of the logic is general enough that it may perhaps be fitted to a Bayesian subjectivist or Bayesian logicist program, if one desires to do that.

There are also classical frequentist approaches to statistical inference, which are largely non-Bayesian. These approaches eschew the use of prior probabilities, viewing them as too subjective to employ in an objective evaluation of scientific claims. So they do not draw on Bayes’ theorem. These classical frequentist approaches where most famously developed by R. A. Fisher (1922) and by Neyman & Pearson (1967). Among the most prominent contemporary treatments of non-Bayesian frequentist inductive logic is articulated in the work on error statistics by Mayo and Spanos (see, e.g., Mayo (1996) (1997), and Mayo and Spanos (2006)). For a detailed account of frequentist approaches see the entry on Philosophy of Statistics, in this Encyclopedia.