Bayes' theorem

In probability theory, Bayes' theorem (often called Bayes' law after Thomas Bayes) relates the conditional and marginal probabilities of two random events. It is often used to compute posterior probabilities given observations. For example, a patient may be observed to have certain symptoms. Bayes' theorem can be used to compute the probability that a proposed diagnosis is correct, given that observation. (See example 2)

As a formal theorem, Bayes' theorem is valid in all common interpretations of probability. However, it plays a central role in the debate around the foundations of statistics: frequentist and Bayesian interpretations disagree about the ways in which probabilities should be assigned in applications. Frequentists assign probabilities to random events according to their frequencies of occurrence or to subsets of populations as proportions of the whole, while Bayesians describe probabilities in terms of beliefs and degrees of uncertainty. The articles on Bayesian probability and frequentist probability discuss these debates in greater detail.
Contents
[hide]

* 1 Statement of Bayes' theorem
* 2 An example
* 3 Bayes' theorem in terms of likelihood
* 4 Derivation from conditional probabilities
* 5 Alternative forms of Bayes' theorem
o 5.1 Bayes' theorem in terms of odds and likelihood ratio
o 5.2 Bayes' theorem for probability densities
o 5.3 Abstract Bayes' theorem
o 5.4 Extensions of Bayes' theorem
* 6 Further examples
o 6.1 Example 1: Drug testing
o 6.2 Example 2: Bayesian inference
o 6.3 Example 3: The Monty Hall problem
* 7 Historical remarks
* 8 See also
* 9 References
o 9.1 Versions of the essay
o 9.2 Commentaries
o 9.3 Additional material

[edit] Statement of Bayes' theorem

Bayes' theorem relates the conditional and marginal probabilities of events A and B, where B has a non-vanishing probability:

P(A|B) = \frac{P(B | A)\, P(A)}{P(B)}.

Each term in Bayes' theorem has a conventional name:

* P(A) is the prior probability or marginal probability of A. It is "prior" in the sense that it does not take into account any information about B.
* P(A|B) is the conditional probability of A, given B. It is also called the posterior probability because it is derived from or depends upon the specified value of B.
* P(B|A) is the conditional probability of B given A.
* P(B) is the prior or marginal probability of B, and acts as a normalizing constant.

Intuitively, Bayes' theorem in this form describes the way in which one's beliefs about observing 'A' are updated by having observed 'B'.

[edit] An example

Suppose there is a co-ed school having 60% boys and 40% girls as students. The girl students wear trousers or skirts in equal numbers; the boys all wear trousers. An observer sees a (random) student from a distance; all they can see is that this student is wearing trousers. What is the probability this student is a girl?

It is clear that the probability is less than 40%, but by how much? Is it half that, since only half the girls are wearing trousers? The correct answer can be computed using Bayes' theorem.

The event A is that the student observed is a girl, and the event B is that the student observed is wearing trousers. To compute P(A|B), we first need to know:

* P(A), or the probability that the student is a girl regardless of any other information. Since the observers sees a random student, meaning that all students have the same probability of being observed, and the fraction of girls among the students is 40%, this probability equals 0.4.
* P(A'), or the probability that the student is a boy regardless of any other information (A' is the complementary event to A). This is 60%, or 0.6.
* P(B|A), or the probability of the student wearing trousers given that the student is a girl. As they are as likely to wear skirts as trousers, this is 0.5.
* P(B|A'), or the probability of the student wearing trousers given that the student is a boy. This is given as 1.
* P(B), or the probability of a (randomly selected) student wearing trousers regardless of any other information. Since P(B) = P(B|A)P(A) + P(B|A')P(A'), this is 0.5×0.4 + 1×0.6 = 0.8.

Given all this information, the probability of the observer having spotted a girl given that the observed student is wearing trousers can be computed by substituting these values in the formula:

P(A|B) = \frac{P(B|A) P(A)}{P(B)} = \frac{0.5 \times 0.4}{0.8} = 0.25.

As expected, it is less than 40%, but more than half that.

Another, essentially equivalent way of obtaining the same result is as follows. Assume, for concreteness, that there are 100 students, 60 boys and 40 girls. Among these, 60 boys and 20 girls wear trousers. All together there are 80 trouser-wearers, of which 20 are girls. Therefore the chance that a random trouser-wearer is a girl equals 20/80 = 0.25.

It is often helpful when calculating conditional probabilities to create a simple table containing the number of occurrences of each outcome, or the relative frequencies of each outcome, for each of the independent variables. The table below illustrates the use of this method for the above girl-or-boy example
Girls Boys Total