Bayes Theorem

Bayes' Theorem is a fundamental principle in probability theory that provides a way to update the probability of a hypothesis based on new evidence. It relates conditional probabilities and helps in calculating the probability of an event given prior knowledge or data.

Formula of Bayes' Theorem

The general formula of Bayes' Theorem is:

P(H|E) = (P(E|H) * P(H)) / P(E)

Where:

• P(H|E) is the posterior probability, the probability of the hypothesis $H$ given the evidence $E$.
• P(E|H) is the likelihood, the probability of the evidence $E$ given that the hypothesis $H$ is true.
• P(H) is the prior probability, the initial probability of the hypothesis $H$ before considering the evidence.
• P(E) is the marginal likelihood, the total probability of the evidence $E$ across all possible hypotheses.

Intuition Behind Bayes' Theorem

Bayes' Theorem helps in revising our belief about a hypothesis by incorporating new data. It allows us to update the probability of an event (hypothesis) when new evidence is available.

Example of Bayes' Theorem

Let’s consider an example of medical testing for a certain disease. Assume there is a test for a rare disease, and we want to know the probability that a person has the disease given that they tested positive.

Given Data:

• P(Disease) = 0.001 (The prior probability of having the disease is 0.1% or 1 in 1,000 people).
• P(No Disease) = 1 - P(Disease) = 0.999 (The prior probability of not having the disease is 99.9%).
• P(Positive|Disease) = 0.99 (The probability of testing positive if the person has the disease, i.e., the test's sensitivity, is 99%).
• P(Positive|No Disease) = 0.05 (The probability of testing positive if the person does not have the disease, i.e., the false positive rate, is 5%).

Now, we want to calculate the posterior probability that the person has the disease given that they tested positive, which is P(Disease|Positive).

Step 1: Calculate the Marginal Likelihood

First, we calculate the total probability of testing positive, P(Positive). This can happen either if the person has the disease or if they do not have the disease. Therefore, we calculate it as:

P(Positive) = P(Positive|Disease) * P(Disease) + P(Positive|No Disease) * P(No Disease)

Substituting the values:

P(Positive) = (0.99 * 0.001) + (0.05 * 0.999)

P(Positive) = 0.00099 + 0.04995 = 0.05094

Step 2: Apply Bayes' Theorem

Now, we can use Bayes' Theorem to calculate the posterior probability P(Disease|Positive):

P(Disease|Positive) = (P(Positive|Disease) * P(Disease)) / P(Positive)

Substituting the values:

P(Disease|Positive) = (0.99 * 0.001) / 0.05094

P(Disease|Positive) ≈ 0.00099 / 0.05094 ≈ 0.0194

So, P(Disease|Positive) ≈ 0.0194, or about 1.94%.

Interpretation of the Result:

Even though the person tested positive, the probability that they actually have the disease is only 1.94%. This is because the disease is rare (low prior probability), and the test has a relatively high false positive rate.

Conclusion

Bayes' Theorem provides a systematic way to update probabilities as new evidence becomes available. In the medical testing example, it demonstrates that even a positive result may not always imply a high probability of having the disease, especially when the disease is rare. This principle is widely applicable in various fields, including medicine, finance, and machine learning.

Ques. Let blue,green and red be three classes of objects with prior probabilities given by P(blue)=¼,P(green)=½,P(red)=¼.

Let there be three types of objects: pencils,pens and paper.Let the class-conditional probabilities of these objects be given as follows.

Use Bayes classifier to classify pencil,pen and paper.

• P(pencil/green)=⅓,
• P(pen/green)=½,
• P(paper/green)=⅙,
• P(pencil/blue)=½,
• P(pen/bue)=⅙,
• P(paper/blue)=⅓,
• P(pencil/red)=⅙,
• P(pen/red)=⅓,
• P(paper/red)=½.

Answer: