Bayesian Inference

Bayesian inference is a statistical method for updating the probability of a hypothesis as new evidence becomes available, via Bayes' theorem. Its central idea is captured in one relation: the posterior probability is proportional to the likelihood times the prior probability.

The three elements

The method combines three pieces. The prior probability captures the belief about the hypothesis before seeing the data. The likelihood measures how probable the observed evidence is if the hypothesis were true. And the posterior probability is the updated belief after observing the evidence, combining the previous two. The process can be chained: today's posterior is tomorrow's prior.

Not the same as “artificial intelligence”

A confusion is worth correcting: Bayesian inference is not a synonym for artificial intelligence. It is a framework of statistics and probability, with applications far beyond AI—science, medicine, engineering. It is used within some machine-learning methods, such as the naive Bayes classifier, Bayesian networks or Bayesian hyperparameter optimization, but that does not make the two concepts interchangeable. Against the Bayesian approach, the frequentist one treats parameters as fixed unknown values, whereas the Bayesian treats them as random variables with a distribution; both are legitimate and the choice depends on the problem.

Uses in machine learning

In machine learning, Bayesian inference is especially useful for quantifying uncertainty: Bayesian neural networks treat the weights as distributions rather than single values, which allows estimating how much confidence each prediction deserves. It also underpins Bayesian hyperparameter optimization, Bayesian networks and recursive estimation methods such as the Kalman filter.