Bayesian Networks

A Bayesian network is a probabilistic graphical model that represents a set of random variables and their conditional dependencies through a directed acyclic graph. The nodes are the variables and the directed edges indicate which variable depends directly on which. Judea Pearl formalized them in 1988. The edges can be interpreted as causal relationships, but this is not mandatory.

How they encode probability

Each node carries a conditional probability table that gives the probability of its values given those of its “parents.” With these, the network compactly represents the joint distribution of all the variables as the product of those local probabilities. This exploits the graph's conditional independencies so as not to have to enumerate the full joint distribution, which would grow unmanageably.

Inferring with the network

A Bayesian network makes it possible to compute the probability of some variables given others that are observed, which serves both for diagnosis—from effects to causes—and for prediction. There are two routes: exact inference, such as variable elimination, precise but costly in complex networks—the general problem is computationally very hard—and approximate inference, based on sampling, when exact inference is infeasible.

Where they are used

They are used in medical diagnosis, fault detection, spam filtering and risk assessment, and are tied to causal reasoning. Their temporal extension, dynamic Bayesian networks, models sequences; the hidden Markov model is, in fact, one of their simplest special cases.