Mean Field Inference in Dependency Networks: An Empirical Study

Dependency networks are a compelling alternative to Bayesian networks for learning joint probability distributions from data and using them to compute probabilities. A dependency network consists of a set of conditional probability distributions, each representing the probability of a single variable given its Markov blanket. Running Gibbs sampling with these conditional distributions produces a joint distribution that can be used to answer queries, but suffers from the traditional slowness of sampling-based inference. In this paper, we observe that the mean field update equation can be applied to dependency networks, even though the conditional probability distributions may be inconsistent with each other. In experiments with learning and inference on 12 datasets, we demonstrate that mean field inference in dependency networks offers similar accuracy to Gibbs sampling but with orders of magnitude improvements in speed. Compared to Bayesian networks learned on the same data, dependency networks offer higher accuracy at greater amounts of evidence. Furthermore, mean field inference is consistently more accurate in dependency networks than in Bayesian networks learned on the same data.

Download:

• Paper (pdf)
• Model files (zip)
  • Tree-based Bayesian network
  • Tree-based dependency network
  • Table-based Bayesian network
• Additional scripts and experimental data available on request

Extra charts:

• Relative time differences
• Relative CMLL differences
• Experimental results for BN learned with table CPDs

All inference code is available in the Libra Toolkit, distributed under a modified BSD license (Libra license) which permits free inclusion in other commercial or open-source projects. The code release used for these experiments was Libra version 0.4.0.