Tutorial on Bayesian Methods

Presented at the Workshop on Maximum Entropy and Bayesian Methods, held in Santa Fe, New Mexico, July 31, 1995 by

Wray Buntine
Wray.Buntine@hiit.fi
Written while at RIACS and Heuristicrats Research Inc.

Peter Cheeseman, Caelum Research Corp.
Computational Sciences Division
NASA Ames Research Center, MS 269-2
Moffett Field, CA, 94035-1000
[voice] +1 (415) 604-4946
cheesem@ptolemy.arc.nasa.gov

Outline

Parts of the tutorial are included below. Slides are in Postscript (Version 3.0) generated at 4 slides per page. When viewed you usually only see one slide at a time, so it is best to print the slides. Files with extension .gz have been compressed using gzip, available from The Free Software Foundation .

Basic probability theory
an introduction to the laws of probability by Peter; this and the next three topics (4 slides per page totalling 8 pages) (pdf 44 KB) (ps 162 KB) (ps.gz 49 KB)
Simple examples of Bayesian inference
reasoning about a double-headed coin, and HIV-tests, by Peter
Types of probabilistic inference
outlining induction, transduction, projective, MAP and direct inference, by Peter
Advanced modeling
considers overfitting, subjectivity, and Occam's razor, by Wray; (4 slides per page totalling 5 pages) (pdf 61 KB) (ps 414 KB) (ps.gz 46 KB)
Case studies
by Peter; not included
Graphical models
techniques for Bayesian modeling in the large, by Wray (4 slides per page totalling 3 pages) (pdf 57 KB) (ps 484 KB) (ps.gz 74 KB)
Decisions and Computation
introduction to decision theory and some computational methods, by Wray (pdf 40 KB) (ps 312 KB) (ps.gz 36 KB)
Other views and ideas
by Wray (4 slides per page totalling 4 pages) (pdf 33 KB) (ps 238 KB) (ps.gz 21 KB)
Additional slides
Polynomial fits (pdf 25 KB) (ps 41 KB) (ps.gz 12 KB)
Posterior samples (pdf 29 KB) (ps 45 KB) (ps.gz 13 KB)
Selected references
List (pdf 52 KB) (ps 40 KB) (ps.gz 13 KB). Two references of note (from many excellent texts and references) are

Bayesian Theory by J.M. Bernardo and A.F.M. Smith, John Wiley, 1994; a reference book
Bayesian Data Analysis by A. Gelman and J.B. Carlin and H.S. Stern and D.B. Rubin, Chapman & Hall, 1995; a graduate text book

Relevant on-line books and articles

Probability Theory: The Logic of Science by E.T. Jaynes. A book under development by Jaynes, the Bayesian physisist and advocate of the Maximum Entropy principle.
Bayesian Inference in Astrophysics by Tom Loredo. Tutorial introduction to Bayesian statistics.
An Introduction to Minimum Message Length Inference, and information and applications of MML and Bayesian reasoning at Minimum Message Length Encoding
Probabilistic networks survey paper on computational methods for Bayesian learning with probabilities
Bayes Factors and Model Uncertainty, by Robert Kass and Adrian Raftery, to appear in JASA.
Bayesian Model Selection in Sociology, by Adrian Raftery (with Discussion by Andrew Gelman & Donald Rubin, and Robert Hauser, and a Rejoinder)

Related pages of interest

Uncertainty in Artificial Intelligence:
area applies probabilistic reasoning to problems in intelligent systems, expert systems, planning, diagnosis, learning.
Inst. of Statistics and Decision Sciences, Duke Univ.
contains pointers to lots of good statistical stuff
International Society for Bayesian Analysis
annual conferences
Pattern Recognition Information
some pointers to good pattern recognition stuff
Bretthorst's site
for Bayesian and Maximum Entropy papers
Updated 5 May 2007 by Kenneth M. Hanson
wray@Ultimode.COM