MCMC sampling introduction

1 Motivation

1.1 Monte Carlo (MC)

where is iid sample from the posterior distribution .

However, obtaining the iid sample is difficult, and the complete form of the posterior distribution must be known.

1.2 MCMC (Markov Chain MC)

MCMC generally generates a random sequence satisfying markov properties, which is ergodic and the limit distribution is . Based on Markov chains, get "not independent" samples from that have the same effect as iid sample.

2 Basic properties

The Markov chain satisfies: stationary distribution, ergodic (or irreducile and aperiodic).

  • Markov property

  • Transition Probability (kernel)

    Homogeneous: the transition kernel are the same for all .

  • Marginal Distribution

3. Stationary (Invariant) Chains

A -finite measure is invariant for the transition kernel (and for the associated chain) if

The invariant distribution is also referred to as stationary if is a probability measure, since implies that for every ; thus, the chain is stationary in distribution.

4. Detailed balance condition

Detailed balance condition

A Markov chain with transition kernel satisfies the detailed balance condition (reversible) if there exists a function satisfying

for every .


Detailed balance condition provides a sufficient but not necessary condition for to be a stationary measure associated with the transition kernel .


Suppose that a Markov chain with transition function satisfies the detailed balance condition with a probability density function. Then:
(1) The density is the invariant density of the chain.
(2) The chain is reversible.


5. Ergodicity

To prove the convergence (independence of initial conditions)

  • positive recurrent
    State is positive recurrent, if where is the first time to return state .
  • aperiodic
    A state has period ,

where gcd is greatest common divisor. Aperiodic means the gcd of any state is 1.

6. Irreducible

For any state , the probability of this chain going from state to state is positive. Namely, for ,

The law of large numbers for Markov chains

Suppose is a Markov chain with countable state space , and the transition probability matrix is , and suppose that it is irreducible and has stationary distribution . Then for any bounded functions and any initial distribution, have


  • A given Markov chain may have more than one invariant distribution.
  • Stationary + ergodicity equilibrium distribution (unique)
  • Stationary + irreducible + aperiodic unique(A ergodic Markov chain is irreducible. )