MCMC sampling introduction
1.1 Monte Carlo (MC)
However, obtaining the iid sample is difficult, and the complete form of the posterior distribution must be known.
1.2 MCMC (Markov Chain MC)
MCMC generally generates a random sequence satisfying markov properties, which is ergodic and the limit distribution is . Based on Markov chains, get "not independent" samples from that have the same effect as iid sample.
2 Basic properties
The Markov chain satisfies: stationary distribution, ergodic (or irreducile and aperiodic).
Transition Probability (kernel)
3. Stationary (Invariant) Chains
A -finite measure is invariant for the transition kernel (and for the associated chain) if
The invariant distribution is also referred to as stationary if is a probability measure, since implies that for every ; thus, the chain is stationary in distribution.
4. Detailed balance condition
Detailed balance condition
A Markov chain with transition kernel satisfies the detailed balance condition (reversible) if there exists a function satisfying
for every .
Detailed balance condition provides a sufficient but not necessary condition for to be a stationary measure associated with the transition kernel .
Suppose that a Markov chain with transition function satisfies the detailed balance condition with a probability density function. Then:
(1) The density is the invariant density of the chain.
(2) The chain is reversible.
To prove the convergence (independence of initial conditions)
State is positive recurrent, if
A state has period ,
where gcd is greatest common divisor. Aperiodic means the gcd of any state is 1.
For any state , the probability of this chain going from state to state is positive. Namely, for ,
The law of large numbers for Markov chains
Suppose is a Markov chain with countable state space , and the transition probability matrix is , and suppose that it is irreducible and has stationary distribution . Then for any bounded functions and any initial distribution, have
A given Markov chain may have more than one invariant distribution.
Stationary + ergodicity equilibrium distribution (unique)
Stationary + irreducible + aperiodic unique(A ergodic Markov chain is irreducible. )