Winnie的学习手册
2022/08/26阅读：11主题：前端之巅同款
MCMC sampling introduction
1 Motivation
1.1 Monte Carlo (MC)
However, obtaining the iid sample is difficult, and the complete form of the posterior distribution must be known.
1.2 MCMC (Markov Chain MC)
MCMC generally generates a random sequence satisfying markov properties, which is ergodic and the limit distribution is . Based on Markov chains, get "not independent" samples from that have the same effect as iid sample.
2 Basic properties
The Markov chain satisfies: stationary distribution, ergodic (or irreducile and aperiodic).

Markov property

Transition Probability (kernel)

Marginal Distribution
3. Stationary (Invariant) Chains
A finite measure is invariant for the transition kernel (and for the associated chain) if
The invariant distribution is also referred to as stationary if is a probability measure, since implies that for every ; thus, the chain is stationary in distribution.
4. Detailed balance condition
Detailed balance condition
A Markov chain with transition kernel satisfies the detailed balance condition (reversible) if there exists a function satisfying
for every .
Proof.
Remark：
Detailed balance condition provides a sufficient but not necessary condition for
to be a stationary measure associated with the transition kernel
.
Theorem
Suppose that a Markov chain with transition function satisfies the detailed balance condition with a probability density function. Then:
(1) The density is the invariant density of the chain.
(2) The chain is reversible.
Reversible:
5. Ergodicity
To prove the convergence (independence of initial conditions)

positive recurrent
State is positive recurrent, if 
aperiodic
A state has period ,
where gcd is greatest common divisor. Aperiodic means the gcd of any state is 1.
6. Irreducible
For any state , the probability of this chain going from state to state is positive. Namely, for ,
The law of large numbers for Markov chains
Suppose is a Markov chain with countable state space , and the transition probability matrix is , and suppose that it is irreducible and has stationary distribution . Then for any bounded functions and any initial distribution, have
Remark.

A given Markov chain may have more than one invariant distribution. 
Stationary + ergodicity equilibrium distribution (unique) 
Stationary + irreducible + aperiodic unique(A ergodic Markov chain is irreducible. )
作者介绍
Winnie的学习手册
在读统计博士，主要分享可靠性统计等