Recently, I have been trying to learn about a concept called "Mixing Times" in Markov Chains (https://en.wikipedia.org/wiki/Markov_chain_mixing_time). These are used to analyze how long it takes for a Markov Chain to reach its stationary distribution (provided it has one).
I have been reading about this (I found limited information) and it seems that Mixing Times in Markov Chains are closely tied to the second eigenvalue of the transition matrix.
I was curious as to why this is and thus tried to derive/fill in the gaps for some of this by mashing different references together along with my own attempts to blend them together.
For a layman like myself, there were many parts that I needed to expand. I will show my whole work and then show the side results at the end.
MAIN PART OF QUESTION
Begin with a discrete-time Markov chain with state space $S = \{1, 2, ..., n\}$ and transition matrix $P$. Remember that the transition matrix $P$ is stochastic, meaning: $$ \sum_{j=1}^n P_{ij} = 1 \text{ for all } i $$
Apparently, by the Perron-Frobenius theorem (https://en.wikipedia.org/wiki/Perron%E2%80%93Frobenius_theorem), $P$ has a largest eigenvalue $\lambda_1 = 1$, and all other eigenvalues have magnitude less than or equal to $1$: $$ 1 = \lambda_1 > |\lambda_2| \geq |\lambda_3| \geq \cdots \geq |\lambda_n| $$
Let $v_i$ and $w_i$ be the right and left eigenvectors of $P$ corresponding to $\lambda_i$. These satisfy: $$ Pv_i = \lambda_i v_i \text{ and } w_i^T P = \lambda_i w_i^T $$
The spectral decomposition of $P$ is: $$ P = \sum_{i=1}^n \lambda_i v_i w_i^T $$
The stationary distribution $\pi$ satisfies (note the difference between to the power of $t$ and transpose): $$ \pi^T P = \pi^T $$
This means $\pi^T$ is the left eigenvector corresponding to $\lambda_1 = 1$.
Consider $P^t$, the $t$-step transition matrix: $$ P^t = (\sum_{i=1}^n \lambda_i v_i w_i^T)^t = \sum_{i=1}^n \lambda_i^t v_i w_i^T $$
We can separate the first term (corresponding to $\lambda_1 = 1$) from the rest (notice how the index in the sum changes): $$ P^t = v_1 w_1^T + \sum_{i=2}^n \lambda_i^t v_i w_i^T $$
Note that $v_1$ is a column of ones (denoted as $\mathbf{1}$) and $w_1^T = \pi^T$, so: $$ P^t = \mathbf{1}\pi^T + \sum_{i=2}^n \lambda_i^t v_i w_i^T $$
Now, let $p^t$ be the distribution after $t$ steps. We can write: $$ p^t = p^0 P^t = p^0 (\mathbf{1}\pi^T + \sum_{i=2}^n \lambda_i^t v_i w_i^T) $$
Simplify: $$ p^t = \pi + \sum_{i=2}^n \lambda_i^t (p^0 v_i) w_i^T $$
The error in distribution after $t$ steps is: $$ p^t - \pi = \sum_{i=2}^n \lambda_i^t (p^0 v_i) w_i^T $$
Taking the norm of both sides: $$ \|p^t - \pi\| \leq \sum_{i=2}^n |\lambda_i|^t \|p^0 v_i\| \|w_i\| $$
Since $|\lambda_2| \geq |\lambda_i|$ for $i > 2$, we can bound this by: $$ \|p^t - \pi\| \leq |\lambda_2|^t \sum_{i=2}^n \|p^0 v_i\| \|w_i\| $$
The sum is a constant independent of $t$, so we can write: $$ \|p^t - \pi\| \leq C |\lambda_2|^t $$ where $C = \sum_{i=2}^n \|p^0 v_i\| \|w_i\|$
We can see that $|\lambda_2|$ dominates the convergence behavior. As $t$ increases, $|\lambda_2|^t$ decays more slowly than $|\lambda_i|^t$ for $i > 2$, becoming the primary factor in determining how quickly $p^t$ approaches $\pi$.
The mixing time, which is roughly the time needed to get close to the stationary distribution, can be estimated by solving:
$$ C |\lambda_2|^t \approx \epsilon $$
Taking logarithms:
$$ t \approx \frac{\log(\epsilon/C)}{\log(|\lambda_2|)} \approx \frac{\text{constant}}{1 - |\lambda_2|} $$
Again, we can see that the mixing time/convergence is proportional to $\frac{1}{1 - |\lambda_2|}$.
CONCLUSION: This concludes the main part, beyond this I will try to show some of my algebra
1) Why the stationary distribution π is the left eigenvector corresponding to λ₁ = 1.
Stationary Distribution:
$$ \pi^T P = \pi^T $$
Compare this to the definition of a left eigenvector:
$$ w_i^T P = \lambda_i w_i^T $$
If we set $λᵢ = 1$ and $wᵢ = π$ in the eigenvector equation, we get:
$$ \pi^T P = 1 \cdot \pi^T $$
This is exactly the same as the equation defining the stationary distribution. Therefore, $π$ is a left eigenvector of $P$ with eigenvalue 1. We know from the Perron-Frobenius theorem that the largest eigenvalue of $P$ is $λ₁ = 1$. All other eigenvalues have magnitude less than or equal to $1$.
Since the stationary distribution corresponds to the eigenvalue $1$, and $1$ is the largest eigenvalue, we can conclude that the stationary distribution $π$ is specifically the left eigenvector corresponding to $λ₁ = 1$.
2) Logic for how we got $P^t$
We start with the spectral decomposition of P from step 5:
$$ P = \sum_{i=1}^n \lambda_i v_i w_i^T $$
To compute $P^t$, we need to multiply $P$ by itself $t$ times. Here is what happens when we multiply $P$ by itself:
$$ P^2 = P \cdot P = (\sum_{i=1}^n \lambda_i v_i w_i^T) \cdot (\sum_{j=1}^n \lambda_j v_j w_j^T) $$
When we multiply these sums, we get terms of the form:
$$ (\lambda_i v_i w_i^T) \cdot (\lambda_j v_j w_j^T) $$
Remember that eigenvectors is that they are orthonormal, i.e.:
$$ w_i^T v_j = \begin{cases} 1 & \text{if } i = j \\ 0 & \text{if } i \neq j \end{cases} $$
Using this property, we can simplify the multiplication:
$$ (\lambda_i v_i w_i^T) \cdot (\lambda_j v_j w_j^T) = \lambda_i \lambda_j v_i (w_i^T v_j) w_j^T = \begin{cases} \lambda_i^2 v_i w_i^T & \text{if } i = j \\ 0 & \text{if } i \neq j \end{cases} $$
This means that when we multiply P by itself, we get:
$$ P^2 = \sum_{i=1}^n \lambda_i^2 v_i w_i^T $$
Thus, when we multiply P by itself t times, we get:
$$ P^t = \sum_{i=1}^n \lambda_i^t v_i w_i^T $$
3) Why $v₁$ is a column of $1$:
$P$ is a stochastic matrix, meaning:
$$ \sum_{j=1}^n P_{ij} = 1 \text{ for all } i $$
Consider the right eigenvector equation for $λ₁ = 1$:
$$ Pv_1 = 1 \cdot v_1 $$
If we assume $v₁$ is a column of $$:
$$ P = \begin{pmatrix} P_{11} & P_{12} & \cdots & P_{1n} \\ P_{21} & P_{22} & \cdots & P_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ P_{n1} & P_{n2} & \cdots & P_{nn} \end{pmatrix} \begin{pmatrix} 1 \\ 1 \\ \vdots \\ 1 \end{pmatrix} $$
When we multiply:
$$ P = \begin{pmatrix} \sum_{j=1}^n P_{1j} \\ \sum_{j=1}^n P_{2j} \\ \vdots \\ \sum_{j=1}^n P_{nj} \end{pmatrix} $$
But since $P$ is stochastic, so each of these sums equals $1$:
$$ P = \begin{pmatrix} 1 \\ 1 \\ \vdots \\ 1 \end{pmatrix} = $$
Thus, $P\cdot = $, which is exactly the eigenvector equation for $λ₁ = 1$.
4) Expansion of $P^t$
Start with the equation from step 10:
$$ p^t = p^0 P^t = p^0 (\mathbf{1}\pi^T + \sum_{i=2}^n \lambda_i^t v_i w_i^T) $$
Distribute $p^0$ over the parentheses:
$$ p^t = p^0 \mathbf{1}\pi^T + p^0 \sum_{i=2}^n \lambda_i^t v_i w_i^T $$
Looking at $p^0 πᵀ$: $p^0$ is a row vector ($1$ × $n$) , $$ is a column vector of ones ($n × 1),πᵀ$ is a row vector ($1$ × $n$)
When we multiply $p^0 \cdot $, we get:
$$ p^0 \mathbf{1} = \sum_{j=1}^n p^0_j = 1 $$
This sum equals $1$ because $p^0$ is a probability distribution.
So, $p^0 πᵀ$ simplifies to just $πᵀ$:
$$ p^0 \mathbf{1}\pi^T = 1 \cdot \pi^T = \pi^T $$
For the second term, we can move the scalar $λᵢᵗ$ inside the parentheses:
$$ p^0 \sum_{i=2}^n \lambda_i^t v_i w_i^T = \sum_{i=2}^n \lambda_i^t (p^0 v_i) w_i^T $$
Note that $p^0 vᵢ$ is a scalar ($1$ × $n$ times $n × 1 = 1 × 1$).
Putting it all together:
$$ p^t = \pi^T + \sum_{i=2}^n \lambda_i^t (p^0 v_i) w_i^T $$
If we write $p^t$ as a row vector, so we can transpose both sides:
$$ (p^t)^T = \pi + \sum_{i=2}^n \lambda_i^t (p^0 v_i) w_i^T $$
5) Application of the triangle inequality:
$$ p^t - \pi = \sum_{i=2}^n \lambda_i^t (p^0 v_i) w_i^T $$
Take norm of both sides:
$$ \|p^t - \pi\| = \left\|\sum_{i=2}^n \lambda_i^t (p^0 v_i) w_i^T\right\| $$
Here we use the triangle inequality, i.e. the norm of a sum is less than or equal to the sum of the norms:
$$ \|p^t - \pi\| = \left\|\sum_{i=2}^n \lambda_i^t (p^0 v_i) w_i^T\right\| \leq \sum_{i=2}^n \left\|\lambda_i^t (p^0 v_i) w_i^T\right\| $$
Here we use the properties of norms. For a scalar $a$ and vectors $x$ and $y$, we have $\|axy^T\| = |a| \|x\| \|y\|$. In our case, $a = \lambda_i^t$, $x = p^0 v_i$, and $y = w_i$. Applying this:
$$ \|p^t - \pi\| \leq \sum_{i=2}^n |\lambda_i|^t \|p^0 v_i\| \|w_i\| $$
The eigenvalues are constant, therefore $|\lambda_2|^t$ is a constant with respect to the summation index $i$, so we can factor it out:
$$ \|p^t - \pi\| \leq |\lambda_2|^t \sum_{i=2}^n \|p^0 v_i\| \|w_i\| $$