Markov Chains and Unemployment

Honours Intermediate Macro

Discrete-state stochastic processes, transition matrices, stationary distributions, and unemployment dynamics

Author

Affiliation

Jesse Perla

University of British Columbia

Markov Chains

A model of a stochastic process with a discrete number of states.

Random Variables and Mathematical Expectation

Notation for discrete states:

$n = 1, \ldots, N$ represents possible “states of the world” (e.g., individual unemployed, employed, …)
$\pi_n = {\mathbb{P}_{}\left( {\text{state of the world is } n} \right)}$
- $\pi_n \geq 0$, $\sum_{n=1}^{N} \pi_n = 1$, i.e., the world must be in one of the states
- Stack as a vector: $\pi \equiv \begin{bmatrix} \pi_1 & \ldots & \pi_N \end{bmatrix}$
Random variable $Y \in \left\{{y_1, \ldots, y_N}\right\}$
Values mapping states of the world for r.v. $Y$: $y = \begin{bmatrix} y_1 \\ y_2 \\ \vdots \\ y_N \end{bmatrix}$

e.g., if event $n$ is unemployed, then income if unemployed is $y_n$

Mathematical Expectation:

${\mathbb{E}_{{}}\left[ {Y} \right]} = \sum_{n=1}^{N} {\mathbb{P}_{}\left( {Y = y_n} \right)} y_n = \sum_{n=1}^{N} \pi_n y_n = \pi \cdot y$ (i.e., inner product)
- Weight the realizations with the probabilities
e.g., if probability of unemployment is $\pi_1 = 0.1$, income from unemployment insurance is $y_1 = 15{,}000$; probability of employment is $\pi_2 = 0.9$, income from employment is $y_2 = 40{,}000$. Then expected income (or average across states of world):
- ${\mathbb{E}_{{}}\left[ {Y} \right]} = (0.1 \times 15{,}000) + (0.9 \times 40{,}000)$
We could use this to find an individual’s expected income at some point in the future. Alternatively, we can use this to find averages for a continuum of population—a step towards aggregation.
e.g., if 10% of population is unemployed at $15,000 and 90% of population is employed at $40,000, then the average income is ${\mathbb{E}_{{}}\left[ {Y} \right]}$

Transitions

Let $\phi$ = probability to become employed, and $\alpha$ = probability to lose job.

Figure 1: Two-state Markov chain for employment transitions

Let State $1 \leftrightarrow E$ (Employed), State $2 \leftrightarrow U$ (Unemployed).

Transition Matrix:

\[ P = \begin{array}{@{}cc@{}} \begin{array}{@{}cc@{}} E_{t+1} & U_{t+1} \end{array} \\ \begin{bmatrix} 1-\alpha & \alpha \\ \phi & 1-\phi \end{bmatrix} \end{array} \kern0.5em \begin{array}{@{}l@{}} \quad \\ E_t \\ U_t \end{array} \]

Let $\pi_t$ be the probability mass function (pmf) of a random variable of an agent’s employment status at time $t$. This is a probability mass function (pmf) since the possible events are discrete.

If employed at time 0: $\pi_0 = \begin{bmatrix} 1 & 0 \end{bmatrix}$
If 50% chance of employment at time 3: $\pi_3 = \begin{bmatrix} \frac{1}{2} & \frac{1}{2} \end{bmatrix}$

Evolution of Probability Distribution

Find the evolution of the probability mass function for the random variable with the transition matrix $P$. A property of Markov chains:

\[ \pi_{t+1} = \pi_t \cdot P \]

Careful with the order of the matrix product!

Iterate forward:

\[ \pi_{t+j} = \pi_t \cdot P^j \]

Example: Started employed at $t=0$, i.e., $\pi_0 = \begin{bmatrix} 1 & 0 \end{bmatrix}$.

Probability of unemployment/employment at $t=1$:

\[ \pi_1 = \pi_0 \cdot P = \begin{bmatrix} 1 & 0 \end{bmatrix} \begin{bmatrix} 1-\alpha & \alpha \\ \phi & 1-\phi \end{bmatrix} = \begin{bmatrix} 1-\alpha & \alpha \end{bmatrix} \]

At time 2:

\[ \pi_2 = \pi_1 \cdot P = \begin{bmatrix} 1-\alpha & \alpha \end{bmatrix} \cdot \begin{bmatrix} 1-\alpha & \alpha \\ \phi & 1-\phi \end{bmatrix} = \begin{bmatrix} (1-\alpha)^2 + \alpha\phi & (1-\alpha)\alpha + \alpha(1-\phi) \end{bmatrix} \]

Interpretation:

\[ = \begin{bmatrix} {\mathbb{P}_{}\left( {E \to E, E \to E} \right)} + {\mathbb{P}_{}\left( {E \to U, U \to E} \right)} & {\mathbb{P}_{}\left( {E \to E, E \to U} \right)} + {\mathbb{P}_{}\left( {E \to U, U \to U} \right)} \end{bmatrix} \]

Iterating forward:

\[ \pi_{t+j} = \pi_t \cdot \underbrace{P \cdot P \cdots P}_{j \text{ times}} = \pi_t \cdot P^j \]

Stationarity and asymptotics. One possibility:

\[ \pi_{\infty} = \lim_{j \to \infty} \pi_{t+j} = \lim_{j \to \infty} \pi_t \cdot P^j \]

Another is to find a $\pi_{\infty}$ which doesn’t change:

\[ \pi_{\infty} = \pi_{\infty} P \]

Questions:

Does a unique limit exist? Is it independent of $\pi_t$?
Is there an absorbing state? (e.g., all end up unemployed forever)
These answers depend on $P$.
In some cases, we will refer to $\pi_{\infty}$ as the stationary distribution.

Example: Stationary Distribution

Non-Degenerate Stationary Distribution

Consider:

\[ P = \begin{bmatrix} 1-\alpha & \alpha \\ \phi & 1-\phi \end{bmatrix}, \quad 0 < \alpha < 1, \quad 0 < \phi < 1 \]

The stationary random variable $\pi_{\infty}$ satisfies:

\[ \pi_{\infty} = \pi_{\infty} \cdot P \]

i.e., it’s the r.v. associated with $P$ such that it doesn’t change between periods.

Remark: In linear algebra, the left eigenvector associated with the unit eigenvalue.

To find $\pi_{\infty}$:

Use software to find the left eigenvector, or
Solve system for simple examples:

Let $\bar{\pi} = \text{prob of being employed}$; $\pi_{\infty} = \begin{bmatrix} \bar{\pi} & 1-\bar{\pi} \end{bmatrix}$.

Then:

\[ \begin{bmatrix} \bar{\pi} & 1-\bar{\pi} \end{bmatrix} = \begin{bmatrix} \bar{\pi} & 1-\bar{\pi} \end{bmatrix} \cdot \begin{bmatrix} 1-\alpha & \alpha \\ \phi & 1-\phi \end{bmatrix} \]

\[ \Rightarrow \begin{bmatrix} \bar{\pi} \\ 1-\bar{\pi} \end{bmatrix}' = \begin{bmatrix} \bar{\pi}(1-\alpha) + (1-\bar{\pi})\phi \\ \bar{\pi} \cdot \alpha + (1-\bar{\pi})(1-\phi) \end{bmatrix}' \]

First equation:

\[ \bar{\pi} = (1-\alpha)\bar{\pi} - \phi\bar{\pi} + \phi \implies (1 - (1-\alpha) + \phi)\bar{\pi} = \phi \]

\[ \boxed{\bar{\pi} = \frac{\phi}{\alpha + \phi}} \tag{1}\]

\[ \boxed{\pi_{\infty} = \begin{bmatrix} \frac{\phi}{\alpha + \phi} & \frac{\alpha}{\alpha + \phi} \end{bmatrix}} \tag{2}\]

Second equation: would find identical solution (luckily, since there is only 1 variable and 2 equations).

Unemployment Application

Assume:

\[ P = \begin{bmatrix} 1-\alpha & \alpha \\ \phi & 1-\phi \end{bmatrix} \quad \begin{array}{c} \leftrightarrow E \\ \leftrightarrow U \end{array} \]

Invariant Distribution (i.e., “long run”):

\[ \bar{\pi} = {\mathbb{P}_{}\left( {E} \right)}, \quad 1-\bar{\pi} = {\mathbb{P}_{}\left( {U} \right)} \]

solves:

\[ \begin{bmatrix} \bar{\pi} & 1-\bar{\pi} \end{bmatrix} \begin{bmatrix} 1-\alpha & \alpha \\ \phi & 1-\phi \end{bmatrix} = \begin{bmatrix} \bar{\pi} & 1-\bar{\pi} \end{bmatrix} \]

From the equation $\bar{\pi}(1-\alpha) + \phi(1-\bar{\pi}) = \bar{\pi}$:

\[ \bar{\pi} = \frac{\phi}{\alpha + \phi}, \quad 1-\bar{\pi} = \frac{\alpha}{\alpha + \phi} \]

Dividing top and bottom by $\phi\alpha$:

\[ 1 - \bar{\pi} = \frac{1/\phi}{1/\phi + 1/\alpha} \]

Average Unemployment Spell

In each period an unemployed person gets a job with probability $\phi$. Otherwise, stays unemployed.

Figure 2: Waiting time to find employment

Let $N$ be the random variable “length of time it takes to find a job”. $N=1$ means 1 period.

Let $p_j = {\mathbb{P}_{}\left( {N = j} \right)}$.

Then:

$p_1 = \phi$ (success on first try)
$p_2 = \phi(1-\phi)$ (fail, success)
$p_3 = \phi(1-\phi)^2$ (fail, fail, success)

\[ \boxed{p_j = \phi(1-\phi)^{j-1}} \]

This is a proper probability distribution:

\[ \sum_{j=1}^{\infty} p_j = \phi \sum_{j=1}^{\infty} (1-\phi)^{j-1} = \phi \sum_{j=0}^{\infty} (1-\phi)^j = \frac{\phi}{1-(1-\phi)} = 1 \]

Another geometric series result:

\[ \boxed{\sum_{j=1}^{\infty} j a^{j-1} = \frac{1}{(1-a)^2} \text{ for } |a| < 1} \]

Expected waiting time:

\[ {\mathbb{E}_{{}}\left[ {N} \right]} = \text{expected/mean time to find a job} = \sum_{j=1}^{\infty} j \cdot p_j = \phi \sum_{j=1}^{\infty} j(1-\phi)^{j-1} = \phi \cdot \frac{1}{(1-(1-\phi))^2} = \frac{1}{\phi} \]

So the average number of periods in unemployment $= \frac{1}{\phi}$.

More generally, this is the mean waiting time for a geometric distribution: if arrivals happen with probability $a$, then the expected wait time $= \frac{1}{a}$.

Summary of Formulas

$\bar{\pi} =$ proportion employed, $1-\bar{\pi} =$ proportion unemployed
$\bar{\pi} = \dfrac{\phi}{\alpha + \phi}, \quad 1 - \bar{\pi} = \dfrac{\alpha}{\alpha + \phi} = \dfrac{1/\phi}{1/\phi + 1/\alpha}$
$\mathbb{E}_{}\left[{\text{\# of periods to become employed}} \; \middle| \; {\text{start unemployed}} \right] = \frac{1}{\phi}$
$\mathbb{E}_{}\left[{\text{\# of periods to become unemployed}} \; \middle| \; {\text{start employed}} \right] = \frac{1}{\alpha}$

Calibration Example

Lets say you had the following data;

Average unemployment duration = 16.8 weeks = 3.87 months
Civilian unemployment: 4.7%
Employment / population: 63%
Labor force / population: 66%
Civilian population: 231 million
Civilian labor force: 153 million = 231 × 66% (not institutional military, etc.)
Unemployment: 7 million = 153 million × 4.7%

Stationary Distribution:

$1 - \bar{\pi} = 0.047$ (proportion unemployed)
$\frac{1}{\phi} = 3.87$ (average unemployment length, in months)

From the equation for the stationary distribution:

\[ 1 - \bar{\pi} = \frac{1/\phi}{1/\phi + 1/\alpha} \]

\[ \Rightarrow 0.047 = \frac{3.87}{3.87 + 1/\alpha} \]

Solving for $\frac{1}{\alpha}$:

\[ \frac{1}{\alpha} = 78.8 \]

i.e., average job length is 78 months.

So, the transition matrix is:

\[ P = \begin{bmatrix} 1 - \frac{1}{78.8} & \frac{1}{78.8} \\ \frac{1}{3.87} & 1-\frac{1}{3.87} \end{bmatrix} \approx \begin{bmatrix} 0.987 & 0.013 \\ 0.258 & 0.742 \end{bmatrix} \]

Stationary distribution:

\[ \pi_{\infty} = \begin{bmatrix} 0.953 & 0.047 \end{bmatrix} \]

Questions:

Total jobs destroyed/month: $0.013 \times 146\text{ million} \approx 1.85\text{ million}$
If employed worker today, what is the probability to be employed in $j$ months?

\[ {\mathbb{P}_{}\left( {E \text{ at } j} \right)} = \begin{bmatrix} 1 & 0 \end{bmatrix} \cdot \left( \begin{bmatrix} 1 & 0 \end{bmatrix} P^j \right)' \]

What about as $j \to \infty$? ${\mathbb{P}_{}\left( {E \text{ at } j \to \infty} \right)} = \bar{\pi}$

The economy is away from its stationary equilibrium: $\pi_0 \neq \pi_{\infty}$. What is the predicted sequence of unemployment rates?

\[ \pi_j = \begin{bmatrix} 0 & 1 \end{bmatrix} \cdot \left[ \pi_0 P^j \right]' \]

Appendices

Absorbing State of Unemployment Back

Let $P = \begin{bmatrix} \alpha & 1 - \alpha \\ 0 & 1 \end{bmatrix}$,

i.e., $\alpha$ chance to stay employed, and in unemployment never get a job (“absorbing”).

Let $\pi_0 = \begin{bmatrix} a & 1-a \end{bmatrix}$.

\[ \pi_1 = \pi_0 P = \begin{bmatrix} a & 1-a \end{bmatrix} \begin{bmatrix} \alpha & 1-\alpha \\ 0 & 1 \end{bmatrix} = \begin{bmatrix} \alpha a & (1-\alpha)a + 1 - a \end{bmatrix} = \begin{bmatrix} \text{kept job} & \text{lost job or never had one} \end{bmatrix} \]

\[ \pi_2 = \pi_1 P = \begin{bmatrix} \alpha a & (1-\alpha)a + 1 - a \end{bmatrix} \begin{bmatrix} \alpha & 1-\alpha \\ 0 & 1 \end{bmatrix} = \begin{bmatrix} \alpha^2 a & (1-\alpha)\alpha a + (1-\alpha)a + (1-a) \end{bmatrix} \]

Note: $\alpha^2 a$ represents keeping job twice.

General pattern:

\[ \pi_j = \pi_0 \cdot P^j = \begin{bmatrix} \alpha^j \cdot a & 1 - \alpha^j \cdot a \end{bmatrix} \]

\[ \lim_{j \to \infty} \pi_j = \begin{bmatrix} 0 & 1 \end{bmatrix} \]

i.e., all end up unemployed, independent of $\pi_0$.

Or via powers of $P$:

\[ P^2 = \begin{bmatrix} \alpha & 1-\alpha \\ 0 & 1 \end{bmatrix} \begin{bmatrix} \alpha & 1-\alpha \\ 0 & 1 \end{bmatrix} = \begin{bmatrix} \alpha^2 & \alpha(1-\alpha) + (1-\alpha) \\ 0 & 1 \end{bmatrix} = \begin{bmatrix} \alpha^2 & 1-\alpha^2 \\ 0 & 1 \end{bmatrix} \]

Generalize:

\[ P^j = \begin{bmatrix} \alpha^j & 1-\alpha^j \\ 0 & 1 \end{bmatrix}, \quad \lim_{j \to \infty} P^j = \begin{bmatrix} 0 & 1 \\ 0 & 1 \end{bmatrix} \]

\[ \pi_{\infty} = \lim_{j \to \infty} \pi_0 P^j = \begin{bmatrix} a & 1-a \end{bmatrix} \begin{bmatrix} 0 & 1 \\ 0 & 1 \end{bmatrix} = \begin{bmatrix} 0 & 1 \end{bmatrix} \]

Alternatively, solve $\pi_{\infty} = \pi_{\infty} \cdot P$:

\[ \begin{bmatrix} \bar{\pi} & 1-\bar{\pi} \end{bmatrix} = \begin{bmatrix} \bar{\pi} & 1-\bar{\pi} \end{bmatrix} \begin{bmatrix} \alpha & 1-\alpha \\ 0 & 1 \end{bmatrix} \]

Equation: $\bar{\pi} = \alpha \cdot \bar{\pi} + 0$

If $\alpha < 1$, then $\bar{\pi} = 0$, so $\pi_{\infty} = \begin{bmatrix} 0 & 1 \end{bmatrix}$.

No Ergodic Distribution Back

Consider:

\[ P = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix} \]

i.e., switch from whatever you had.

\[ P^2 = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix} \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix} = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} \]

\[ P^3 = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix} \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix} \]

\[ P^j = \begin{cases} \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} & \text{if } j \text{ even} \\ \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix} & \text{if } j \text{ odd} \end{cases} \]

$\lim_{j \to \infty} P^j$ doesn’t exist in general.