---
title: "Solving Equilibrium Problems with Machine Learning"
sort-order: 11
---

::: {.content-hidden}
$$
{{< include /_macros.tex >}}
$$
:::

# Problem Formulation

## Broader Program of Quantitative Macroeconomics
- One key goal is to build an artificial laboratory in order to
  - conduct counterfactual experiments
  - probe a structure to better understand an economic mechanism
  - interpret data through a structural lense
- Economic questions are often posed as conditional expectations or functions of a distribution
  - e.g., IRF of aggregate output, MPC of lower vs. upper quantile, tail parameter of wealth distribution
  - Almost all "questions" are posed as low-dimensional functions, even if they are of high-dimensional objects

## Equilibrium Markov Processes
- See Sargent

## High-Dimensional Models of Economic Dynamics
- Recursive formulations, no heuristic approximations
- Functional equation: $\ell : \mathcal{F} \times \Xdom \to \R$, scalar $\ell(\cdot) \geq 0$ without loss of generality

$$
\ell(f, X) = 0 \quad \text{for all } X \in \Xdom
$$

- $X \in \Xdom$ is the state (e.g., $\R^N$ for capital of $N$ firms, idiosyncratic states, or entire distributions)
- $f \in \mathcal{F}$ is one or more functions of the state  (e.g., capital investment, value functions)
- $\ell(\cdot, \cdot)$ stacks structural equations:
  - Bellman equations, Euler equations
  - Market clearing conditions
  - Fixed points, implicit functions

## Law of Motion and Expectations
- The $\ell(\cdot, \cdot)$ contains some calculations specific to economic dynamics:

    $$
    X' = \Gamma(X, W, \omega; f)
    $$

    - $W$ possibly high-dimensional "idiosyncratic" shocks
    - $\omega$: aggregate shocks
- Conditional expectations of future states:

    $$
    \expec{g(X') \mid X} = \expec{g(\Gamma(X, W, \omega; f)) \mid X}
    $$

- Implicit functions (fixed points, market clearing) conditional on $f$

# Global Solutions and the Curse of Dimensionality
## **Question**: How to find "global solution", $f$, where $\ell(f,X) \approx 0$ on arbitrary points $X$ in a high-dimensional $\Xdom$?{.title-slide}

## **Answer**: You can't. The curse of dimensionality is inescapable.{.title-slide}
### For that particular goal, it is far worse than you think

## Distinct Aspects of the Curse of Dimensionality
- Parameterized approximation $f_{\theta}\in\HC(\Theta)$ and $M$ points/samples $\Xtrain \subset \Xdom$
- Many algorithms (e.g., Chebyshev polynomials+collocation nodes, neural networks) solve

$$
\min_{\theta} \frac{1}{|\Xtrain|}\sum_{X \in \Xtrain} \ell(f_{\theta},X),\quad \text{nesting }\expec{q(X,W)\,|\,X} \approx \sum_{s=1}^S \alpha_s q(X,W_s)
$$

- Do the following need to increase **exponentially with $N$**? 

  1. The cost of optimization (perhaps as $M$ or $\theta$ grows)
  2. The size and flexibility of $\HC(\Theta)$ (e.g., a coefficient per node $M$ used in collocation)
  3. $M$, the number of grid points or samples in $\Xtrain$
  4. $S$, the number of quadrature points/samples for expectations at each $X\in\Xtrain$

## Optimization is Not Inherently a Challenge
- By this point in the course, we are not scared of parameterization.
  - [High-dimensional optimization](../slides/optimization.qmd) with $\partial_{\theta}\ell(f_{\theta},\cdot)$ can be independent of $N$ via [VJPs](../slides/differentiation.qmd#frechet)
  - [Sum-decomposable objectives](../slides/optimization.qmd#finite-sum-objectives) like $\frac{1}{|\Xtrain|}\sum_{X \in \Xtrain} \ell(f_{\theta},X)$ can use approximate gradients
- In fact, optimization is often easier in higher dimensions
  - More parameters = more degrees of freedom
  - Multiplicity with massive $\theta$ can even work to our advantage in some cases
- So a reasonable assumption that the $\min_{\theta} = 0$ (i.e., "interpolating" solution) possible for many different $\theta$

## Parameterization is Also Not Cursed (Especially in Practice)
- "Universal Approximation" theorems (mostly) about bias. Changing $\HC(\Theta)$ until $f_{\theta}\approx f$
  - **Question:** Can you approximate any function in $\mathcal{F}$ and how does this scale with $N$?
  - **Answer:** Mostly yes.  Sometimes just asymptotic in size of $\theta$.  Often without exponential increase in $N$ for many $\HC(\Theta)$ (e.g., kernels, neural networks)  
- Theorems are encouraging, but **vacuous**. No practical guidance on bounds or intuition
  - Often true but generic (e.g., apply to neural networks, splines and/or kernels)
  - Neural networks work far, far better in practice than these theorems suggest

## Remaining Challenges: Generalization and Integration
- The remaining concerns are generalization from $\Xtrain$ to $\Xdom$ (i.e., $M$ and $S$ scaling with $N$)
- So in principle we can solve large optimization problems and have sufficiently flexible approximations
  - Not universally true, but will be the case for most macro applications
  - Failures typically involve requirement for maintaining hard combinatorial constraints, costless off-equilibrium behavior anywhere in a large dimensional state-space
- Which leaves the two key challenges for solving these types of methods:
  1. $M$, the number of grid points or samples in $\Xtrain$
  2. $S$, the number of quadrature points/samples for expectations at each $X\in\Xtrain$

## Reminder: [High-Dimensional Geometry/Probability](high_dimensional_probability.qmd)
- Recall that in high dimensions, "most" of the volume of a hypercube is [in the corners](#all-corners)
  - Suggests that for many economic problems we cannot hope to cover $\Xdom$ uniformly with a finite $M$. Global" solutions seem impossible
- But for high-dimensional stochastic processes, most of the probability mass ["concentrates"](../slides/high_dimensional_probability.qmd#concentration-of-measure) in tiny regions of the space
  - So perhaps we do not need to worry about covering all of $\Xdom$ uniformly
- Will show this "concentration" logic hels both the function approximation of $f_{\theta}$ and calculating $\expec{\cdot}$
  - Often have a "blessing of dimensionality" instead
  - Caveat: sensitive to the underlying distributions[misspecification](../slides/high_dimensional_probability.qmd#concentration-double-edged)

# Statistical Learning Formulation

## From Uniform to Statistical
- Old (unattainable) goal: $\ell(f, X) \approx 0$ for all $X \in \Xdom$

$$
\arg\min_{f \in \mathcal{F}} \sup_{X \in \Xdom} \ell(f, X)
$$

- Instead, solve

    $$
    \fpop \equiv \arg\min_{f \in \mathcal{F}} \expec[X \sim \popdist(\cdot;\fpop)]{\ell(f, X)}
    $$

    - $\popdist$ = "population distribution" - key departure from traditional methods
    - We assume an interpolating solution exists: $\ell(\fpop, X) = 0$ for all $X \in \Xdom$
    - $f(X)$ with $X \sim \popdist$ might be "concentrated"

## Empirical Risk Minimization (ERM) {#sec-erm}
- Classic approach is to
  - Choose parameterized function $f_\theta \in \HC(\Theta)$ (e.g., neural networks, polynomials)
  - Take points $\Xtrain \iidsim \popdist$, minimize empirical risk: [[Error Decomposition]{.button}](#sec-err-decomp)

$$
\thetaerm \equiv \arg\min_{\theta \in \Theta} \frac{1}{|\Xtrain|} \sum_{X \in \Xtrain} \ell(f_\theta, X)
$$

- This looks like the old optimization problem - what's different?
  - **Crucially**: $\Xtrain \iidsim \popdist$ makes this approximate risk minimization
  - Nests traditional methods: Chebyshev nodes on hypercube $\approx$ uniform $\popdist$
- While a common objective, is this correctly specified here?

## Dependence of the Population Distribution on the Solution
- The $\popdist(\cdot; \fpop)$ often depends on the solution $\fpop$ itself
  - E.g., in macro models, $\popdist$ might be generated from trajectories of an initial condition
  - In some cases could use a stationary distribution (but be careful!)
- Challenge is that you need to take the $\grad_\theta$ of the samples from $\popdist(\cdot; f_\theta)$
  - See [previous lecture](../slides/optimization.qmd#score-function-estimation-(reinforce)) on REINFORCE

## A Means to an End?
- We don't actually care about the
  $$
  \thetaerm \equiv \arg\min_{\theta \in \Theta} \frac{1}{|\Xtrain|} \sum_{X \in \Xtrain} \ell(f_\theta, X)
  $$

- Alternative objectives generate better representations, adjust for bias, etc. 

## Decomposing the Errors Reminder from [this lecture](../slides/intro_to_ml.qmd)
- Two sources of approximation error (bias-variance tradeoff):
  1. **Approximation error $\apperror$** (bias): $f_{\thetaerm} \in \HC(\Theta)$ may not represent $\fpop$
  2. **Generalization error $\generror$** (variance): Finite sample $\Xtrain \iidsim \popdist$

- Decomposition following @bottou2007tradeoffs, @probml1:

$$
\small
\expec[\Xtrain \iidsim \popdist]{\min_\theta \hat{R}(\theta, \Xtrain) - \min_f R(f, \popdist)} = \underbrace{R(\ferm, \popdist) - R(\fpop, \popdist)}_{\apperror(\ferm)} + \underbrace{\expec[\Xtrain]{\hat{R}(\thetaerm, \Xtrain) - R(\ferm, \popdist)}}_{\generror(\ferm)}
$$

- Modern ML shows we can often reduce both (double-descent, inductive bias)
- With flexible $\HC(\Theta)$ like neural networks: often $\apperror(\ferm) \approx 0$
- Then error is $\generror(\ferm) \approx \hat{R}(\thetaerm, \Xtrain) - \hat{R}(\thetaerm, \Xtest)$


## Sensitivity to $\popdist$
- Misspecification of $\popdist$ usually innocuous for small $N$ if $|\Xtrain|$ is large
- With large $N$, small changes in $\popdist$ can have dramatic effects
  - Unless $|\Xtrain| \propto \exp(N)$
- Recall from [previous lecture](high_dimensional_probability.qmd):
  - Small mean shifts cause distance $\epsilon\sqrt{N}$ between distributions
  - Small variance differences cause complete separation
- Key insight: in high dimensions, **matching $\popdist$ matters more than $|\Xtrain|$**

## Key Tradeoffs
- No "global" solutions $\implies$ cannot conduct arbitrary counterfactuals
  - But may cover all economically relevant states for particular questions
- Sensitivity "out of distribution": all bets are off if $\popdist$ changes
  - Key is matching $\popdist$ for counterfactuals of interest, not size of $\Xtrain$
- Finding appropriate $\Xtrain$ is challenging due to feedback
  - See @taming for approaches

# Neural Networks and Representations

## NN Often Needed in Practice, Not in Theory
- Neural networks are a very flexible class $\HC(\Theta)$
  - Encapsulating polynomials, splines, etc.
  - Many parameters, even $|\Theta| \gg |\Xtrain|$
- Continuum of $\theta$ with $\ell(f_\theta, X) \approx 0$ for all $X \in \Xtrain$
- Regularization/inductive bias select particular solutions
- "Universal approximation" theorems are encouraging but **vacuous**
  - No practical guidance on bounds

## Mental Model: Representation Learning
- Think of neural networks as nested approximations:

$$
f_\theta(X) \equiv g_\theta(h_\theta(X))
$$

- $h_\theta : \Xdom \to \R^L$ maps state to latent **representation**
- $g_\theta : \R^L \to \R$ maps representation to output/policy
- Both are very flexible (many parameters, nonlinear compositions)

## Deep Learning Can "Find the State"
- Nesting in $h_\theta(\cdot)$ can learn ideal transformations of state space
- $h_\theta$ may map $X$ to a place where it is less lonely:
  - $L$ might be lower-dimensional
  - $\{h_\theta(X) : X \in \Xtrain\}$ might be close to low-dimensional manifold
- Advantages over manually crafting representations ("finding the state is an art"):
  - May be difficult to find right simplification in complicated state spaces
  - Simplifications may only hold approximately
  - Ideal representation may depend on $\popdist$

## Robustness of Good Representations
- Unless $|\Xtrain| \propto \exp(N)$, even $\text{Hull}(\Xtrain)$ has vanishing volume
- But what really matters is variation in the **latent space** induced by $h_\theta(X)$
- If representation is sufficient for task, small $\Xtrain$ may provide sufficient variation

![Variation in latent space matters more than $|\Xtrain|$](hull.png){width="60%"}

- e.g., $h(X) = \frac{1}{2}(X_1 + X_2)$ latent, $\Xtrain \iidsim \mathcal{U}[0,1]^2$
- Blue region: points in $\Xdom$ that map inside hull of data in representation space

# Permutation Invariance

## Motivation
- **"To a macroeconomist, a firm is a nameless point in a histogram"**
- Payoff functions often symmetric: market clearing depends on supply/demand, not firm names
- Permutations are one of many invariances - see @bloem2019probabilistic

## Definition: Permutation Invariance and Equivariance
Let $\mathcal{S}_N$ be the set of all $N!$ permutation matrices of size $N \times N$.

::: {.callout-note icon=false}
### Definition: Permutation Invariance and Equivariance

- $f : \R^N \to \R$ is **permutation invariant** if, for all $\pi \in \mathcal{S}_N$:

$$
f(\pi X) = f(X)
$$

- $f : \R^N \to \R^N$ is **permutation equivariant** if, for all $\pi \in \mathcal{S}_N$:

$$
f(\pi X) = \pi f(X)
$$
:::

- Example: price function $p(X) \equiv 1 - \frac{1}{N}\sum_{n=1}^N |X_n|^\nu$ is permutation invariant

## Representation Theorem

::: {.callout-warning icon=false}
### Proposition: Permutation Invariant Representation (@zaheer2017deep, @wagstaff2019limitations)

Let $f : \R^{N+1} \to \R$ be continuous and permutation-invariant under $\mathcal{S}_N$. Then there exist $L \leq N$ and continuous functions $\rho : \R^{L+1} \to \R$ and $\phi : \R \to \R^L$ such that:

$$
f(x, X) = \rho\left(x, \frac{1}{N}\sum_{n=1}^N \phi(X_n)\right)
$$
:::

- Any permutation invariant function can be represented as:
  1. Apply $\phi$ to each element → map to $\R^L$
  2. Aggregate (mean, sum, max) pointwise in $L$ to get latent state
  3. Function $\rho$ maps latent state and other inputs to output

## Dimension Reduction?
- This is a pointwise representation theorem motivating choice of $h_\theta$ and $g_\theta$
- Note: $\bar{\phi}(X) \equiv \frac{1}{N}\sum_{x \in X} \phi(x) \in \R^L$ maps to $h_\theta$ in our mental model
  - Interpretation: calculating $L$ statistics about distribution **within** $X$
- $L \leq N$ is **worst-case**. Some $f$ and $\rho$ need $N$ summary statistics
- When $L$ unknown, choose some $\hat{L}$
  - What if $\hat{L} \neq L$? Does $\hat{L}$ need to grow with $N$?
- Answers won't come from worst-case theorem. Key is to consider $X \sim \popdist$

# Concentration Applied to Equilibrium

## Recap: Concentration of Measure
*"A random variable that depends in a Lipschitz way on many independent variables (but not too much on any of them) is essentially constant."* - @ledoux2001concentration

- See [previous lecture](high_dimensional_probability.qmd) for foundations
- **Non-asymptotic**: bounds hold for finite $N$
- Need to formalize "not too much on any of them"
  - Natural connection to permutation invariance: if $X$ enter symmetrically, hard for single index to dominate

## Expected Gradient Bounds

::: {.callout-note icon=false}
### Definition: Expected Gradient Bounded in $N$

Let $f : \R^N \to \R$ and $z \sim \mathcal{N}(\mathbf{0}_N, \mathbf{I}_N)$. Then $f$ has expected gradient bounded in $N$ if there exists $C$ independent of $N$ such that:

$$
\expec{\|\nabla f(z)\|^2} \leq \frac{C}{N}
$$
:::

::: {.callout-warning icon=false}
### Proposition: Expected Gradient Bounds for Symmetric Functions

Let $f : \R^N \to \R$ be continuously differentiable and permutation-invariant. For $z \sim \mathcal{N}(\mathbf{0}_N, \mathbf{I}_N)$, $f$ has expected gradients bounded in $N$ if:

$$
\expec{\left(\frac{\partial f(z)}{\partial z_1}\right)^2} \leq \frac{C}{N^2}
$$
:::

- Especially easy to achieve with permutation invariance
- May need to rescale (e.g., average output per firm instead of aggregate)

## Concentration When Gradients Bounded

::: {.callout-warning icon=false}
### Proposition: Concentration When Expected Gradients Bounded

For $z \sim \mathcal{N}(\mathbf{0}_N, \Sigma)$ with spectral radius $\spectralradius{\Sigma}$ independent of $N$. If $f : \R^N \to \R$ has expected gradient bounded in $N$ and $z^1$ is a single draw:

$$
\prob{|f(z^1) - \expec{f(z)}| \geq \epsilon} \leq \frac{\spectralradius{\Sigma} C}{\epsilon^2} \frac{1}{N}
$$
:::

- Can calculate expected value with a **single draw** $z^1$ for large enough $N$!
  - But you cannot use mean/median/mode - it has to be sampled
- Many variations for properties of $f$ and $z$ - see @boucheron2013concentration
- In many cases (Lipschitz $f$, bounded $z$), convergence is exponential

## Blessing of Dimensionality for Expectations
- Inside Euler equation: need price forecast $\expec{p(X') \mid X}$
- $p(X')$ is permutation invariant → usually has bounds on curvature
- $X'$ driven by aggregate $\omega$ and idiosyncratic $W \in \R^N$:

$$
X' = \Gamma(X, W, \omega)
$$

- Apply concentration for Monte Carlo with **single draw** $W^1$:

$$
\expec{p(\Gamma(X, W, \omega)) \mid X, \omega} \approx p(\Gamma(X, W^1, \omega))
$$

- Connection to heterogeneous agent macro:
  - We condition on aggregate shock and intuitively apply LLN
  - Our **non-asymptotic** results show this works for finite $N$ too

# Concentration and Generalization

## Our Last Concerns
- Recall statistical learning with $\Xtrain \iidsim \popdist$:

$$
\fpop \equiv \arg\min_{f \in \mathcal{F}} \expec[X \sim \popdist]{\ell(f, X)}, \quad
\thetaerm \equiv \arg\min_{\theta} \frac{1}{|\Xtrain|} \sum_{X \in \Xtrain} \ell(f_\theta, X)
$$

- Does permutation invariant representation introduce bias?
- Does generalization require $|\Xtrain|$ to grow exponentially with $N$?
- Risk minimization involves expectations - can we use concentration here too?

## Conditionally IID with Finite Dimension

::: {.callout-note icon=false}
### Definition: Conditionally IID with Finite Dimension

Random variables $X_1, \ldots, X_N$ admit a **conditionally IID representation of finite dimension** if there exists latent $\nu \in \R^d$ with $d < \infty$ such that $X_1, \ldots, X_N \mid \nu$ are IID.

Denote distribution of $\nu$ as $\popdist_\nu$ and conditional distribution as $X \iidsim \popdist_X(\nu)$.
:::

- Relatively weak: By de Finetti's theorem, any exchangeable sequence admits this (though not necessarily finite $d$)
- Interpretation: $\nu$ is the latent aggregate state in macro models
  - Usually low-dimensional by construction

## Intuition on Concentration for Function Approximation
1. For exchangeable $X \sim \popdist$, factor: $X \iidsim \popdist_X(\nu)$ and $\nu \sim \popdist_\nu$
2. Since $X \mid \nu$ are IID, $f(X)$ fulfills concentration conditional on $\nu$
3. Neural networks could learn $\hat{\nu}_\theta(X) \approx \nu \in \R^d$
4. As $N$ increases, dimension $d$ controls complexity, **not $N$**
5. **Blessing of dimensionality**: larger $N$ better estimates underlying $\nu$

- With permutation invariant representation, as long as $\hat{L} \geq d$, should be possible to approximate $f(X)$

## Target Function Approximation
- Consider risk minimization to approximate permutation invariant target $\fpop$:

$$
\min_{f \in \mathcal{F}} \expec[X \sim \popdist]{(f(X) - \fpop(X))^2}
$$

- Use law of iterated expectations to factor:

$$
\min_{f \in \mathcal{F}} \expec[\nu \sim \popdist_\nu]{\expec[X \iidsim \popdist_X(\nu)]{(f(X) - \fpop(X))^2 \mid \nu}}
$$

- Concentration: loosely, $\fpop(X) \xrightarrow{P \mid \nu} \rho^*(\nu)$ as $N \to \infty$
- Our permutation invariant representation has this form with $h(X) = \frac{1}{N}\sum \phi(x)$

## Crude Conjecture: Blessing of Dimensionality

::: {.callout-warning icon=false}
### Crude Conjecture: Blessing of Dimensionality for Function Approximation

Maintain assumptions and add conditions for identifiability of $\nu$ and regularity. For any fixed $|\Xtrain|$, as $N \to \infty$ with sufficiently flexible $\rho_\theta$ and $\phi_\theta$ with latent dimension $\hat{L}$:

- As long as $\hat{L} \geq d$, this will be bias-free in the limit
- Since $\rho_\theta$ and $\phi_\theta$ are independent of $N$, complexity controlled by $d$
:::

- Statement/proof in progress - trying to get non-asymptotic bounds
- While low-dimensional $\hat{L} = d$ is sufficient, $\nu$ is not necessarily integer moments
- Interpret NNs as finding "adaptive basis functions" $\phi_\theta(\cdot)$ - see @wilson2025deeplearningmysteriousdifferent

## Framework Conclusion
- Statistical learning reformulation enables tractable solutions
- Permutation invariance + exchangeability → blessing of dimensionality
- Complexity controlled by latent dimension $d$, not state dimension $N$
- Deep learning works very well in practice, not strictly necessary in theory

# Application Examples

## Overview
- **Parametric example**: Generalized mean - validates theory
- **Economic example**: Lucas-Prescott (1971) with $N$ firms

# Parametric Example: Generalized Mean

## Setup
- Target function capturing common patterns in heterogeneous agent macro:

$$
\fpop(X) \equiv \left(\frac{1}{N}\sum_{n=1}^N |X_n|^\zeta\right)^{1/\zeta}
$$

- Population distribution $\popdist$ constructed as:
  - Aggregate state: $a \sim \mathcal{U}(a_{\min}, a_{\max})$
  - Then: $X \iidsim \mathcal{N}(a, \sigma^2)$
- Latent dimension $d = 1$ (just the aggregate state $a$)

## Deep Learning Approximation
- **Goal**: Demonstrate blessing of dimensionality and $N \gg \hat{L} \gtrsim d$ sufficiency
- Solve ERM: $\min_\theta \frac{1}{M}\sum_{X \in \Xtrain}(f_\theta(X) - \fpop(X))^2$
- Permutation invariant architecture:

$$
f_\theta(X) \equiv \rho_\theta\left(\frac{1}{N}\sum_{n=1}^N \phi_\theta(X_n)\right)
$$

- $\phi_\theta : \R \to \R^{\hat{L}}$ and $\rho_\theta : \R^{\hat{L}} \to \R$
- Use $M = 10$ points (!) for various $N$ and $\hat{L}$

## Results: Blessing of Dimensionality

| $N$ | $\hat{L}=1$ | $\hat{L}=2$ | $\hat{L}=3$ | $\hat{L}=4$ | $\hat{L}=5$ | $\hat{L}=20$ | NONE |
|----:|------------:|------------:|------------:|------------:|------------:|-------------:|-----:|
| 2   | 1.85%       | 1.19%       | 1.91%       | 1.27%       | 1.70%       | 1.75%        | 1.36%|
| 4   | 1.04%       | 1.70%       | 1.76%       | 0.76%       | 0.97%       | 0.83%        | 1.65%|
| 8   | 0.57%       | 0.57%       | 0.71%       | 0.64%       | 0.60%       | 0.56%        | 3.68%|
| 16  | 0.50%       | 0.49%       | 0.51%       | 0.45%       | 0.42%       | 0.48%        | 4.91%|
| 32  | 0.32%       | 0.35%       | 0.32%       | 0.28%       | 0.37%       | 0.51%        | 5.69%|
| 64  | 0.32%       | 0.27%       | 0.27%       | 0.33%       | 0.28%       | 0.29%        | 5.77%|
| 128 | 0.32%       | 0.22%       | 0.24%       | 0.28%       | 0.23%       | 0.22%        | 5.77%|

Mean absolute relative error on test set. NONE = generic NN without invariance.

## Confirms Intuition
- Fit with tens of thousands of parameters and only $M = 10$ draws in $\R^N$
- Since $d = 1$, $\hat{L} \geq 1$ is sufficient. Small benefits for increasing $\hat{L}$
- For fixed $\hat{L}$, as $N$ increases: **blessing of dimensionality**!
- Generic NN without imposing invariance does poorly as $N$ increases
  - Shows benefits of using economic insights

# Lucas-Prescott (1971) Example

## Model Setup (@LucasPrescott1971)
- $N$ price-taking firms, capital $X \in \R^N$
- Idiosyncratic shock $W_n \sim \mathcal{N}(0,1)$, aggregate shock $\omega \sim \mathcal{N}(0,1)$
- Law of motion with investment $u_n$ and adjustment cost $\frac{\gamma}{2}u_n^2$:

$$
X'_n = (1-\delta)X_n + u_n + \sigma W_n + \eta \omega
$$

- Inverse demand for price-taking firms:

$$
p(X) \equiv 1 - \frac{1}{N}\sum_{n=1}^N |X_n|^\nu, \quad \nu \geq 1
$$

- If $\nu = 1$: linear dynamics, quadratic payoff → exact LQ solution
- If $\nu \neq 1$: nonlinear, no closed-form baseline

## Recursive Equilibrium
- Decision problem taking others' policy $\hat{u}(\cdot, X)$ as given
- Take FOCs, set $\hat{u} = u$ for Rational Expectations Equilibrium
- **Euler equation**:

$$
\gamma u(x, X) = \beta \expec{p(X') + \gamma(1-\delta)u(x, X') \mid x, X}
$$

- **Residuals**:

$$
\ell(u, X) \equiv \left(\gamma u(X) - \beta \expec{p(X') + \gamma(1-\delta)u(X') \mid X}\right)^2
$$

## Applying Concentration to Expectations
- Single draw $W^1 \iidsim \mathcal{N}(0,1)$ for idiosyncratic shocks
- Gaussian quadrature with weights/nodes $\alpha, \omega \in \R^S$ for aggregate shock:

$$
\ell(u_\theta, X) \equiv \left(\gamma u_\theta(X) - \beta \sum_{s=1}^S \alpha_s\left[p(X'(W^1, \omega_s)) + \gamma(1-\delta)u_\theta(X'(W^1, \omega_s))\right]\right)^2
$$

- Approximate with permutation invariant architecture:

$$
u_\theta(X) = \rho_\theta\left(\frac{1}{N}\sum_{n=1}^N \phi_\theta(X_n)\right)
$$

## ERM and Population Distribution
- Challenge: $\popdist$ is not known a priori
- Simple iterative approach:
  1. Pick initial $\theta^0$
  2. Generate $\Xtrain(\theta^i)$ by simulating with $u_{\theta^i}$
  3. Solve for $\theta^{i+1}$, repeat until convergence

$$
\theta^{i+1} \equiv \arg\min_{\theta} \frac{1}{|\Xtrain(\theta^i)|} \sum_{X \in \Xtrain(\theta^i)} \ell(u_\theta, X)
$$

- Benefit: eventual $\Xtrain$ and $\Xtest$ come from same distribution
- Caution: heuristic, no convergence theory

## Architecture Comparison
- $\Xtrain$: 16 simulated trajectories to $T = 63$ from $X_0$
- Three architectures for $\phi$:
  1. **$\phi$(Identity)**: $\phi(x) = x$, no parameters (just mean)
  2. **$\phi$(Moments)**: first 4 moments, no parameters
  3. **$\phi_\theta$(Deep Sets)**: flexible NN with $\hat{L} = 4$
- $\rho$ is a flexible NN (4 layers) in all cases
- Solves in approximately 10-30 seconds on laptop, roughly independent of $N$

## Linear Case: $\nu = 1$
- Known closed-form baseline: $u(X) = H_0 + \frac{H_1}{N}\sum_{n=1}^N X_n$
- $\hat{L} = 1$ is sufficient for all $\Xdom$ (stronger than needed)

![Comparison between LQ-regulator solution and deep learning architectures for $\nu = 1$, $N = 128$](linear-baseline-theory-vs-predicted.png){width="80%"}

## Linear Case: Policy Errors

![Policy errors for $\nu = 1$, $N = 128$. Dark blue: median. Shaded: 10th-90th and 25th-75th percentiles.](identity_moments_deep_sets_linear_relative.png){width="85%"}

## Nonlinear Case: $\nu \neq 1$
- Same code and algorithms - architecture did not have special case for $\nu$
- Check accuracy via $\ell(u_{\thetaerm}, \cdot)$ and transversality condition
- Deep Sets outperforms moments-based architecture

![Optimal policy along equilibrium paths for $\nu = [1.0, 1.3, 1.5]$, $N = 128$](deep-sets-nonlinear-var-nu.png){width="80%"}

## Nonlinear Case: Euler Residuals

![Euler residuals for $\nu = 1.5$, $N = 128$ comparing Moments and Deep Sets](moments-deep-sets-nonlinear-residual.png){width="85%"}

## Key Insights
- Solves $N = 128$ in seconds
- Performance roughly independent of $N$
- Deep Sets most robust across initial conditions (97% success vs 48-59%)
- Even if integer moments sufficient, learning $\phi_\theta$ may be innocuous


# Conclusion

## Summary
- Solve heterogeneous agent macro models as **statistical learning problems**
  - While not global in theory, may be global in practice
- High dimensions can be a **blessing**, not a curse
- Underlying complexity controlled by size of **latent aggregate state**, not number of agents
- Key is concentration of measure when:
  1. Computing conditional expectations over idiosyncratic shocks
  2. Approximating functions of exchangeable state variables
- **Deep learning** isn't necessary in theory, but works very well in practice
- Symmetry and exchangeability are powerful tools to exploit problem structure


# Appendices

## ML Papers in Economics
- @fernandez2019financial, @MALIAR202176, @AzinovicEtAl, @han2022deepham, @payne2025deep
- Many more in progress

## References
::: {#refs}
:::
