Error Propagation
Small repeated errors don't just add up — they converge, wander, or explode. Why error propagation will be a defining principle of the post-AI world.
Error propagation is going to be a defining principle in the post-AI world.
In statistics, there’s a term called error propagation, or Propagation of uncertainty. When trying to predict the future (e.g., in time series forecasting), you might have a series of daily sales numbers - 10, 20, 30, 40, 50, etc. A simple prediction here would be 60 - if every prior value was increasing by 10, perhaps that trend will continue and the next word should be 60.
What would happen if you predicted 59 instead? And you only predicted by +9 thereafter? Your next predictions would be 59, 68, 77, 86, 95, etc. After 10 predictions, you’d be 10 whole points off. After 100 predictions, you’d be 100 whole points off — but only 9.5% wrong. Here’s the percentage error at each step:
| Prediction step | Truth | Prediction | Absolute error | Percentage error |
|---|---|---|---|---|
| 1 | 60 | 59 | 1 | 1.67% |
| 2 | 70 | 68 | 2 | 2.86% |
| 5 | 100 | 95 | 5 | 5.00% |
| 10 | 150 | 140 | 10 | 6.67% |
| 25 | 300 | 275 | 25 | 8.33% |
| 50 | 550 | 500 | 50 | 9.09% |
| 100 | 1050 | 950 | 100 | 9.52% |
And here’s truth vs. prediction as a chart:
In this example, error doesn’t compound though. It asymptotically hits a plateau where every subsequent prediction is only 10% wrong.
At step k, truth = 50 + 10k and prediction = 50 + 9k, so:
% error = k / (50 + 10k) = 1 / (10 + 50/k) → 10% as k → ∞Our error will be known if our bias is known and fixed. However, once you introduce random noise, you get a lot more error, and in unpredictable ways.
Three error regimes, side by side:
| Your scenario: P_n = 9n | True autoregressive error: e_t = ρe_{t−1} + ε_t | Multiplicative compounding: P_n = 1.08·P_{n−1} | |
|---|---|---|---|
| Model structure | P_n = 9·n | y_t = βX_t + e_t, where e_t is AR(1) | P_n = rate × P_{n−1} |
| Error driver | Direct slope mismatch (Δ = 1) | Memory of previous residual e_{t−1} | Percentage growth mismatch |
| Absolute error (e_n) | n (linear growth) | Fluctuates or drifts randomly | Exponential explosion |
| Percentage error | Fixed at 10% | Fluctuates around zero / constant mean | Explodes toward 100% |
A fixed slope mismatch is the kindest failure — bounded and predictable. Autoregressive noise inherits memory, so errors wander. A multiplicative mismatch compounds exponentially.
As we use LLMs, which have a known bias, there will be a trend towards higher entropy and higher chaos. This is why goals are powerful: it gives the LLM a destination. It can self-correct in loops.
Models need truth to anchor against. LLMs need a definition of truth. The definition of truth might be subjective (how you want your website to look). The LLM’s ability to reduce error propagation and achieve truth lies in its ability to bridge the gap. This fundamentally is an issue of projection - how the LLM interprets your tokenized truth (“put blue button on the right”) and how that maps to its expected value of “a blue button on the right”.
We will fundamentally deal with some definition of error. The SWE Bench at 53% means there’s error. There always will be. The question is: how much are we willing to deal with that?
In some cases, error is fine and asymptotically reduces to something negligible in our eyes. In other systems, the error might be…10% of death.
“Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”
— Evan Hubinger (@EvanHub), Sep 8, 2026
Other Ideas