Chaos Lab

Applications · Machine learning

Chaos in reservoir computing & AI

Chaotic recurrent networks make surprisingly good substrates for short-term prediction of nonlinear time series. The idea: a fixed random recurrent network projects an input into a high-dimensional reservoir, and only a linear readout is trained.

Echo state network on Mackey-Glass

state update:  s_{t+1} = (1 − α) s_t + α · tanh(W_in u_t + W s_t)
read-out:       ŷ_t = b + W_out · s_t   (ridge regression)
prediction:     run network closed-loop, feeding back ŷ_t as next input.

ρ near 1 puts the reservoir near the "edge of chaos" — good for nonlinear
time-series prediction. Standard ESN forecasts ~50 lookahead steps of MG
extremely well; further out, chaotic divergence saturates the error.

Rule 30 — Wolfram class III (chaotic)

Wolfram's canonical chaos: aperiodic, used in Mathematica's RandomInteger.

Each cell's next state depends on its left/self/right triple. 8 possible triples give 8 output bits, encoding the rule as a number 0-255. Wolfram's four-class taxonomy: I homogeneous, II periodic, III chaotic, IV complex (edge of chaos).

Echo state networks

Jaeger (2001): random recurrent networks tuned to the 'edge of chaos' (spectral radius near 1) match LSTM performance on many time-series tasks with orders-of-magnitude less training.

Liquid state machines

Maass (2002): the same idea with spiking neurons; biological plausibility, real-time computation on continuous-time inputs.

Edge-of-chaos computation

Langton (1990) and Wolfram class IV: maximum computational power tends to live at the boundary between order and chaos. Trainable wave-like dynamics there outperform fully ordered or fully chaotic networks.

Chaotic optimisation

Chaotic particle-swarm and genetic algorithms use logistic-map or PWLCM-driven perturbations to escape local optima. Effective in low dimensions; ad-hoc otherwise.

Back to all applications.

Quick quiz

Test yourself on ai

8 multiple-choice questions. Pick an answer for each, then submit to see explanations.

  1. Q1.Reservoir computing trains:

  2. Q2.Jaeger's echo-state network paper appeared in:

  3. Q3.Maass's liquid-state machine uses:

  4. Q4.Edge-of-chaos hypothesis (Langton 1990) says:

  5. Q5.Pathak et al. (2018) used reservoir computing to:

  6. Q6.Chaotic optimisation algorithms add:

  7. Q7.Spiking neurons exhibit chaos because:

  8. Q8.AI reservoir computing's appeal in production:

0 of 8 answered