Applications · Machine learning
Chaos in reservoir computing & AI
Chaotic recurrent networks make surprisingly good substrates for short-term prediction of nonlinear time series. The idea: a fixed random recurrent network projects an input into a high-dimensional reservoir, and only a linear readout is trained.
Echo state network on Mackey-Glass
state update: s_{t+1} = (1 − α) s_t + α · tanh(W_in u_t + W s_t)
read-out: ŷ_t = b + W_out · s_t (ridge regression)
prediction: run network closed-loop, feeding back ŷ_t as next input.
ρ near 1 puts the reservoir near the "edge of chaos": good for nonlinear
time-series prediction. Standard ESN forecasts ~50 lookahead steps of MG
extremely well; further out, chaotic divergence saturates the error.Rule 30 : Wolfram class III (chaotic)
Wolfram's canonical chaos: aperiodic, used in Mathematica's RandomInteger.
Each cell's next state depends on its left/self/right triple. 8 possible triples give 8 output bits, encoding the rule as a number 0-255. Wolfram's four-class taxonomy: I homogeneous, II periodic, III chaotic, IV complex (edge of chaos).
Echo state networks
Jaeger (2001): random recurrent networks tuned to the 'edge of chaos' (spectral radius near 1) match LSTM performance on many time-series tasks with orders-of-magnitude less training.
Liquid state machines
Maass (2002): the same idea with spiking neurons; biological plausibility, real-time computation on continuous-time inputs.
Edge-of-chaos computation
Langton (1990) and Wolfram class IV: maximum computational power tends to live at the boundary between order and chaos. Trainable wave-like dynamics there outperform fully ordered or fully chaotic networks.
Chaotic optimisation
Chaotic particle-swarm and genetic algorithms use logistic-map or PWLCM-driven perturbations to escape local optima. Effective in low dimensions; ad-hoc otherwise.
Back to all applications.