maaz.code logo
Simple view

Complete

SINDy vs. ISINDy on the Lorenz System

Recovering the Lorenz equations from data with sparse regression, and making it robust to noise.

  • Coefficient error at 5% noise: SINDy98.5%
  • Coefficient error at 5% noise: ISINDy3.3%
  • Python
  • NumPy
  • SciPy
  • Scikit-Learn
  • Matplotlib

Equation discovery with SINDy

SINDy (Sparse Identification of Non-linear Dynamics) discovers governing equations from data. Given state measurements X\mathbf{X}, a library of candidate functions Θ(X)\Theta(\mathbf{X}) and a coefficient matrix Ξ\Xi, it looks for the sparsest Ξ\Xi such that

X˙=Θ(X) Ξ\dot{\mathbf{X}} = \Theta(\mathbf{X})\,\Xi

minimising ∥X˙−Θ(X)Ξ∥22+λ∥Ξ∥1\lVert \dot{\mathbf{X}} - \Theta(\mathbf{X})\Xi \rVert_2^2 + \lambda \lVert \Xi \rVert_1 with the STLSQ (Sequential Thresholded Least Squares) algorithm. The test case is the Lorenz system.

from portfolio.demos import lorenz
lorenz.run(sigma=10, rho=28, beta=8/3, integrator="rk4") # live, in your browser

Output will appear here. Press ▶ to run the Lorenz attractor.

The problem: noise

Standard SINDy needs the derivative X˙\dot{\mathbf{X}}, and numerical differentiation amplifies high-frequency noise, so on noisy data it often fails to find the true dynamics.

The fix: Integral SINDy (ISINDy)

ISINDy integrates the equations in time instead:

X(t)−X(t0)=Θint(X) Ξ\mathbf{X}(t) - \mathbf{X}(t_0) = \Theta_{\text{int}}(\mathbf{X})\,\Xi

Integration acts as a low-pass temporal filter and removes the need to differentiate noisy data, which makes the regression far more robust.

Part of my research internship with Prof. Saurav Sengupta, IIT (ISM) Dhanbad.

True vs. discovered

Non-zero coefficients: true system 7, SINDy 60, ISINDy 8. The sparser the fit, the closer it is to the real physics.

ẋ = …
termtrueSINDyISINDy
1·1.456·
x−σ15.784−9.960
yσ−1.7909.978
z·−0.825·
x²·−0.090·
xy·−0.180·
xz·−1.283·
y²·0.125·
yz·0.388·
z²·0.077·
x³·0.250·
x²y·−0.395·
x²z·0.008·
xy²·0.245·
xz²·0.010·
y³·−0.054·
y²z·−0.002·
yz²·0.002·
z³·−0.002·
x³z·−0.004·
x²yz·0.007·
xy²z·−0.005·
y³z·0.001·
ẏ = …
termtrueSINDyISINDy
1·1.569·
xρ19.96427.654
y−12.851−0.849
z·−0.707·
x²·0.027·
xy·−0.350·
xz−1−0.579−0.993
y²·0.184·
yz·−0.174·
z²·0.065·
x³·−0.001·
x²z·0.005·
xyz·0.005·
xz²·−0.005·
y²z·−0.004·
yz²·0.002·
z³·−0.002·
ż = …
termtrueSINDyISINDy
1·5.746−0.951
x·−5.549·
y·2.497·
z−β−4.185−2.626
x²·−0.360·
xy10.9980.999
xz·0.412·
y²·0.099·
yz·−0.150·
z²·0.105·
x³·0.020·
x²y·−0.007·
x²z·0.012·
xy²·−0.006·
xyz·−0.002·
xz²·−0.008·
y³·0.002·
y²z·−0.002·
yz²·0.002·
z³·−0.002·

Library: all 35 monomials up to degree 4 (scikit-learn PolynomialFeatures). Fitted with STLSQ on 100,000 samples with 5% Gaussian noise (SINDy threshold 0.001, ISINDy 0.06). Printed to 3 decimals, so "·" can hide terms smaller than 0.0005.