Cross Validated
2026-11-03 05:56 UTC
By Mr.BrainStorm
AI-113-20261103-social-media-072e0e21
Preventing trivial constant solutions in multi-step Neural ODEs for time-series forecasting
Greetings, colleagues. I am training a Neural ODE to forecast the non-linear dynamics of a physical system. To avoid error accumulation, I use a multi-step rollout loss (e.g., integrating the ODE over a horizon of 24 steps and computing the MSE over the entire trajectory). However, the model consistently collapses to a trivial solution: it predicts a near-zero derivative (a constant or a very slow linear trend), essentially flattening the curve. While this minimizes the average MSE over the long horizon, it completely fails to capture the true high-frequency dynamics (see the attached image). What regularization methods or architectural constraints—such as Lipschitz continuity, spectral normalization, or physics-informed penalties—are best suited to ensure that a Neural ODE learns the true nonlinear dynamics, rather than collapsing into a constant function when trained over long horizons?
Greetings, colleagues. I am training a Neural ODE to forecast the non-linear dynamics of a physical system. To avoid error accumulation, I use a multi-step rollout loss (e.g., integrating the ODE over a horizon of 24 steps and computing the MSE over the entire trajectory). However, the model consistently collapses to a trivial solution: it predicts a near-zero derivative (a constant or a very slow linear trend), essentially flattening the curve. While this minimizes the average MSE over the long horizon, it completely fails to capture the true high-frequency dynamics (see the attached image). What regularization methods or architectural constraints—such as Lipschitz continuity, spectral normalization, or physics-informed penalties—are best suited to ensure that a Neural ODE learns the true nonlinear dynamics, rather than collapsing into a constant function when trained over long horizons?
Full article content could not be extracted automatically. Read the original below.
Source:
Cross Validated
· stats.stackexchange.com