Deep Learning Forecasting
Deep learning forecasting uses neural networks to learn temporal representations, covariate interactions, and multi-horizon outputs. Neural models can be useful for large collections of related series, long histories, multiple covariates, nonlinear temporal relationships, and shared representations across entities.
They are not automatically superior. They often require more compute, tuning, regularization, and diagnostic effort than statistical or tabular machine learning models. On small datasets, simple baselines or gradient-boosted trees may perform better.
Neural forecasting setup
Many neural forecasters consume a context window of past target values and covariates, then emit forecasts for one or more horizons. Static metadata can be embedded or encoded. Future-known covariates such as holidays or planned promotions can be passed for the forecast horizon.
The core advantage is representation sharing. A model can learn that similar seasonal shapes, launch curves, or event responses appear across many series, even when each individual series is short.
| Architecture | Core mechanism | Best when |
|---|---|---|
| LSTM | recurrent hidden state over the sequence | moderate horizons, ordered dependence matters |
| Temporal convolution | dilated causal convolutions | long context, parallel training, stable gradients |
| Transformer-based | self-attention over the context window | long-range dependencies, many covariates |
| N-HiTS | basis expansion, multi-resolution backcast/forecast | long horizons where temporal shape dominates covariates |
| Temporal Fusion Transformer | variable selection, gating, attention, direct multi-horizon | mixed static, known-future, and observed covariates |
None of these is a default winner: the choice follows from horizon length, covariate richness, and how much data is available to share across series.
N-HiTS
N-HiTS is a neural architecture designed for efficient long-horizon forecasting. It uses hierarchical interpolation and multi-resolution decomposition to represent patterns at different temporal scales. Blocks produce a backcast, which explains part of the input history, and a forecast, which contributes to future predictions.
The multi-resolution structure helps with long horizons because the model can allocate coarse components to slow-moving trends and finer components to local variation. Typical hyperparameters include hidden-layer sizes, number of blocks, learning rate, batch size, maximum training steps, and dropout.
N-HiTS is a good candidate when the dataset contains many related series, the horizon is long, and covariates are less central than temporal shape. It still requires careful validation against seasonal baselines and tabular models.
Temporal Fusion Transformer
The Temporal Fusion Transformer is a multi-horizon architecture designed for static covariates, time-varying known inputs, and time-varying observed inputs. Its components include static covariate encoders, variable-selection networks, gating, recurrent processing, multi-head attention, and direct multi-horizon outputs.
Static encoders condition the model on entity attributes. Variable-selection networks learn weights over input variables. Gating controls information flow. Recurrent layers process local temporal structure, while attention can connect relevant positions across the context window.
Interpretability claims should be handled carefully. Attention weights and variable-selection weights can aid analysis, debugging, and hypothesis generation, but they should not automatically be treated as causal explanations.
Auto-tuned neural variants
Auto-tuned variants search over architecture, learning rate, batch size, dropout, training steps, and related hyperparameters. This can reduce manual tuning effort, but it increases the risk of overfitting validation periods if the search is not nested or evaluated on untouched forecast origins.
Practical guidance
- Use neural forecasting when many related series can support representation sharing.
- Compare against naive, seasonal-naive, statistical, and tabular machine learning baselines.
- Monitor overfitting by horizon and by series group.
- Keep covariate availability explicit, especially for future-known inputs.
- Treat attention and variable-selection outputs as diagnostic signals, not causal proof.
- Budget for more tuning, compute, and artifact management than simpler models require.
Common failure modes
- Training a large neural model on a small set of short series.
- Using future observed covariates that would not be available in production.
- Reporting only global accuracy and missing poor performance on sparse series.
- Overinterpreting attention weights.
- Tuning architecture on the final backtest period.
Connections
Deep learning forecasting usually follows machine learning forecasting but replaces hand-built lags with learned sequence representations. The main local variants are RNN and LSTM forecasting, temporal convolutional networks, transformer-based forecasting, and N-BEATS and N-HiTS.
References
Nav