Autoregressive Models
An autoregressive model predicts a value from earlier values of the same series. It is the simplest mathematical expression of temporal persistence: if the recent past is above its long-run mean, the near future is likely to remain above the mean, but the strength and decay of that persistence are learned from data.
An AR model has the form
The lag coefficients are not generic feature importances. They define the dynamics of the process. In an AR(1), gives mean reversion: shocks decay geometrically instead of persisting forever. Positive coefficients create smooth persistence; negative coefficients create alternating overshoot. Higher-order AR models can represent damped oscillations and dependence at multiple short lags, but they also become easier to overfit.
The model assumes the target process is stationary or has already been transformed into one. That is why AR models sit close to stationarity and ARMA. In an ARIMA model, the AR component is applied after differencing. In a machine-learning forecaster, the same idea appears as lag feature engineering for forecasting, except the regression model may be nonlinear and may pool many related series.
AR order is often screened with the PACF: a pure AR process tends to show a partial-autocorrelation cutoff after the true order, while the ACF decays. Real data rarely follows the textbook pattern exactly, so order choice should be checked by residual autocorrelation and rolling-origin validation. Strong one-step fit is not enough, because recursive multi-step AR forecasts feed on their own previous predictions.
Connections
Autoregression is the AR component of ARMA, ARIMA, and SARIMA. It also explains why lag features matter in machine-learning forecasting and why sequence models such as RNN and LSTM forecasting can be viewed as learned nonlinear autoregressions.
References
Nav