Stationary and integrated autoregressive neural network processes

We consider autoregressive neural network (ARNN) processes driven by additive noise. Sufficient conditions on the network weights (parameters) are derived for the ergodicity and stationarity of the process. It is shown that essentially the linear part of the ARNN process determines whether the overall process is stationary. A generalization to the case of integrated ARNN processes is given. Least squares training (estimation) of the stationary models and testing for non-stationarity are discussed. The estimators are shown to be consistent and expressions on the limiting distributions are given.