Identifying stochastic processes with mixture density networks

In this paper we investigate the use of mixture density networks (MDNs) for identifying complex stochastic processes. Regular multilayer perceptrons (MLPs), widely used in time series processing, assume a gaussian conditional noise distribution with constant variance, which is unrealistic in many applications, such as financial time series (which are known to be heteroskedastic). MDNs extend this concept to the modeling of time-varying probability density functions (pdfs) describing the noise as a mixture of gaussians, the parameters of which depend on the input. We apply this method to identifying the process underlying daily ATX (Austrian stock exchange index) data. The results indicate that MDNs modeling a non-gaussian conditional pdf tend to be significantly better than traditional linear methods of estimating variance (ARCH) and also better than merely assuming a conditional gaussian distribution. (author's abstract)