Evaluating Neural Spatial Interaction Modelling by Bootstrapping

This paper exposes problems of the commonly used technique of splitting the available
data in neural spatial interaction modelling into training, validation, and test sets that
are held fixed and warns about drawing too strong conclusions from such static splits.
Using a bootstrapping procedure, we compare the uncertainty in the solution stemming
from the data splitting with model specific uncertainties such as parameter
initialization. Utilizing the Austrian interregional telecommunication traffic data and
the differential evolution method for solving the parameter estimation task for a fixed
topology of the network model [ i.e. J = 9] this paper illustrates that the variation due to
different resamplings is significantly larger than the variation due to different parameter
initializations. This result implies that it is important to not over-interpret a model,
estimated on one specific static split of the data. (authors' abstract)