4 Towards dealing with the complexity of modelling agents - modelling modelling

4.2 The complexity, specificity, error trade-off

There are several desirable formal restrictions on these measures (which can be found in [15]) and there are philosophical justifications and consequences for this three-dimensional analysis of models [9]. Here I will just look at a three examples to illustrate the different possibilities that can result.

1. A risk-averse agent might take the error rate as the primary criterion whilst accepting the model complexity as representing only a limitation on its resources. It would be very tolerant to vagueness (i.e. low specificity) only accepting a more specific model where one could be found without losing any significant accuracy. This would be particularly appropriate in safety-critical situations.

2. On the other hand if an agent is merely trying to predict some out-of-sample data as well as possible on average, then there is no particular reason not to choose quite a specific model. In addition (and depending somewhat on the problem domain) one might not want to choose an overly complex or simple model for fear of overfitting or overpredicting (e.g. as illustrated by [17]).

3. Finally, if an agent had an external source of candidate models and it knew that there would have been a tendency to elaborate (make more complex) these when they were relatively unsuccessful on in-sample data, applying a heuristic of preferring simplicity when the evidence is equal would be appropriate.

Generated with CERN WebMaker