The Possible Incomensurability of Utilities and the Learning of Goals - Bruce Edmonds
If one is trying to simultaneously maximise more than one indicator and these are incommensurable, then this does not represent a goal in the sense of something that is a direct guide to action. In this case an agent in such a situation will have to learn its goals, or in other words, to model its indicators. This is a qualitatively different process from sub-goaling. In sub-goaling, one is looking to reduce the main goal to more less ambitious sub-goals by systematic means, whereas in modelling indicators one might accept approximate and partial realisations of the indicators.
This is particularly relevant when one changes goals. If you are using the single utility optimization model you have to change the utilities, in this model one can induce a new goal. For example*1, if one were a soldier who had certain aims: to reach a certain location in addition to some other aims such as survival, and harming the enemy and it came to pass that the first aim was impossible then it makes more sense to say that we have changed the goals rather than that the utility of reaching the location has changed*2.
It sometimes happens that the actions of an agent (or even groups of agents) can change unboundedly quickly, even when faced with a smooth and slowly changing environment (e.g. the stock market crash of 1987 which occurred in the apparent absence of any significant economic news). Indeed sometimes a continuous, slowly changing and uni-directional environmental aspect can cause an individual to "flip" back-and-forth between alternative courses of action. This is difficult to model using a single utility function optimization procedure - you have to choose a highly complex and counter-intuitive function to capture it. Using the model of learning goals this becomes much simpler - the agent could have two different equally-successful models of its goals - the choice of which is fairly arbitrary. The flipping could then be explained by merely postulating that the agent was "trying out" these alternative models, finding neither satisfactory for any length of time.
One possible mechanism for effective decision making in the face of incommensurable goals is given in [7] in terms of a model of "deliberative coherence". Another possibility is that such goal learning could take place in a more general framework where learning is represented by a process of modelling by agents, as described in [5].
Generated with CERN WebMaker