7. A Bottom-up Investigation of Internal Context in an Artificial Agent

7.2. The Learning Algorithm

The learning algorithm is based upon the neural network invented by Chialvo and Bak in [4]. This algorithm learns a set of mappings between single inputs and single outputs using purely negative feedback. The information is stored as the set of weights associated with the arcs connecting a set of nodes. The algorithm works as follows: an input node is fired, then the arc from this node with the greatest weight fires which fires the node it leads to; this carries on until an output node is fired; if the output is correct then nothing happens but if it is wrong all the weights on fired arcs are depressed and this change in weight is redistributed to other arcs.

This `learning by mistakes' algorithm is very efficient to train and use. Also, due to the fact that successful associations are not positively reinforced, the network remains in a critical state so that if the environment it is learning about returns to a previous state, the associations it had learned are relatively quickly re-established. In other words, since an arc either fires or it does not, the network only needs to marginally depress previously correct associations and hence does not totally `forget' them. The topology of the network turns out not to be critical for the working of this algorithm. This algorithm is illustrated in section 3.

Figure 3. Chialvo and Bak's learning algorithm

I have adapted this algorithm in two ways: firstly, to allow the network to learn mappings from combinations of inputs to combinations of outputs and, secondly, to add `switching' arcs that can turn other arcs on or off.

The first adaption was achieved by introducing a global critical level for the network. Arcs leading from a fired node can only fire if their weight is greater than the critical level. The critical level is gradually changed so as to ensure that the appropriate level of firing occurs. As before, all fired arcs that cause output nodes to fire when they should not have are depressed.

Secondly, arcs are of two kinds: switched and unswitched. Unswitched are as already described, switched arcs are enabled by an arc that leads to them (an enabling arc)- they can only fire if: (1) the nodes they come from have fired; (2) their weight is greater than the critical level; and (3) the arc that leads to them has fired. If the firing of an enabling arc leads to the over-firing of an output node then it is depressed in a similar manner to other arcs but to a lesser degree (e.g. depressed by 25% of the value by which the other arcs are depressed). This algorithm is illustrated in figure 4.

Figure 4. Generalised version of the algorithm

The Pragmatic Roots of Context - Bruce Edmonds - 31 MAR 99

[Next] [Previous] [Up] [Top] [Contents]

Generated with CERN WebMaker