Transferring Evolved Reservoir Features in Reinforcement Learning Tasks

K. Chatzidimitriou, I. Partalas, P. Mitkas, I. Vlahavas, “Transferring Evolved Reservoir Features in Reinforcement Learning Tasks”, Recent Advances in Reinforcement Learning, Lecture Notes in Computer Science, Springer-Verlag, 7188, pp. 213-224, 2012.

The major goal of transfer learning is to transfer knowledge acquired on a source task in order to facilitate learning on another, dif- ferent, but usually related, target task. In this paper, we are using neu- roevolution to evolve echo state networks on the source task and transfer the best performing reservoirs to be used as initial population on the tar- get task. The idea is that any non-linear, temporal features, represented by the neurons of the reservoir and evolved on the source task, along with reservoir properties, will be a good starting point for a stochastic search on the target task. In a step towards full autonomy and by taking advan- tage of the random and fully connected nature of echo state networks, we examine a transfer method that renders any inter-task mappings of states and actions unnecessary.We tested our approach and that of inter- task mappings in two RL testbeds: the mountain car and the server job scheduling domains. Under various setups the results we obtained in both cases are promising.