Below is the abstract of my MS Thesis, completed at UMBC. If you’d like additional information, please feel free to email me.

Problem

The problem of learning to control an agent in an arbitrary environment is difficult. In robotics, the standard approach is to hand-code and manually fine-tune a robot’s perception of its environment and the actions it should take given its current state. This is both time-consuming and expensive. A better approach is to learn features and action policies without significant manual intervention. We investigate this problem in the context of learning image features to control a fovea that moves over the images.

Approach

Using a Growing Neural Gas (GNG) neural network, a type of self-organizing feature map similar to Kohonen networks, we extract features of images by sampling NxM patches in images. Several sampling methods are investigated, including random sampling, randomly walking the fovea around an image and sampling every k moves, and structured walking where a fovea moves North to South and East to West through each image.

After features are learned by the GNG network, we place controllers at each node and use reinforcement learning to learn how to move between features. We use the degree of match between the foveated image patch and a set of topologically related GNG nodes as the state, and learn to move the fovea so as to maximize the degree of match between the foveated patch and a target GNG node’s prototype. The reinforcement learning algorithm, Q-learning, interacts with a learned GNG network using a Cerebellar Model Articulation Controller (CMAC) function approximator.

Contribution

Contributions of this work include determining the impact of network parameters (number of nodes, patch size) and sampling methods (random, random walk, structured walk) on learned features, and an understanding of how to perform local control (as opposed to using a monolithic policy as in most RL approaches) based on learned features.