Benchmarking Modular Genetic Programming on Deep Memory Tasks

Abstract

Partially observable tasks require a learning agent to make decisions based on the previous state, hence a requirement for memory. There is a trade-off between the flexibility and specificity of the memory. This impacts the ability of the agent to solve specific tasks versus generalize to a range of tasks. Recently, a suite of `deep memory tasks’ was proposed to evaluate different approaches to partially observable problems. In this thesis, a canonical tree-structured genetic programming (GP) framework is assumed as the starting point, with memory taking the form of a list. The interface to memory requires that canonical GP is deployed as a modular co-evolutionary framework to support multiple outputs. An empirical evaluation is performed using three deep memory benchmarks to showcase the relative strength/weaknesses of this approach. We also compare our findings with neural solutions to distinguish between the relative contribution of GP versus list-based memory.