Application of a Factorial Designed Experiment to Optimize Selection of Reinforcement Learning Observations for a Hexapod Trajectory Following Task

Abstract

This thesis describes the development of a hexapod simulator built in the MATLAB Simscape environment, with the goal of testing the potential for a designed experiment to be use in the selection of observations for a reinforcement learning controlled hexapod design. The hexapod is controlled using a novel combination of a central pattern generator consisting of six coupled Hopf oscillators, and mapping functions with parameters updated via a reinforcement learning agent. The reinforcement learning agent is trained to control the hexapod using the Deep Deterministic Policy Gradient (DDPG) algorithm on a trajectory following task. Through implementation of a designed experiment testing different combinations of observations, a model is formulated to estimate the observations required to maximize the hexapod training reward. The model is validated in the simulator and the capabilities of the hexapod are further demonstrated on more complex path following tasks.