A Comparison of PPO, TD3 and SAC Reinforcement Algorithms for Quadruped Walking Gait Generation
Open Access
- 1 January 2023
- journal article
- research article
- Published by Scientific Research Publishing, Inc. in Journal of Intelligent Learning Systems and Applications
- Vol. 15 (01), 36-56
- https://doi.org/10.4236/jilsa.2023.151003
Abstract
Deep reinforcement learning (deep RL) has the potential to replace classic robotic controllers. State-of-the-art Deep Reinforcement algorithms such as Proximal Policy Optimization, Twin Delayed Deep Deterministic Policy Gradient and Soft Actor-Critic Reinforcement Algorithms, to mention a few, have been investigated for training robots to walk. However, conflicting performance results of these algorithms have been reported in the literature. In this work, we present the performance analysis of the above three state-of-the-art Deep Reinforcement algorithms for a constant velocity walking task on a quadruped. The performance is analyzed by simulating the walking task of a quadruped equipped with a range of sensors present on a physical quadruped robot. Simulations of the three algorithms across a range of sensor inputs and with domain randomization are performed. The strengths and weaknesses of each algorithm for the given task are discussed. We also identify a set of sensors that contribute to the best performance of each Deep Reinforcement algorithm.Keywords
This publication has 9 references indexed in Scilit:
- Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a SurveyPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2020
- Assessing Transferability From Simulation to Reality for Reinforcement LearningIEEE Transactions on Pattern Analysis and Machine Intelligence, 2019
- Learning agile and dynamic motor skills for legged robotsScience Robotics, 2019
- MIT Cheetah 3: Design and Control of a Robust, Dynamic Quadruped RobotPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2018
- Sim-to-Real: Learning Agile Locomotion For Quadruped RobotsPublished by Robotics: Science and Systems Foundation ,2018
- Asymmetric Actor Critic for Image-Based Robot LearningPublished by Robotics: Science and Systems Foundation ,2018
- Domain randomization for transferring deep neural networks from simulation to the real worldPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2017
- ANYmal - a highly mobile and dynamic quadrupedal robotPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2016
- MuJoCo: A physics engine for model-based controlPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2012