Combining RL and MPC for Bipedal Walking

Status: Completed Course: Robot Learning and Control Tools: Python, MuJoCo, PyTorch

Report

Abstract

This project implements a hybrid control strategy combining Reinforcement Learning (RL) with Model Predictive Control (MPC) for robust and adaptive bipedal walking on uneven terrain. The RL component learns a policy for nominal walking, while MPC provides real-time corrections for disturbances.

Methodology

The hybrid architecture consists of two main components:

RL Policy: A deep neural network trained using Proximal Policy Optimization (PPO) to learn nominal walking gaits.
MPC Controller: Real-time optimization that adjusts the foot placement and joint torques based on current state feedback.

Results

The hybrid controller showed significant improvement over standalone RL or MPC approaches, demonstrating:

50% reduction in fall rate on uneven terrain
Adaptive response to external perturbations
Smooth transition between different walking speeds