Pterosaur Robot
Editor’s note, Feb 2025: The head-walking above is the current state of the project - not quite walking but close! I didn’t realize how much reward function tweaking I’d have to do to get a custom model to walk, reward hacking is a hard problem! I intend to return to this project at some point but other RL use cases are interesting me more at the moment.
Pterosaurs were the membrane-winged flying animals that lived alongside the dinosaurs. Their locomotion (walks on 4 legs, flighted) is very interesting, so I want to make a robot that walks (and maybe eventually flies) in the same way.
The first step was making a representative CAD model of the robot. This model was then converted to an XML description of the joints and motion dynamics, which is used by the RL libraries I’m using.
I’m starting the project with the motion control part. I’m using reinforcement learning to train a neural network to walk.
I’m using MuJoCo as a physics simulator, Brax for machine learning policies + training, and Jax for runtime optimization. All three are developed by Google Deepmind, and I’m also using google GPUs to train, through google colab.
I haven’t quite got it walking yet (as you can see), but I’m getting close. Once I do get it walking semi-gracefully, I can start thinking more about the mechatronic part of this project.
Developing a walking robot will be difficult, and developing a flapping + flying robot will be almost impossible. Thus, this is a pretty long-term project.
Below are the reward curves (and other related info) from a recent training run.
Slide deck I presented of the initial progress: