Over the past decade or so, roboticists and computer scientists have tried to use reinforcement learning (RL) approaches to train robots to efficiently navigate their environment and complete a variety of basic tasks. Building affordable robots that can support and manage the exploratory controls associated with RL algorithms, however, has so far proved to be fairly challenging.
Researchers at Aalto University and Ote Robotics have recently created RealAnt, a low-cost, four-legged robot that can effectively be used to test and implement RL algorithms. The new robotics platform, presented in a paper pre-published on arXiv, is a minimalistic and affordable real-world version of the ‘Ant’ robot simulation environment, which is often used in RL research.
“The initial inspirations for our work were RL studies that successfully demonstrated learning to walk from scratch on ant-like quadruped and humanoid robot simulations,” Jussi Sainio, co-founder of Ote Robotics, told Tech Xplore. “The underlying premise with RL algorithms is that programming a robot to do tasks becomes much easier and more ‘natural’—one just needs to define the available sensor measurements, motor actions, then set a target goal and plug them all into a reinforcement learning algorithm, which figures out the rest.”
Initially, RL algorithms would only perform well if they were trained on robot simulations for thousands of hours. More recently, however, computer scientists were able to teach ant-inspired, quadruped robots to walk using very little training data, attaining what is known as a high sample efficiency. This made directly training the robots in the real-world feasible, removing the need for simulation-based training.
“We quickly realized that RealAnt-like walking robots were not easily and affordably available, especially for reinforcement learning, which can easily damage the robot with abusive controls,” Sainio explained. “There was no complete combined software and hardware stack that one could take and get started with real-world reinforcement learning, compared to the simulator environments. I thus started to build my own robot and interface software prototypes.”
The primary objective of the recent work by Sainio and his colleagues was to create a simple and low-cost robotics platform based on existing baseline RL solutions. Such a platform would allow more researchers to build and test autonomous robots that can complete a variety of basic tasks in the real world.
RealAnt, the four-legged robot they created, is versatile, minimalistic and low-cost. Moreover, it can autonomously learn to walk, moving its legs in a coordinated way, and can sense its position and orientation within a given environment. Using RL algorithms, Real Ant can be trained to carry out a variety of simple and yet valuable tasks.
“One can think of the RealAnt platform as a real-world version of the ‘Ant’ simulator environment, which is a popular benchmark for RL,” Sainio said. “It is one of the easiest platforms to get started with reinforcement learning and real-world robots. The main advantage of the RealAnt platform is that it is readily and affordably available.”
Building RealAnt costs approximately $410 in materials and its individual components are easy to obtain. Moreover, the robot can be assembled in less than an hour once its individual parts are prepared. Both its hardware and software are open source and a fully assembled robot can also be easily purchased online on Ote Robotic’s website.
Its low cost of fabrication and the fact that it is easy to assemble makes RealAnt accessible to a vast amount of people worldwide. In addition, it is easier to deploy in larger numbers than more expensive and sophisticated robots on the market today.
“The RealAnt platform includes the required robot hardware (motors, sensors) and software stack (communications, tracking) for interfacing with the robot, and our published baseline reinforcement learning solution serves as an example how it can be taught to walk from scratch,” Sainio said. “The example solution is simple and uses little guidance for each learning task—in ML terms, we don’t do much manual reward engineering to shape the learning performance—which makes defining new tasks simple and straightforward.”
One of the reasons why the RealAnt robot is more affordable than other existing quadrupeds that support RL is that its body moves using 8 low-cost smart servo motors, rather than more expensive and sophisticated motors. In addition, to track its position and orientation, the robot utilizes AR tags that can be easily printed on paper and an inexpensive external webcam.
“The robot body parts are all 3-D printed and they are small enough to be printed using most consumer 3-D printers,” Sainio said. “This makes the platform cheaper to produce and modify than robots with components that are made using laser-cut or machined metal or plastic sheet approaches. As RealAnt design uses low-cost motors, we drive them carefully by limiting their maximum torque and thus they can withstand continuous rough movements during random exploration and task training.”
Sainio and his colleagues evaluated RealAnt both in simulations and real-world experiments. The robot performed remarkably well in all of these tests, showing great promise for a wide range of applications.
So far, most ML and RL techniques for robotics applications were primarily trained in simulated environments. The researchers hope that RealAnt will open up new exciting possibilities in the field, as the robot can be trained and tested both in simulations and in the physical world.
“RealAnt can serve as a real-world robotics environment and a benchmark for RL, helping to ground simulated environments with reality,” Sainio explained. “Making a real physical, legged robot that can learn to walk or to do other tasks from scratch, without a simulator, is still a relatively new and rare feat. Real-world robotics is hard to get right, so I consider it a nice achievement to create a completely new, minimal robot platform that is capable of RL.”
The robotics platform created by Sainio and his colleagues could soon help other teams to test their RL and ML algorithms on a real robot. The researchers hope that RealAnt will foster development for a wide range of applications, for instance in agricultural settings, where autonomously learning robots could be used to root out weeds and harvest plants, helping to promote biodiversity and perhaps even to reduce the use of pesticides.
“We now intend to fine-tune and extend the RealAnt platform to enhance the capabilities of the hardware, such as giving the robot more advanced sensing capabilities and possibly manipulators, and running multiple robots concurrently, by building on top of the basic platform that is now available online,” Sainio said. “We’re also researching ways to make the RealAnt walk or complete other more challenging tasks even faster, reducing the training time further.”
Robot hands one step closer to human thanks to AI algorithms
More information:
Rinu Boney et al. RealAnt: An open-source low-cost quadruped for research in real-world reinforcement learning. arXiv:2011.03085 [cs.RO]. arxiv.org/abs/2011.03085
2020 Science X Network
Citation:
RealAnt: A low-cost quadruped robot that can learn via reinforcement learning (2020, December 4)
retrieved 5 December 2020
from https://techxplore.com/news/2020-12-realant-low-cost-quadruped-robot.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.
