MIT Researchers Introduce Enhanced Training Technique HPT for Robots

On October 28th, MIT researchers introduced a new technique for training general-purpose robots, promising improvements in efficiency and adaptability. Inspired by large language models (LLMs), which you might know from generative AI tools, this method is called heterogeneous pre-trained transformers (HPT). By combining diverse data from various domains and modalities, HPT allows robots to learn new tasks without needing to start training from scratch.

Advancing robot training with HPT

Lirui Wang, an electrical engineering and computer science graduate student and lead author of the study, explains, “In robotics, people often claim that we don’t have enough training data. But in my view, another big problem is that the data come from so many different domains, modalities, and robot hardware. Our work shows how you’d be able to train a robot with all of them put together.”

Also interesting: MIT Develops Soft Robots With Flexible and Stretchy Bodies

The main advantage of this technique is its ability to integrate data from different sources into a unified system. This approach is similar to how large language models are trained, showing proficiency across many tasks due to their extensive and varied training data. HPT enables robots to learn from a wide range of experiences and environments.

HPT offers significant benefits by reducing the time and cost of training robots. Traditionally, training robots for specific tasks has been resource-intensive in terms of time and money. Robots can achieve better performance with this new method, with data showing a 20% improvement in simulations and real-world applications compared to previous techniques.

Jialiang Zhao, a co-author and fellow EECS graduate student, adds, “Our dream is to have a universal robot brain that you could download and use for your robot without any training at all. While we are just in the early stages, we are going to keep pushing hard and hope scaling leads to a breakthrough in robotic policies, like it did with large language models.”

Impact on industry and future prospects

This development has important implications for industries that rely on robotics. As robots become more adaptable and efficient, their manufacturing, healthcare, and logistics use could be more streamlined. The ability to quickly adapt to new tasks and environments without extensive retraining could lead to wider adoption of robotics across various sectors.

Before you go: What Robots Can Do inside of You [Video]

Looking ahead, the researchers aim to create a universal “robot brain” that can be downloaded and used across different robotic platforms. Although still in its early stages, this vision holds the potential to transform robotics by enabling seamless adaptability and application without the current barriers of data specificity and hardware limitations.

In summary, MIT’s development of HPT marks a significant step forward in the field of robotics, offering a glimpse into a future where robots are not only more capable but also more accessible and versatile. As this technology develops, it could redefine the role of robots in society, making them essential tools across many industries.

YouTube: PoCo – Policy Composition From and For Heterogeneous Robot Learning (Lirui Wang, Jialiang Zhao, Yilun Du, Edward Adelson, Russ Tedrake)

Photo credit: The images shown are courtesy of the researchers and MIT and were provided as part of a press release.
Source: MIT press release