A robot equipped with a Gemini Robotics AI model can throw a basketball into the hoop.
A few days ago, Google's DeepMind has applied the large language model (LLM) Gemini to robots. With the model, the company says, robots can accomplish certain tasks without having to observe the movements of other robots. For example, "dunk" a mini basketball into the hoop on the table.
DeepMind is one of the companies that is trying to develop general-purpose bots using chatbot technology. However, considering that such models are prone to produce errors and harmful results, this technical path has security implications.
The research team hopes to develop a machine that is intuitive to operate and can perform a variety of physical tasks without human supervision or pre-programmes. Carolina Parada, head of the robotics team at DeepMind, noted that by connecting the Gemini model, developers can improve the robot's ability to "understand natural language and perceive the physical world at a finer level."
The model, called Gemini Robotics, was released on March 12. Alexander Khazatsky, co-founder and AI researcher at CollectedAI, an American artificial intelligence (AI) company, commented that this is a "small and tangible step" towards the goal of universal robots.
The DeepMind team is based on its state-of-the-art vision and language model, Gemini 2.0, which is trained by analyzing patterns in massive amounts of data.
The team has developed a dedicated version of Gemini to improve the capabilities of tasks involving 3D physics and spatial reasoning, such as predicting the trajectory of an object or identifying the same part of an object from images taken from different angles.
In addition, the researchers trained the model with thousands of hours of hands-on, remotely operated robot demo data. This allows the robot's "brain" to perform real-world tasks, similar to how LLMs can associate the next word in a sentence through learning.
The researchers tested Gemini Robotics on humanoid robots and robotic arms employing harmonic reducers, covering tasks that emerged in training as well as new tasks that had not been exposed to them. They say that the robot with the model outperforms its competitors in both familiar and new tasks with adjusted details.
In tasks that require delicate manipulation, such as origami or zipping up a backpack, the robot has a success rate of more than 70% after watching less than 100 demonstrations. Robots using other models almost all failed.
Khazatsky believes that the Google team has done a great job of implanting common sense into the robot's "brain," but he notes that the real leap will come from learning from data collected in the "chaotic real world" rather than in a lab environment.
Security becomes a significant challenge when applying such models. Vikas Sindhwani, robotics and AI researcher at DeepMind in New York, USA, said: "Initially, robots will keep a safe distance from humans. In the future, we will gradually implement more interactive and collaborative tasks. "