Google DeepMind has launched Gemini Robotics, a new on-device vision language action model designed for autonomous robotics that operates without cloud support.

This model is optimized for local execution, requiring minimal computational resources and allowing for rapid experimentation with dexterous manipulation tasks.

In demonstrations, Gemini Robotics showcased its versatility by enabling robots to perform complex tasks like unzipping bags and folding clothes.

Initially trained on Google's ALOHA robots, the model has been adapted for use with the bi-arm Franka FR3 robot and the humanoid Apollo robot from Apptronik.

Generative AI enhances the model's capabilities, allowing robots to generalize from fewer training examples and learn new tasks after just 50 to 100 demonstrations.

The on-device model operates independently of data networks, making it suitable for latency-sensitive applications and effective in environments with intermittent connectivity.

Developers can customize the Gemini model for specific applications, providing greater flexibility in robotic tasks.

Accompanying the model's launch, Google is releasing a Gemini Robotics SDK that enables developers to train robots on new tasks using the MuJoCo physics simulator.

Initially, access to the model and SDK will be limited to a select group of trusted testers to gather feedback on performance and safety.

Carolina Parada, head of robotics at Google DeepMind, emphasized that the cloud-free approach enhances the reliability of robots in challenging environments.

Benchmark tests indicate that Gemini Robotics On-Device performs comparably to its cloud-based counterpart while surpassing other unnamed on-device models.

The model allows robots to understand commands and execute tasks requiring fine motor skills, demonstrating its adaptability in various scenarios.