Accelerating Physical AI: A Trio of Innovations

NVIDIA’s advancements, unveiled at the Conference on Robot Learning, encompass three key areas: a next-generation physics engine, advanced robot foundation models, and enhanced development frameworks. These tools are designed to work in concert, creating a powerful ecosystem for robotic innovation.

Newton Physics Engine: Redefining Simulation Fidelity

Humanoid robots, with their intricate joints and dynamic balance requirements, push the limits of existing physics engines. To address this, NVIDIA, in collaboration with Google DeepMind and Disney Research, and managed by the Linux Foundation, has introduced Newton, an open-source, GPU-accelerated physics engine. Built on NVIDIA Warp and OpenUSD, Newton enables more precise and realistic simulations, allowing robots to learn complex tasks with greater accuracy. It seamlessly integrates with popular robot learning frameworks like MuJoCo Playground and NVIDIA Isaac Lab.

Isaac GR00T N1.6: Imbuing Robots with Reasoning Capabilities

To truly operate in the real world, robots need more than just physical dexterity; they require the ability to understand ambiguous instructions and adapt to unforeseen circumstances. The latest iteration of Isaac GR00T N1.6, an open robot foundation model soon available on Hugging Face, integrates NVIDIA Cosmos Reason, a vision language model specifically designed for physical AI. Cosmos Reason acts as the robot’s “brain,” translating vague commands into detailed, step-by-step action plans based on prior knowledge, common sense, and a deep understanding of physics.

NVIDIA Isaac Lab 2.3: A Modular Framework for Robot Learning

The latest version of NVIDIA Isaac Lab, an open-source, modular robot learning framework built on NVIDIA Isaac Sim and OpenUSD, is now available as an early developer release. Version 2.3 introduces advanced whole-body control and expanded teleoperation capabilities for more efficient data collection. OpenUSD’s interoperability ensures seamless integration between these simulations, foundation models, and learning frameworks, enabling scalable robot learning pipelines.

Industry Leaders Embrace the New Robotics Paradigm

Leading robotics developers are already harnessing these technologies to accelerate their work. Agility Robotics is utilizing NVIDIA Isaac Lab to train a whole-body control foundation model for its Digit robot, leveraging Isaac Sim and OpenUSD to create precise digital twins of customer facilities. Lightwheel has developed the Lightwheel Simulation Platform on NVIDIA Omniverse and is building simulation-ready assets using the NVIDIA USD Search API. Mentee Robotics is leveraging NVIDIA’s three-computer architecture and OpenUSD for synthetic data generation in Isaac Sim. Universal Robots is employing the NVIDIA Isaac platform for comprehensive robot simulation and learning, utilizing OpenUSD to create interoperable digital twins for validating cobot safety protocols. And Wandelbots is helping Volkswagen streamline automation projects with Wandelbots NOVA and Isaac Sim.

What impact will these advancements have on the future of human-robot collaboration? And how will these tools democratize access to advanced robotics development?

Furthermore, the robotics community is actively contributing to this ecosystem. NVIDIA Omniverse ambassador Dylan Tobin has created an AI chatbot trained on Isaac Sim workflows to assist developers navigating Omniverse.

Frequently Asked Questions About NVIDIA’s Robotics Advancements

• What is the primary benefit of a “sim-first” approach to robot development?

  A “sim-first” approach allows for parallel training of numerous robot instances using both real-world and synthetic data, significantly accelerating the learning process and reducing the risks associated with real-world testing.

• How does OpenUSD contribute to more efficient robot learning?

  OpenUSD provides a scalable and interoperable data standard, enabling developers to build physically accurate virtual worlds where robots can practice and refine their skills before deployment.

• What is NVIDIA Newton, and how does it improve physics simulation for robotics?

  NVIDIA Newton is an open-source, GPU-accelerated physics engine designed to address the limitations of existing engines when simulating complex humanoid robots. It enables more precise and realistic simulations.

• What role does NVIDIA Cosmos Reason play in robot intelligence?

  NVIDIA Cosmos Reason is a vision language model that serves as the “brain” of the robot, enabling it to understand ambiguous instructions and create step-by-step action plans.

• Where can developers learn more about NVIDIA’s robotics technologies and OpenUSD?

  Developers can explore resources such as the NVIDIA Robotics Fundamentals Learning Path, the NVIDIA Developer Blog, and the Alliance for OpenUSD website.