NVIDIA's Open-Source Robot Brain & The Future of AI

TL;DR

NVIDIA dropped an open-source foundation model for robots, making it easier for anyone to build the next generation of AI. The race is officially on. #VentureStep #AI #Robotics

INTRODUCTION

The world of robotics is moving at a breakneck pace, with major announcements dropping seemingly every few weeks. From the hyper-athletic (and slightly terrifying) Boston Dynamics Atlas to the sleek and aesthetically pleasing Figure 02, the hardware is evolving rapidly. But the true game-changer might not be in the physical mechanics, but in the software that powers their artificial minds.

In this episode of Venture Step, host Dalton Anderson dives deep into the most significant recent development in the space: NVIDIA's Project GR00T and their Isaac platform. Dalton explains how NVIDIA has released a powerful, open-source foundation model for humanoid robots, a strategic move designed to democratize robotics development and create an entire ecosystem dependent on their specialized chips.

This episode is a comprehensive tour of the current robotics landscape. Dalton breaks down NVIDIA’s strategy, explains the technology behind their Omniverse simulation engine, and showcases the incredible results of their collaboration with 1X AI. He also provides a comparative look at the latest demos from Figure AI and Boston Dynamics, highlighting their vastly different approaches to design and function.

KEY TAKEAWAYS

• NVIDIA's Project GR00T is a foundational AI model designed to accelerate the development of humanoid robots by providing a powerful, open-source starting point.
• By open-sourcing its robotics model, NVIDIA is replicating a classic tech playbook: democratize the software to sell the specialized hardware (chips) it runs on.
• The pace of innovation is staggering, with companies like 1X demonstrating the ability to teach a robot complex tasks in just one week using NVIDIA's platform.
• While companies like Figure AI focus on sleek, aesthetically pleasing designs, Boston Dynamics prioritizes hyper-efficient and sometimes unsettling movement capabilities.
• Robots are moving beyond demos into practical applications, from assembling car parts for BMW to unloading 90,000 pounds of boxes daily for Gap.

FULL CONVERSATION

Dalton: Welcome to Venture Step podcast, where we discuss entrepreneurship, industry trends, and the occasional book review. I spoke about robots and Nvidia figure 01 and some other robots. I also touched on Boston Dynamics, and then I also touched on Tesla's Optimus robot. In that episode, I also touched on the infrastructure that Nvidia was building.

Dalton: The episode is called "AI Butler's and NVIDIA's Superbrain," which was a horrible name looking back, but I spoke about that in March of 2024. In that episode, I talked about the different programs that Nvidia was building and how they all pieced together into this seamless training experience for the robots. You could easily put on some VR headsets and some gloves, control the robot, teach the robot how to do something, and take that 3D render.

Dalton: You then take that 3D rendering and the colors, throw that into Omniverse, and there you go. From that one or two-hour training experience with the robot in person, now you've trained for years on that one item in the virtual world, and then you can transfer that back to your many robots on your fleet.

Introducing Project GR00T: An Open-Source Foundation Model

Dalton: Quite dense, quite interesting. But then last month, NVIDIA had their NVIDIA AI day. And during that conference, they spoke about one thing that I didn't get to touch on, because there's been so much stuff coming out. Everyone wants to make their announcement. But NVIDIA announced their group, Isaac, and that was their robotics open-source model that you can use Omniverse to train new applications for.

Dalton: But it already has a lot of things that are built in. It's kind of like if you used ChatGPT, but then you wanted to do it for health insurance or manufacturing. It knows manufacturing, but it might need a little bit of help in certain areas. And so, Nvidia built this foundational model for companies to train on it. Instead of me explaining it, I'm going to share a quick video of the actual explanation.

NVIDIA's Strategy: Democratize the Software, Sell the Hardware

Dalton: It's pretty cool, right? It's something similar to when Google was first starting. The more people that use the internet, the better. The more people that are building robots, the better for Nvidia, because Nvidia is the one offering the chips for the robots, especially for smaller companies that don't have the infrastructure to build their own chips.

Dalton: The only companies that have the infrastructure to build their own chips would be companies that are foundries or companies that can afford the R&D costs and overhead. The companies that can afford to do that are very minimal, and the companies that are interested in that kind of thing are pretty much just Google and Meta. And then anyone else building chips, they build chips because that's what they do.

Dalton: So, Nvidia is always going to be building chips. Intel's going to be building chips, AMD's going to be building chips, but they're not competing with Nvidia. Google has their TPUs or tensor processing units, and then Meta is working on a chip in-house.

But as I was saying, if you're any kind of startup or if you're not Meta or Google or some massive trillion-dollar market cap company, then your easiest bet is to use Nvidia's hardware, their GPUs, their infrastructure to do real-time tele-operating and training.

What Is Omniverse and How Does It Train Robots?

Dalton: And then use Omniverse and Cosmos. It didn't explain it in that video, but Omniverse is a physics-based, real-world engine, and it's used to simulate the world. It's supposed to be as close to a real-world simulation as it can get. There's all sorts of crazy things like simulating global warming or different events or if you drop a rock, how a meteor hits, how big the waves are, whatever it may be.

Dalton: It also has a practical application for things like training robots, and that's what Nvidia is using. I think it took them like seven years to build the engine. It took a long time. And using Omniverse and Cosmos, you're able to train the robot and it's just overall sick.

Case Study: 1X AI and NVIDIA's One-Week Collaboration

Dalton: To emphasize how sick it is, I'm going to share this one-minute video of an NVIDIA and 1X collaboration. Basically, in this collaboration—and I didn't know this until earlier today—I'd watched the demo from 1X about their Neo robot, and the Neo robot was doing household tasks in a home with another human, which is unusual because a lot of these robot demos are in controlled environments. They're in a lab, they're in a warehouse.

Dalton: I think Tesla has a video of a robot in a house, but it didn't seem like people actually live there. Whereas 1X and Nvidia did a partnership and their engineers worked together for about a week. Once you have that background, you can see, wow, they did all this in one week. It's pretty cool.

For them to be able to do everything they said in a week with just a little bit of collaboration and hard work, I think it emphasizes how big of a deal that is, where instead of everyone building their own model, there's this already pre-built model that people can just attach onto.

Dalton: That will dramatically reduce the resources required to get started and allows people to focus on the things that they need to, like specified tasks for the robot that it doesn't come with out of the box. Whereas before, people were all building their own models and it doesn't make much sense.

The Power and Promise of Open-Source Innovation

Dalton: Everything is closed source. From NVIDIA's perspective, people using the model, having it be open-source and integrating very well with Nvidia products, it's a win-win for Nvidia. If you open-source it and that's the preferred model for your hardware to run these AI chips on, then it just makes everything more seamless. And you know, the kid at home who's in high school or middle school that dreams about robots, they're going to use the open-source model, the model that's easiest to use.

Dalton: When that person grows up, they'll be using that same company probably. When I started learning programming, I was in the Google Developers Club, and that's a club sponsored by Google. They give you hundreds of dollars of Google credits, you build projects, and you do that using Google products. It's an example of how this can trickle down over a 10-year period.

Dalton: It's a game-changer because it removes the barriers to entry by a lot and it also allows people to tinker and try things out.

Once you put something in the hands of millions and millions of people, people are going to build cool stuff.

Dalton: It may be from someone that doesn't have an established background and wouldn't be able to interact with these things unless they were working at that company with their closed-source model. I'm a big believer in open-source simply because it gives everyone an opportunity, but an opportunity doesn't mean that there will be results. A lot of results are by determination and grit. You can see that's true with a lot of these highly esteemed universities like Harvard, Stanford, and MIT. They have open-sourced all of their courses and lectures, and people just don't do it.

The Unprecedented Velocity of Tech Advancement

Dalton: Kind of on a tangent there, but really what I'm trying to say is that it's really cool because it allows builders to build, and things are going to become even more interesting. It seems like every month there's something crazy that comes out. It's just hard to keep up with and understand the velocity at which the world is moving at the moment in terms of technology.

I don't even know what six months looks like from now, because six months from now there could be five monumental announcements all at one time.

Dalton: It's very exciting. I'm thrilled with it, and I think that it's going to open up a lot of opportunities for other shops to get into robotics. You could be an engineering shop, but you don't necessarily have all the infrastructure to build robots. But now that you have this open-source model, really all you need to do is integrate all the sensors and you could have a crappy robot. Doesn't mean your robot's any good, but you can at least try to build something.

A Look at Figure AI's New Helix Model

Dalton: I want to talk about two more robots. While I was writing an article for one of my podcast episodes, I had remembered about the Figure 01 robot and I was like, "Oh, I wonder how that company is doing." Behold, they have an announcement with this Helix AI. They now have another cool announcement that happened a month ago that I'd missed. It's a real-time reasoning model. I think prior their reasoning model was a partnership with OpenAI.

Dalton: The video shows two robots in a home lab putting away groceries. They're collaborating, handing each other items, and they both know where stuff goes. It's pretty neat. But this is the new robot, and these arms are buff. There are some big arms, just solid steel. The demo where they're pushing the robot around and it was deflecting it... my goodness.

Dalton: Overall, of all the robots, I think I appreciate Figure's robots the most on a design perspective. I think the design is beautiful. The glossy gray with the matte blacks and the lights in the right areas. The way it looks, it just looks sleek. It looks great, honestly.

The Unsettling Efficiency of the Boston Dynamics Atlas

Dalton: Now we're transitioning over to the Boston Dynamics robot and you'll see how this is more like searching on Mars; it's less chill and relaxing. I'm talking about the Boston Dynamics Atlas robot, which I think is kind of freaky. They're talking about how it's the most efficient robot and it's got all these cool features. It can do somersaults and run, but man, is it freaky.

Dalton: They're doing this one demo where the robot is picking up a part for manufacturing sequencing. Instead of turning around, it does a full 180 with its head and then just starts walking backward and does a 180 with its torso. It's freaky.

Dalton: In the beginning of the video, they have the robot laying down and they turn it on. It does this freaky, exorcist thing and comes up but flips its arms and legs around 360 degrees to get up. Freaky stuff. It just looks so weird.

The 180 head turn and torso turn is a bit unsettling.

Dalton: I feel more comfortable with the Figure robot because the Figure robot seems chill. This is Black Mirror. The Atlas robot did a whole bunch of crazy stuff where it did a cartwheel, a handstand, and it did breakdancing moves. I just wanted to show you the 180 contortions. They said that they did that to become more efficient, which I get, but man, is it unsettling.

From Demos to Real-World Deployment: BMW and Beyond

Dalton: The main gist of it is that things are heating up. Figure 01, and now Figure 02, completed a successful robot test with BMW on their manufacturing plant, putting together rear bumpers. Normally you would have a human put the pieces together, but in the testing, Figure 02 completed the assembly for BMW's autonomous manufacturing line. BMW said it was successful and they're looking forward to reevaluating the autonomous robot capacity on their assembly line.

Dalton: They're sensitive about it because they're going to have to either move people or fire them, especially in a highly unionized industry. But if they're doing the testing for it, they're definitely open to it. You're not going to fly robots to Germany and test it in your plant and then say, "Yeah, I was just thinking about it."

Specialized Robots: Spot and Stretch in the Field

Dalton: Boston Dynamics also has Spot, which is the little robot dog that costs $75,000. Now Boston Dynamics is providing actual value with the robot. Its applications are for monitoring industrial sites. If a sensor reports an anomaly, instead of sending a human out there, Spot would get a notification, turn on from its charging doghouse, walk over, look at it, and all of that data is integrated into a product they built called Orbit.

Dalton: Then there's another robot I wanted to talk about called Stretch. Stretch is another robot by Boston Dynamics that is partnered with Gap. It's a robot that has a manufacturing arm with large suction cups. It sucks onto a package and has an AI scanning system that understands where the package needs to be oriented on the conveyor belt.

Dalton: Basically, instead of somebody unloading and loading truck containers, Stretch is doing that for the workers. The guy in the video was saying that workers would unload trucks, and the average box for Gap was 30 pounds. The bare minimum requirement was to do 3,000 boxes per day. That's 90,000 pounds of moving. It doesn't allow people to work that job too long because of how brutal it is on your back. Stretch allows people to still work the job but not have that level of physical strain.

Instead of the workers moving boxes, they're controlling robots and are certified to control robots.

Dalton: It just provides different information that you can obtain or make use of instead of just moving boxes. So that was what I wanted to talk about today. I was on a robotics marathon. More to come on other topics, but for now, I appreciate everyone listening and wherever you are in this world, have a good day. Thank you for tuning in and I'll talk to you next week. Goodbye.

RESOURCES MENTIONED

• NVIDIA (Project GR00T, Isaac, Omniverse, Cosmos)
• Figure AI (Figure 01, Figure 02, Helix AI)
• Boston Dynamics (Atlas, Spot, Stretch, Orbit)
• 1X AI (Neo robot)
• Tesla (Optimus robot)
• Google (TPUs, Google Developers Club)
• Meta
• Intel
• AMD
• OpenAI
• BMW
• Gap
• Paul Graham

INDEX OF CONCEPTS

NVIDIA, Project GR00T, Isaac, Omniverse, Cosmos, Figure AI, Figure 01, Figure 02, Helix AI, Boston Dynamics, Atlas robot, Spot robot, Stretch robot, Orbit, 1X AI, Neo robot, Tesla, Optimus robot, Google, Tensor Processing Units (TPUs), Google Developers Club, Meta, Intel, AMD, OpenAI, BMW, Gap, Paul Graham, humanoid robots, open-source, foundation model, tele-operating, VR training, physics-based simulation, CAD software