From Mind-Controlled Computers to Humanoid Robots: A Deep Dive Into Neuralink and NVIDIA's Project Groot

TL;DR

NVIDIA's Project Groot is teaching robots to think, while Neuralink's first human patient is playing video games with his mind. This isn't sci-fi anymore—it's reality, and it's here to restore human independence. #VentureStep #AI #Robotics

INTRODUCTION

Welcome to a future once reserved for science fiction, where robots might soon cook your dinner and advanced technology can read your mind. In this episode of Venture Step, we explore two groundbreaking advancements that are making this future a reality: NVIDIA's Project Groot, a foundational model for general-purpose humanoid robots, and Neuralink's first successful human trial, which is already changing lives. ¹

I'm your host, Dalton, and we're diving into the core technologies that power these innovations. We'll break down what a "foundational model" for robotics actually is, using a simple Lego analogy to explain how NVIDIA is creating robots that can learn and adapt to any task, rather than being built for just one. This approach promises to give people back their independence, from elderly care to hazardous cleanup. ^2222222

We'll also explore the touching story of Nolan Arbosh, the first human recipient of a Neuralink implant. ³ Paralyzed from the chest down, Nolan can now control a computer and play complex video games using only his thoughts, a testament to how technology can profoundly improve day-to-day life. We’ll discuss the mechanics, the ethical considerations, and the incredible potential of these world-changing technologies. ⁴

KEY TAKEAWAYS

• NVIDIA's Project Groot is a foundational model for humanoid robots, designed to learn complex, general-purpose tasks through a combination of simulation, imitation, and language understanding.
• The technology stack behind Groot includes Omniverse for hyper-realistic reinforcement learning, LLMs for natural language commands, and Mimikgen for human-led imitation training. ⁵⁵⁵
• Neuralink's first human trial with Nolan Arbosh successfully demonstrates the power of brain-computer interfaces to restore independence for paralyzed individuals, enabling them to control computers with their minds. ⁶⁶⁶⁶
• The long-term vision for Neuralink includes creating a "digital bridge" to bypass spinal injuries, potentially allowing people to walk again by sending brain signals directly to their limbs. ⁷
• These rapid advancements in AI and robotics are happening at a pace that may outstrip society's ability to adapt, raising critical questions about job displacement and the need for new societal guardrails. ⁸⁸⁸

FULL CONVERSATION

Dalton: Welcome to Venture Step Podcasts where we discuss entrepreneurship, industry trends, and the occasional book review. ⁹Get ready for a future where robots cook your dinner, care for your grandparents, and maybe even read your mind. ¹⁰NVIDIA's Project Groot and Neuralink's mind-blowing human trials are making sci-fi a reality. ¹¹ Before you dive in, I'm Dalton, your host. My background is a bit of a mix of programming and insurance. Offline, you could find me running, lost in a good book, or building my side business. ¹²

Introducing Project Groot and Neuralink's Human Trials

Dalton: Today we'll be discussing NVIDIA's Project Groot that was announced two weeks ago during NVIDIA's conference. ¹³ We'll also be discussing Neuralink's human trials. Neuralink was recently approved by the FDA for their first human trial, and that individual had their surgery in January. ¹⁴Two weeks ago, Neuralink sent out a live demo with Nolan, who was the first human where Neuralink is being prescribed to. ¹⁵Nolan is paralyzed from the chest down from a freak diving accident eight years ago, and he lives with his parents. ¹⁶

Dalton: The video discussed how Neuralink has changed his life in only a short time. It's a really touching story about how technology can help out and improve the day-to-day lives of people that you might not see on the surface. 17 They're around, they need help, and they don't want to have to rely on other people. They want to have some level of independence. 18NVIDIA, Neuralink, Figure, and Tesla Optimus have a promise to give people independence again. 19

What is NVIDIA's Project Groot?

Dalton: Groot was announced by Nvidia a couple of weeks ago and is part of the project roadmap for Gear. ²⁰Gear is an additional research arm that NVIDIA spun up with Jim Fan being the head of it. ²¹ Jim Fan is the head of AI research and innovation at NVIDIA. He got his PhD at Stanford and is heading the next chapter of AI research at Nvidia. ²²Groot is a foundational model, and I'm not sure everyone understands what that is, so we'll discuss it. ²³

Understanding Foundational Models With a Lego Analogy

Dalton: I think the best way to do so is with an analogy with Legos. Say that you have three Lego sets. You have one that is just basic Legos. ²⁴Then you have another set that is a kit to build a princess castle, and another set that's a kit to build a Batmobile. ²⁵ Those kits could represent purpose-built models. If you build a Batmobile, the model knows exactly how to build a Batmobile. But if you try to build a princess castle using the Batmobile kit, you're obviously not going to be able to because you won't have the right parts or instructions. ^26262626

That's the point of a foundational model or general purpose model, where you have these basic blocks.

Dalton: A foundational model is where you have these basic Legos and you could build a Batmobile or a princess castle without having the instructions. ²⁷But to do that, you need a solid understanding of your environment, a solid understanding of physics, and a grasp of how your body interacts with your external environment. ²⁸ There are crazy things like, when you step as a robot, how long should you wait until your next step? Or how much force is in the rotation between steps? How do you lower your center of gravity on slippery surfaces? All these crazy things that we don't think are complicated are pretty complicated for robots. ²⁹

The Tech Stack Behind Groot: Omniverse, Isaac Lab, and MimicGen

Dalton: One of the ways you can tackle this is with reinforcement learning, typically done in a simulated environment. ³⁰ The way that Boston Dynamics does it is they have these code-built models where engineers code out the conditions and supplement it with reinforcement learning. That's kind of what Groot is doing. ³¹ They'll have reinforcement learning, which is Omniverse. Omniverse is a hyper-realistic physics-built model. ³² For example, in Dune 2, they used Omniverse to generate and simulate the grains of sand when the sandworm was moving around. That was from Omniverse. ³³

Dalton: They built a lab on top of it called Isaac's Lab. Think about it as a robotics playground where robots can do these reinforcement learning activities. 34What's cool about it is the robot would learn a thousand times faster in the Isaac lab than it would in a real-time simulation. 35The throughput between real-time versus a simulated environment is a thousand times, which is huge. 36

Dalton: So they have this Omniverse piece for reinforcement learning. Then they have LLMs, or large language models. 37 Groot is also going to be using LLMs. Think about Omniverse and Isaac Labs as one building block, and LLMs as another. 38With LLMs hooked up to the robots using the Groot model, robots would be able to communicate back to you and understand tasks on the fly. 39 In the demo, they show someone asking, "Hey, can you give me a high five?" The robot understands the audio, turns it into text, understands what it needs to do, and puts its hand up for the high five. 40

Then they also have this other thing, which is pretty cool, called MimicGen. MimicGen allows a human to control the robot with VR and not only control, but teach the robot what to do. ⁴¹

Dalton: So what you would do is hook up your MimicGen train the robot how to do a task a couple of times in a specific area. ⁴²And then the robot would understand what to do and it would learn, which is crazy. ⁴³

What Jobs Could General-Purpose Robots Do?

Dalton: What kind of jobs would Groot be able to enable for a safer or more efficient workplace? I think there are some pieces of manufacturing that could remove the dangerous human element. ⁴⁴ Sorting of trash, hazardous debris cleanup. Space travel was something that was really interesting. ^45454545 If you're going to set up a base on the moon or Mars, you would need to deploy a fabricator to build out what you need. Then a robot could handle putting the items together to set up the base before humans arrived. ^46464646

Dalton: I think the biggest one is elderly care. You might be thinking these robots are probably $200,000 or $500,000. 47The Tesla Optimus robot, according to Tesla, is estimated to be $20,000 to $25,000, which I think is very reasonable given that it's cheaper than the average price of a car. 48This robot would be able to complete general tasks and help out at home. 49

Neuralink's First Human Trial: The Nolan Arbosh Story

Dalton: Now for Neuralink. I talked about the patient Nolan Arbosh, who became paralyzed from the chest down. ⁵⁰He had to move in with his parents because he can't function by himself anymore and they can't afford a stay-at-home nurse. ⁵¹He was selected to undergo the Neuralink surgery. ⁵² Basically, Neuralink will open up your skull and put a chip in your brain. The chip has a thousand receptors that monitor your neural activity. ⁵³

The chip would eventually understand what you're thinking and they call them neural spikes. These communications are sent via Bluetooth to the device that they're trying to communicate with. ^54545454

Dalton: Insane, crazy stuff. This is straight out of sci-fi, controlling computers with your brain. 55 Nolan has only had the chip functional for a couple of months. Neuralink demoed his experience at his house, and they played chess and just communicated how cool it is to be independent, to be able to talk with his friends on the internet, and play video games whenever he wants without relying on his mom or dad to help him. 56He felt that it was awesome and amazing to have increased independence. 57

Playing Video Games With Just a Thought

Dalton: One of the things that he likes to do is play Age of Empires, which is a 4X strategy game. ^58585858They're quite challenging, but the caveat is the increased time allotment to complete a game. ⁵⁹ He would have to have his mom or dad click everything on the computer, telling them where to click and what to do. It wasn't a good experience because they don't know the game and he can't move as fast as he wants to. It was impossible for him to play. ^60606060

This technology enabled Nolan to play. The first night, he played Age of Empires until about 5:30 or 6 a.m. He wanted to keep playing but his Neuralink died, so he had to charge it and go to bed. ⁶¹

Dalton: I'm sharing the video of Nolan playing chess, and you can see his mouse moving around. He's moving the mouse with his mind. 62He said he started out by thinking "move" in his head, but it didn't work well. 63 He said you need to really think about the movement and manifest where you want it to go and imagine what you want to do. That works. 64Manifesting the ideal result worked with the computer versus him saying the command, which is pretty interesting. 65

The Future Vision: From Mind Control to Restoring Movement

Dalton: That being said, there are some ethical concerns with Neuralink. The results of their test subjects on animals were less than desirable. Quite a few of the test subjects died, and some of the animals were just no longer what they used to be. ⁶⁶People have been bringing that up and are very upset about it, and rightly so. ⁶⁷

Long-term, Neuralink envisions being able to not only connect people to computers, they want to connect people to robots, the Tesla Optimus robot. ⁶⁸

Dalton: They also envision Neuralink being able to allow individuals to walk again. ⁶⁹ The way around it is to put a device in your spine. This would basically make a computer-brain interface. ⁷⁰The user would send commands with their brain, the Neuralink chip decodes the signals into movements, and then it sends the message down to microelectrodes connected to the spinal cord with what they're calling a "digital bridge" to stimulate movement. ⁷¹The digital bridge sends the movement signals to the limbs that are damaged and bypasses the original nervous system. ⁷²Basically, Neuralink wants to create an artificial neural system using your brain as the key compute. ⁷³

When Technology Outpaces Society

Dalton: Tech is very close to just outpacing society. I envision that this time in our current present day is the inflection point for AI and technology advancements that we've never seen before. ⁷⁴We are moving rapidly and I don't think society has put in place the right guardrails. ⁷⁵

Dalton: When electricity came about, people had jobs lighting street lights every night. ⁷⁶ Electricity replaced their job, but it didn't turn on everywhere at once. It was a slower process where workers could re-educate and find another job. ⁷⁷ But with this AI stuff, it's kind of the snap of your finger and it's just there. There is no slow transition where you've got ten years or five or three. ⁷⁸

Dalton: The robots are pretty inexpensive in the grand scheme of things, like $20,000. You could buy five robots and potentially replace five people. 79I think once the robots are ready, large companies will book big orders, and once they get to mass-producing them, it is what it is. 80 I am intrigued but a little unsettled, and I am really excited about the opportunity to adapt along with the robots. It's not about replacing us; it's about how we adapt alongside them and reap the benefits of our relationship. 81

RESOURCES MENTIONED

• NVIDIA
• Project Groot
• Neuralink
• Tesla Optimus
• Boston Dynamics
• Figure AI
• Lockheed Martin
• Lordstown Motors
• Dune: Part Two (Movie)
• Age of Empires (Video Game)
• Stellaris (Video Game)
• Meditations by Marcus Aurelius (Book for next episode)

INDEX OF CONCEPTS

NVIDIA, Project Groot, Neuralink, Nolan Arbosh, Jim Fan, Tesla Optimus, Omniverse, Isaac Lab, Mimikgen, Boston Dynamics, Lockheed Martin, Dune 2, Lordstown, Age of Empires, Stellaris, foundational model, general purpose model, reinforcement learning, LLMs, Large Language Models, brain-computer interface, digital bridge, neural spikes, 4X strategy game