Meta's AI Leap: Llama 3 & Horizon OS Explained

TL;DR

Meta's new Llama 3 AI is outperforming its peers while the open-sourcing of Horizon OS signals a major play to dominate the mixed reality market. The game has changed. #VentureStep #MetaAI #Llama3

INTRODUCTION

In the rapidly evolving world of artificial intelligence, Meta has just made two groundbreaking announcements that are set to reshape the technological landscape. ¹The company has released its next-generation Llama 3 AI models, which are already demonstrating superior performance against established competitors, alongside the strategic decision to open-source Meta Horizon OS, the operating system for its mixed reality devices. ²²²²² These developments signal a major push to lead in both AI and the metaverse.

In this episode of Venture Step, host Dalton delves into the technical details and strategic implications of Meta's latest innovations. ³³With a background in programming and data science, Dalton unpacks the performance benchmarks of Llama 3, his firsthand experience using the new AI for text and image generation, and the crucial differences that set it apart from models like Google's Gemini. ⁴⁴

From the massive computational power Meta has amassed to Mark Zuckerberg's philosophical stance on open-source technology, this discussion provides a comprehensive overview of why these moves matter. We explore how past competitive pressures from platforms like TikTok shaped Meta's current AI strategy and why opening up Horizon OS is a direct challenge to closed ecosystems like Apple's. 555

KEY TAKEAWAYS

• Llama 3 shows a significant leap in performance, with its 8-billion parameter model outperforming the previous generation's 70-billion parameter Llama 2 model. ⁶
• Meta's strategy to open-source its key technologies, including AI models and the new Horizon OS, is designed to foster a robust developer ecosystem and establish itself as the industry standard, directly challenging closed-source competitors like Apple. ⁷⁷⁷⁷⁷⁷
• The inclusion of coding data in Llama 3's training dramatically improved its critical thinking and reasoning capabilities, even though coding isn't a primary use case on Meta's social platforms. ⁸
• Meta is integrating Llama 3 directly into its suite of apps—Facebook, Instagram, and WhatsApp—allowing users to ask questions, generate content, and plan activities within their group chats. ⁹
• The new image generation feature in Meta AI is exceptionally fast and allows for real-time, on-the-fly prompt editing, providing a more interactive and intuitive user experience than competitors. ¹⁰

FULL CONVERSATION

Dalton: Welcome to Venture Step podcast where we discuss entrepreneurship, industry trends, and the occasional book review. ¹¹ Today we're exploring Meta's groundbreaking advancements in AI and mixed reality. From the cutting edge Llama models to the launch of Meta's Horizon OS. ¹²We'll dive into the latest developments that are shaping the future of technology. ¹³ Join us as we uncover the exciting possibilities. Before we dive in, I'm your host Dalton. ¹⁴I've got a bit of a mix of background, a programming, data science and insurance. ¹⁵Offline, you can find me running, building my side business or lost in a good book. ¹⁶

Today, we're going to be discussing Meta's AI leap from Llama 2 to Llama 3. ¹⁷We'll be discussing the Llama 3 features and we will touch on the newly announced Meta Horizon OS. ¹⁸ The information was just announced from Mark Zuckerberg and Meta was communicated on their blog as of 10 hours ago. So there isn't that much information, but we will touch on it because it's very interesting. ¹⁹

How is AI Model Performance Measured?

Dalton: Before I dive in about the models, I'm going to compare them to their prospective peers. ²⁰One thing that is being used pretty often is this thing called MMLU, which stands for Massive Multitask Language Understanding benchmark. ²¹This benchmark is kind of like a SAT for models. ²²There is this 20,000 question or 18,000 question test bank and then there are topics within those. ²³ So there might be STEM, and within STEM there's going to be engineering, coding, and all sorts of stuff. Then they'll have humanities or history. ²⁴

They'll take the average score of all the topics, and then that would be the MMLU score. ²⁵

These smaller models take up less energy. They run faster. And so where these small models really thrive would be on a device, a mobile device, like your phone, or in Meta's case... Meta's RayBan glasses. ²⁶

How Does Llama 3 Compare to Other Models?

Dalton: It's important to know that certain models are built for certain things. Llama 3's eight billion parameter model, or Google's GEMMA seven billion parameter model aren't made to do crazy tasks like building a software for some large company. ²⁷What those models are built for is you want something small that doesn't have to have that much compute or storage space. ²⁸ These models are smaller, they take up less energy, and they run faster. Where these small models really thrive would be on a device, a mobile device, like your phone, or in Meta's case, the Meta Ray-Ban glasses. ²⁹

Llama 3's eight billion parameter model performed significantly better than GEMMA's 7 billion and Mistral's 7 billion parameter model. ³⁰GEMMA was 65.3, Mistral was 58.4 compared to Llama 3's 8 billion parameter model of 68.4. ³¹Another cool thing was Llama 2's 70 billion parameter model is getting outperformed by Llama 3's eight billion parameter model. ³²So there's this significant jump in performance, not only in Llama's iteration from Llama 2 to Llama 3, but also within the same space, it's outperforming its peers who used to be leaders. ³³

I think people are on the edge of their seats because this is their 8 billion parameter model, and it's performing very well. ³⁴Then their 70 billion parameter model slightly scored better than Google's Gemini Pro 1.5, which was recently released. ³⁵People are on the edge of their seats because Meta launched these two models—their small and medium models—and they're coming out with a 408 billion parameter model that is forthcoming. ³⁶They're still training the model, but it is going to be coming out. ³⁷

What is Meta Horizon OS?

Dalton: Meta Horizon OS was recently announced. Mark Zuckerberg and Meta released a blog post announcing that they are going to be launching Meta Horizon OS, which will be the open source operating system for Meta. ³⁸This is the operating system that Meta uses for their Quest and all of their devices like Ray-Ban. ³⁹Basically it's going to be an operating system for any mixed reality devices. ⁴⁰

They're open sourcing their operating system and they're going to have an open sourced marketplace. ⁴¹This allows developers to develop on one platform and reach many users which promotes growth within this space because users get more bang for their buck and developers have more efficiencies. ⁴²

A Hands-On Experience with Llama 3

Dalton: So Llama 3 came with text and image generation. ⁴³In my experience, the text generation was similar to Gemini's tone, where they would say things in a certain way, kind of be witty, warm, and more personable than ChatGPT 4. ⁴⁴I use ChatGPT 4 to help me write scripts or improve my code, but I don't necessarily have it write me stuff because it's just not as personable as Gemini. ⁴⁵

As of today, Gemini is rejecting me. There are two things commonly talked about with these large language models: rejections and hallucinations. 46A hallucination is when you'll ask it something, and it will think it's doing what you asked, but it's completely different and not related. 47A rejection is when you ask the AI to do something and it says, "I really appreciate the help, but I can't do that." 48

It's super frustrating because you can sometimes force it to do what it says it can't do. I tried my best to convince Gemini, "Hey, you actually can generate a template for me," but it was stating, "I'm a large language model. I am only allowed to translate text." 49 And that's exactly what I was wanting it to do. It just wouldn't do it. 50 So I asked my new friend, Llama 3, to help me out. And Llama 3 was able to do it, no problem. 51From my experience, it has a warm nature similar to Gemini, but with less rejection. 52

I figured that the company who created the AI would know how to navigate its parents or parent. With that advice, I was able to get a couple thousand views in one day in less than 24 hours, which was pretty cool. ⁵³

Using Llama 3 for Practical Business Tasks

Dalton: I used Llama 3 to help me set up Venture Step's business page, and I used it to help me set up my new Instagram and link the two together, because Meta is always changing things. ⁵⁴Meta, or Llama 3, helped me figure out how to navigate potential issues. ⁵⁵ I followed Llama's advice on how to set up the business page and Instagram. I also asked it if there was a better way for me to upload 70 videos from my podcast. ^56565656

It gave me guidance and told me about the recently launched Business Suite. ⁵⁷Once you have the two accounts linked up, you could upload one video and then it would post on both platforms. ⁵⁸I figured that the company who created the AI would know how to navigate its parent. ⁵⁹With that advice, I was able to get a couple thousand views in one day in less than 24 hours, which was pretty cool. ⁶⁰

Exploring Real-Time Image Generation

Dalton: I was honestly really impressed with the image generation. I'm on Meta's AI website. It starts off with "imagine something new" and then it has some prompt generation for you. ^61616161I'm going to click on "design a flower dress." ⁶²When you have a prompt inputted, it's going to generate a series of four photos and you have the chance to edit or animate them. ⁶³If you want to change your photo here, it changes on the fly. ⁶⁴

Right now the prompt says, "design a flower dress for a runway show." ⁶⁵ So from there, if I change it to "design a flower dress for a runway show in space on Mars with falling flowers," the people in the background are fading away, and the background is a little more bleak with a shining light on the prominent figure, and there are flowers falling from the sky. It's pretty cool. ^66666666

You can not only generate images, but you can generate images on the fly and know how your prompt is changing the output of your generation, which is pretty cool. ⁶⁷I've never seen that before and it's exceptionally fast. ⁶⁸DALL-E, which is OpenAI's image generation model, is a bit slower than Meta's. ⁶⁹

Llama 3 Integrated into Your Daily Apps

Dalton: They have their meta.ai website, which is like a ChatGPT or Gemini where you log in with your Facebook account. ⁷⁰They also have integrations for one-off questions in Facebook, WhatsApp, and Instagram. ⁷¹Not only are they at the forefront of those apps, they're also within your group chats. ⁷²

Within your group chat, you could be planning a trip and ask Llama to help plan an itinerary, or what are the best times to fly, or great places to stay. ^73737373You can ask all those questions and get answers for them on the fly with sources from Google and Bing already integrated in real time into your group chat with your friends. ⁷⁴You can also make memes and make fun of your friends. ⁷⁵

What is Meta's Long-Term AI Strategy?

Dalton: To explain their strategy, we have to back up a few years to understand why they have so many NVIDIA H100s. ⁷⁶When Instagram was getting battered by TikTok, because Instagram didn't have Reels, Instagram and Facebook were losing users. ⁷⁷The thing that was different between TikTok and Instagram was how they surfaced non-relevant accounts to you. ⁷⁸TikTok shows you upwards of hundreds of millions of unknown accounts based on its algorithm. ⁷⁹

Facebook and Instagram weren't like that. They had to restructure their architecture and database schema to do these things. 80 They also needed to train their Reels algorithm. That took a lot of compute power. 81 When they have these large issues, they double down. Their decision was to double whatever they thought they needed in terms of compute resources. 82828282Fast forward a couple of years, they started working on Llama, and those prior decisions helped out because they had the compute and resources to train the model. 83

People aren't getting any better at being racist... But that's a different situation once you talk about election manipulation on the platform. Governments are going to have substantial resources... and we need to be prepared for that. ^84848484

Why Open Source is a Defense Mechanism

Dalton: Another thing they discovered was the importance of coding data. Originally for Llama 2, they didn't train their model on programming data, thinking it wasn't relevant for the end user. ⁸⁵But they discovered that for critical thinking and reasoning, their models performed much better if they included coding in their training data. ⁸⁶

This brings up why they're so serious about AI. This is from Mark Zuckerberg himself. He's like, "People aren't getting any better at being racist... but that's a different situation once you talk about election manipulation on the platform." 87He continued, "Governments are going to have substantial resources... and we need to be prepared for that. I'm really concerned that if we don't have the best open source AI and AI becomes closed source and it's controlled by the governments, that one government might have this super AI that will take over our platform and manipula88te it in nefarious manners." 89

If one group has a super AI, they can discover security lapses and exploit them without anyone knowing for years. ⁹⁰That was Meta's concern about AI and why they felt they needed to be really serious. ⁹¹They want to be the best open source model they can. ⁹²

The Future of Mixed Reality is Open

Dalton: Mark has gripes with closed source frameworks. He talks about Apple just saying no to product features Meta wanted to launch. ^93939393That really upsets him—not just the revenue, but denying innovation and the next big thing. ⁹⁴Mark doesn't want to have those limitations on the world and he wants things to be free. ⁹⁵

Meta Horizon OS was announced to become open source. ⁹⁶They're industry leading in their space of mixed reality, and now high-level technology companies like Lenovo, Xbox, and Asus will be launching products on their platform in the next couple of years. ⁹⁷A developer only has to develop in one language, and then you could ship it to all these different device companies. ⁹⁸This cements Meta as the mixed reality leader. ⁹⁹It's difficult for a company like Apple to compete when it's open source versus closed source. ¹⁰⁰Most of the time, open source is going to win. ¹⁰¹

It's great for Mark. He's in the meme game now and people seem like they're relating with him more. 102102102102

Today we spoke about Llama 3's innovations and progress, and we also touched on the newly announced Meta Horizon OS. ¹⁰³These things will shape how we interact with the day-to-day in the coming years. ¹⁰⁴ I encourage you to give it a try on your social media accounts. Let me know what kind of questions you asked or what things that you found useful in the comments. ¹⁰⁵Thank you for listening to Venture Step. ¹⁰⁶

RESOURCES MENTIONED

• Meta (Facebook, Instagram, WhatsApp, Quest, Ray-Ban)
• Llama 3, Llama 2, Llama 1
• Meta Horizon OS
• Google (Gemini Pro 1.5, Gemma)
• Mistral
• OpenAI (ChatGPT-4, DALL-E)
• NVIDIA H100s
• TikTok
• Snapchat
• Apple
• Android
• Lenovo
• Xbox
• Asus

INDEX OF CONCEPTS

AI, Android, Apple, Asus, ChatGPT-4, closed source, DALL-E, Facebook, Gemini Pro 1.5, Gemma, Google, H100, hallucinations, Instagram, Lenovo, Llama 1, Llama 2, Llama 3, large language models, Mark Zuckerberg, Massive Multitask Language Understanding (MMLU), Meta, Meta AI, Meta Horizon OS, Meta Quest, mixed reality, Mistral, NVIDIA, open source AI, parameter models, Ray-Ban Glasses, rejections, Reels, Snapchat, TikTok, Venture Step, VR headset, WhatsApp, Xbox