Paul Hsu’s Post

Why did Meta invest so much in Llama3(and previous versions) and open source it? Llama3 compares well with an earlier version of Gpt4 on some benchmarks and outperforms Gpt3.5 on many. A conversation on the bird app, pointed me to a good resource which might help explain why. HT to Niran Babalola for sharing this resource https://lnkd.in/gZucZUaV It is a good read, and here is my take on common perspectives on why they are doing this Since Meta makes money from advertising, and advertisers pay for people engaging/clicking on their ads --> Meta needs more people, or more engagement/clicks from the existing user base. People come to meta to fullfill social and informational needs. More content(from genAI tools) certainly helps, though the jury is out if platforms will get swamped with generic AI generated content. Its possible some of this multi media content that users generate might help with their foray into VR. Incidentally, they also just open sourced their OS for the headsets 🤔 By releasing a good enough model for free they are effectively 'commoditizing their complement'. A complement being some product or service which is usually consumed along with the main product or service and adds value. Hence, if a firm can drive the price of that complement to zero, it can increase consumption of its product/service. In this case, the complement would be commercial LLM's such as OpenAI, which are attracting user attention and being used for content generation. Similarly, one can compare this move to what Google did when open sourcing Android to counteract Apple's IOS platform. Another interesting perspective from Emad mustaqe of Stability AI fame, discusses the value in having the community build on Llama3 and improve it. By his estimates, Meta is using about 10% of its compute to train Llama models, and if the community improves the model by 10%, Meta got a good return on investment. Its not clear if the cost of 10% compute and 10% improvement in the models is equivalent, though meta is also using llama to power AI functionality across interfaces under the Meta.ai banner (https://lnkd.in/gUCM-Pg5) Finally, it could also help Meta attract more developers who see this openness as an attractive option. This openness also means that users/developers will innovate on top of the base models and release many more variants and Meta gets more visibility. It is has been six days since launch and there are over 3500 variants (as on date) based on the two models (two versions each) released. See for yourself at https://lnkd.in/gBVBBCR6

On my never-ending journey to understand how the world works: A few hours ago, Meta released Llama 3, another breakthrough in artificial intelligence. Great benchmarks and impressive takeaways. Totally brilliant. Let’s move away from the technical side, looking for a broader approach. As always. I’ve been immersed in the economics of technology for a while now. There is one thing I haven’t been able to stop thinking about lately, this Zuck strategy. Why is a company investing billions in an open-source technology? Is this just the beginning of something bigger? Reading Carmen Gutierrez tweet confirmed some of my thoughts. People should know what’s coming and be ready for it. In Carmen Gutierrez's words: “To Meta, OAI and others like it are a small nuisance, with the potential to become a real problem later. But by open-sourcing best-in-class models now, they can neutralize that threat completely. If everyone has it, no one does.” This is a perfect example of how tech companies seek to commoditize their product complements: “companies seek to secure a chokepoint or quasi-monopoly in products composed of many necessary & sufficient layers by dominating one layer while fostering so much competition in another layer above or below its layer that no competing monopolist can emerge.” More info here: (https://lnkd.in/dQCH2wHH) So at first glance, the implications are: - Meta keeps its advertising business, where it is already unbeatable (ad targeting remains AI). - Meta creates an infrastructure likely to be monetized in other ways. -Meta employs high-skilled workers who would otherwise be working for competitors. And most importantly: -This leaves the door open for companies to create their in-house models. I can’t even imagine the huge and beneficial implications of this. Giving companies, including small and medium-sized businesses, the ability to integrate artificial intelligence into their processes is a game changer. In the end, Meta is just trying to maintain its already existing superiority in its ad market and, along the way, provides a great opportunity to society. So, thank you competition for improving society.

Last year, Apple released open-source tools for training and inference on Apple silicon. Yesterday, they provided more details about their models running on Apple silicon. While Apple's on-device models currently lag behind GPT-4 in performance, their strategy of leveraging the computing power in personal devices offers numerous advantages, potentially leading to low latency and high energy efficiency. Utilizing the processing capabilities of personal devices incurs no additional cost for Apple and ensures data remains on the device. Even the relatively low costs of using OpenAI models can accumulate over time and block some use cases. It remains to be seen whether Apple's on-device models will become sufficiently advanced for widespread applications.

Apple Foundation Model: What’s the Big Deal for AI Infra? Apple dropped some AI heat in June, and iPhone users are hyped. Sure, it’s a bummer that it only works on iPhone 15 Pro and above, but Apple’s paper on their Foundation Model (AFM-on-Device) shows exciting potential! First, the model is small - just 3 billion parameters, similar size to Phi-3. Apple squished it down using INT4 precision, meaning model weights only needs around 1.5GB of memory. With KV and overheads, the whole thing should stay under 2GB! 🚀 What’s even smarter? Apple introduced the concept of adaptors as Apple has strong controls on the whole stack, meaning iPhone can only load the small portion of the model depending on the app you’re using. If you’re in Mail, it’ll load just what’s needed for email replies. That keeps the model zippy and sharp. Now, here’s the kicker: Apple trained the big server model on 8,192 TPUv4s in Google Cloud (yep, Google’s TPUs!), then distilled it down to the tiny on-device version using 2,048 TPUv5p. If this works, it might shift more attention to TPU training, which could eat into NVIDIA’s GPU dominance. 🤔 If AFM-on-Device performs well, we might see: • More Edge AI use, where small models lighten the memory and compute load. • Data centers can get more flexible with location, as edge AI would reduce latency concerns. That said, some heavy lifting will still need to happen in the cloud (hello, O1!). • A rise in TPU training customers. Before the first iPhone, we’d seen Android and Blackberry. But the iPhone totally changed the world. This time, let’s see if Apple can lead the way for Edge AI! 🧠📱 This post is part of my Independent Study in AI Infrastructure at Stanford GSB, where I’m exploring potential challenges and opportunities in the next 2-3 years. If you have any insights or suggestions, feel free to comment or DM me! https://lnkd.in/gP3kKsTS #AI #Apple #EdgeAI #NVIDIA #TPU #LLM #390IndependentStudy #StanfordGSB #MSx

https://lnkd.in/grAibE9m 🚀 **Apple Unveils Next-Gen Foundation Models for On-Device and Server AI** Apple has launched its Apple Intelligence system, introducing powerful generative models for iOS 18, iPadOS 18, and macOS Sequoia. These include a 3 billion parameter on-device model and a larger server model, enhancing user experiences with text refinement, notification summarization, image creation, and more. Prioritizing privacy and responsible AI, these models aim to streamline daily tasks effectively and securely. Some of the benchmarks published are top of the rung 🙌 Learn more: [Apple Foundation Models](https://lnkd.in/grAibE9m) #AI #MachineLearning #Apple #Privacy #Innovation #Tech 🚀📱💡

US Department of Justice requires Google to sell Chrome, key impact is AI competition The U.S. Department of Justice has mandated Google to divest its Chrome web browser to address monopoly concerns in the search market. This antitrust action could significantly impede Google's ability to leverage its ecosystem for leadership in artificial intelligence (AI). Competition among tech giants, particularly in generative AI, has intensified, with various startups making substantial investments. Google's robust ecosystem, consisting of 15 products with over 500 million users each, enhances user engagement and supports its integrated AI tools. The flagship AI model, Gemini, is embedded in high-traffic applications like Android, Chrome, Play Store, and YouTube, each with at least 2 billion users. This wide reach gives Google a substantial advantage over competitors like Apple and Meta. In addition to the Chrome divestiture, the Justice Department's orders prevent Google from setting Chrome as the default search engine on Android devices and from paying to secure default search status on products from companies like Apple and Samsung. Google claims that these measures could negatively impact Americans and threaten U.S. leadership in technology and AI. The company is set to submit a compliance proposal next month, with a judicial ruling expected in April. #AI #Taiwan https://lnkd.in/gcBkFTWR

Now that all the keynotes are over it's clear that the real tech battle is between Google and OpenAI . Google has the entire package- top notch #GenAI models, endless amount of data, leading web services, the most popular OS, browser, #gmail and #youtube. Microsoft and Apple have no model and their faith is in the hand of OpenAI . Yes, they can make the coolest UX, but it will always be limited by the foundation model. For example, it took Apple 3 years (!!!) to copy the Magic Eraser that google launched in 2021 with the #Pixel 6! Apple always says its main advantage is controlling the #hardware and #software. But in 2024, don't they need to control the #GenAI models as well to win?

Discover why we recommend Microsoft Edge over Google Chrome for an enhanced browsing experience! 🌐💡 Explore the benefits of Edge's seamless integration with Microsoft 365, faster response times, and advanced AI features. Read more: https://lnkd.in/em_mKjrG #MicrosoftEdge #GoogleChrome #TechTips #TechThursday

Apple's Foundation Models At WWDC 2024, Apple introduced its new suite of AI models under the "Apple Intelligence" banner. Integrated into iOS 18, iPadOS 18, and macOS Sequoia, these models aim to enhance everyday user experiences through advanced generative capabilities. Key Highlights Apple unveiled a ~3 billion parameter on-device language model and a larger server-based model, both tailored for efficiency and accuracy. These models power a range of features, including text refinement, notification summarization, and image creation. Apple's commitment to responsible AI development is evident in their focus on user empowerment, authentic representation, careful design, and robust privacy protections. Utilizing the AXLearn framework, Apple's models are trained on licensed and public data, with strict filters to exclude personal information. Techniques like low-bit palletization and grouped-query-attention ensure high performance and efficiency, especially for on-device models. Apple uses adapters to fine-tune models for specific tasks without altering the base parameters, enabling dynamic specialization while maintaining general knowledge. Performance is rigorously tested through human evaluation, focusing on real-world applications and adversarial prompts to ensure safety and effectiveness. Apple’s approach to data privacy is robust. It integrates on-device processing to safeguard user data and leverages Private Cloud Compute for secure server operations. This ensures that personal data remains protected, aligning with Apple's long-standing commitment to privacy. Apple's foundation models represent a significant advancement in AI integration. They enhance user experience across its ecosystem while adhering to principles of responsible AI development and data privacy. For more details, visit https://lnkd.in/gtxia6Gd #apple #foundationalmodels #llm

Apple's annual developers' conference kicked off two days ago. As usual, the internet had a field day, with a tsunami of posts about how Apple just revolutionised AI and that nothing will ever be the same… The boring news of day 1 - partnership with OpenAI, ended being the media focus. There are already hundreds of apps which do this 😴 . I also think it will be tricky to convince regulators and customers that letting Siri decide whether your entire phonebook should be sent to OpenAI is “safe". Then again, this is just my speculation so make of it what you wish 😉 The stock market didn't like the news. Apple stock dropped about 2% (but rebounded yesterday… we'll get to that later) The most interesting part of the announcement was actually about Apple's own line of foundational models. You can find the blog post here: https://lnkd.in/eC3r_JMH. I'll try to break down a few important points: - Apple is aware that LLM benchmarks are getting saturated. There is not much to gain here, but they have to show that they can reach SOTA (and they do). I wouldn't expect the new models to perform better than any of the open source models out there. - Instead, Apple focuses on two important aspects: minituarisation and UI/UX. What we saw yesterday is to me a proof of a trend (first with Microsoft's Phi series of models, now with Apple) to get more performance out of smaller LLMs which can run on-device. This is significant because it breaks away from the dogma of "one LLM to rule them all" and it solves a whole plethora of issues regarding data privacy etc. - Apple is focusing on Human experience. A lot about AI is in UX and whoever gets this right will succeed in the AI race! This was the first time I'm seeing emphasis on evals. like "Human Evaluation of Output Harmfulness" or "Human Satisfaction Score" in a large tech company press release, suggesting this is central to Apple's foundational model program. - Apple has been preparing for the "big AI move" for quite some time. There's been a lot of criticism of the company lagging behind competition, but I think this is because they took their time to prepare the ground (e.g. M series chips), giving Apple an edge over the competition. - Apple invested a lot on technical details of minituarisation (e.g. custom LoRA, model quantisation...). The most interesting detail is the Talaria toolkit, which allows to simulate effects of various optimisations on model performance and speed up the optimisation process. To my best knowledge, there is no open source equivalent of Talaria. Apple is in a good place to pull ahead. They seem to be the only company that prepared the ground well before introducing AI into their products, they are doing impressive things with minituarisation and they are aware that the one who solves the "UI of AI" is the ones who wins. The stock market yesterday seemed to agree (AAPL +6.5% at closing)