Andrew Ng

We are excited to back Morphing Machines. As Dr. Ranjani, CTO and co-founder says: “For the first time in India and globally, we have a processor that is truly runtime reconfigurable, catering to emerging workloads.” While Nvidia’s growth and high valuation are driven by robust demand for “infra-compute” in high-performance GPUs, spurred by Gen AI, there is a distinct need for innovative solutions tailored to a new wave of reactive applications. Enter Morphing Machines. It addresses this demand with its unique product designed for next-gen telecom, autonomous vehicles, collaborative robots, avionics, and online data analytics. This product is specifically engineered to handle heterogeneous workloads and diverse tasks with varying performance demands simultaneously, ensuring a high level of scalability. As we move from an institutionally-backed research team at IISC to productization, Golden Sparrow is collaborating with Speciale Invest and others to support Deepak Shapeti, who joined as CEO three years ago after earning his Masters at Stanford. He brings not only technical expertise but also the business acumen necessary to guide the product to commercial success. Venkat Raju Afreed Faizan Anand Rao Ritik Singh Michael Marmor

68

5 Comments

The 7th workshop on Neural Scaling Laws is about to start: https://lnkd.in/gQSB5k8v Scaling laws, emergency, alignment, transfer learning, with a focused track on multilingual models. Looking forward to kick it off with Irina Rish and discuss all these topics with Yuhai Tu, Paul Bogdan, Hailey Schoelkopf, Darshil Doshi, Rio Yokota, Neha Sengupta, Preslav Nakov, Tatiana Shavrina, PhD, Yishi Xu. Join in person or virtually! #icml2024 #ai

37

CoreWeave and Lambda are growing 1,000% year-over-year reselling NVIDIA's compute to LLMs, AI startups, and anyone building AI features into their product. Why is NVIDIA sweet on CoreWeave and Lambda? - NVIDIA is arming these GPU cloud rebels with priority access to advanced chips to conduct a proxy war through them against cloud giants Amazon Web Services (AWS), Google Cloud and Microsoft Azure, which are all developing their own custom silicon. - AWS and other clouds have become GPU-constrained as a result. For example, even though Azure competes with CoreWeave on cloud GPU, it has signed a $2B+ contract with CoreWeave out of necessity to ensure that it has enough compute to meet its growing demand from OpenAI. - Around the wedge of the GPU, the GPU clouds (CoreWeave, Lambda and Together AI) are now building production-grade, AWS-like tooling for autoscaling, networking, and fine-tuning models to enable them to win on developer experience long after the GPU shortage is over. Check the comments for links to our full memos on CoreWeave, Lambda Labs, and Together AI.

27

6 Comments

🧠🔬 We talk a lot about non-invasive neural interfaces, but let's take a trip down memory lane and look at the OG invasive BCIs that paved the way for what we're doing today! 👀 Ever wondered how a tiny chip in your brain could let you control a robotic arm? Or how some folks are playing video games... without using their hands? 🚀 Our team just dropped an in-depth blog post diving into the wild world of invasive Brain-Computer Interfaces. We're talking Neuralink, Blackrock Neurotech, Synchron - the whole gang! 💡 Highlights include: - Mind-blowing stories of BCI users regaining independence - The latest breakthroughs that sound straight out of a sci-fi movie - A peek into the future of human-machine interaction Whether you're a tech geek, a healthcare pro, or just curious about the future of neurotechnology, this post is a must-read. 🔗 https://bit.ly/46KDstY 🎧🧠 Want to stay in the loop on all things neurotech? Hit subscribe on our newsletter for exclusive updates and insights. Trust me, you don't want to miss what's coming next! What do you think? Is the future of tech... in our heads? Drop your thoughts below! 👇 #Wisear #NeuralInterfaces #Wearables #BCI #BrainImplants #Neuralink #Synchron #Blackrock

22

I recently heard Alexandr Wang, the CEO of Scale AI, be interviewed by Jared Friedman at Y Combinator's alumni reunion. It was probably my favorite talk of the entire event because I got to hear the story of how Scale made the decision to bet big on RLHF when it was an extremely small part of their business. Since making this decision, the company's grown its revenues by a, uh, enviable multiple. This is the kind of decision that every company claims they're capable of but finds excuses not to make in the actual moment (especially once companies aren't the size of a startup) so Scale's move struck me as an important case study. What I took away from Alexandr was that there are a few key ingredients: 1) clear data, 2) first-rate judgement, and perhaps the most subtle one: 3) aligned incentives in your company Without a crystal ball, the next best thing you can do to prepare your company for the future is to have the most current, leading information about the trends in your industry. Scale's positioning around RLHF came from having an extremely close pulse on customer needs because of their existing relationships with companies on the bleeding edge of AI and machine learning. Scale also hires and cultivates extremely dogged and smart people. Great judgement throughout a company comes from attracting top talent, giving them the opportunity to do great work so that they want to stay, and cultivating that talent internally. So while the expectation of great judgement starts at the top, it needs to be felt all the way down the org. Alexandr said his team insisted from very early on that a bet on RLHF data was the right bet given the company's expertise providing data and their predictions for the future trajectory of LLMs. Suffice it to say they've definitely been vindicated in that bet. Lastly, aligned incentives in your company means that everyone--executives and ICs--can freely offer different takes on where the company should focus and how to succeed. This ingredient seems crucial as a company grows in order to maintain the intellectual agility that gives startups their edge. When everyone knows that their colleagues are just as focused on doing great work (and winning) as they are, different opinions aren't just welcome but expected. Very glad I got to hear this talk and a lot more first-hand. The whole day redoubled my feeling that Weave Robotics will continue to benefit tremendously from having this community around us.

63

2 Comments

The best things come in small packages 👝 🚨 Pay attention to the rise of Small Language Models (SLMs)! They will become important going forward: 1️⃣ Efficiency: SLMs achieve comparable or superior performance to LLMs with fewer parameters. Microsoft's new Phi-3 series outperforms larger models on benchmarks like MMLU & MT-bench (Google: https://lnkd.in/gn5zYZHg Microsoft: https://lnkd.in/ghK8fm7V Meta: https://lnkd.in/gkbM7pGs) 2️⃣ Specialization: Trained on curated, high-quality datasets. With 3.8B-14B parameters & advanced datasets, these transformer decoder models match GPT-3.5's capabilities in a compact form. Accurate benchmarking/eval will be important. 3️⃣ Interpretability: Neurons in SLMs are easier to analyze, improving model transparency. 4️⃣ Cost-effective: Less resource-intensive (vs training large models) I have a chance to meet a few exciting startups here. Reach out if you are working on this space 🙋🏻‍♂️ #startups #llm #slm #smalllanguagemodels #phi3 #largelanguagemodels #gpt4 #google #gemma #llama3 #ai #ml #edgeai

24

4 Comments

Learn a development pattern to systematically improve the accuracy and reliability of LLM applications in our new short course, Improving Accuracy of LLM Applications, built in partnership with Lamini and Meta, and taught by Lamini’s CEO, Sharon Zhou, PhD, and Meta’s Senior Director of Partner Engineering, Amit Sangani. (Disclosure: I am an investor in Lamini.) The path to tuning an LLM application can be complex. In this course, you'll learn a systematic sequence of steps for improving accuracy by reducing hallucinations:  - Create an evaluation dataset to measure model accuracy - Add prompt engineering and self-reflection - Fine-tune your model including "memory-tuning" which is a new method of embedding facts in an LLM Using the Llama 3-8B parameter model, you will: - Build a text-to-SQL agent with a custom schema and simulate situations where it hallucinates - Understand the difference between instruction fine-tuning, which gives pre-trained LLMs instructions to follow, and memory fine-tuning - See how Performance-Efficient Fine-tuning (PEFT) techniques like Low-Rank Adaptation (LoRA) reduce training time by 100x and Mixture of Memory Experts (MoME) reduces it even further I appreciate Meta releasing the Llama's family of open models -- this course gives an example of the unique type of work that developers can do with such models. Please sign up here: https://lnkd.in/gyxHwPkp

3,932

72 Comments

1/ 🚀 Enterprise AI is shifting gears! Enterprises are moving from closed-source "Cadillac" models to nimble, cost-effective open-source alternatives. Why? Latency, cost, and control. 2/ ⏱️ Latency kills user experience. Closed-source AI models often can't keep up with real-time demands. Open-source models are stepping in to save the day. #UserExperience 3/ 💸 Closed-source AI models are burning holes in enterprise pockets. Open-source solutions offer a cost-effective alternative without compromising quality. 4/ 🔒 Data privacy matters. Companies prefer hosting AI models in-house rather than sharing secrets with external providers. Open-source models make this possible. #DataPrivacy 5/ 🛠️ The future is compound AI systems! Combining multiple models and tools offers better performance and reliability than relying on a single behemoth. 6/ 📈 The AI compute market is booming, and the shift from training to production workloads is creating massive opportunities. Who will lead the next wave? 7/ 🌐 Open-source AI is not just a trend; it's a fundamental shift in how enterprises deploy and scale AI. Those who adapt will thrive in the AI-driven future. #AI #FutureTech

4

In this clip, Chip Huyen breaks down her new blog post on building generative AI systems. We explore: 🏗️ Common GenAI platform components ❌ AI failure types: information vs. behavior-based 🔍 RAG and context enhancement strategies 🛠️ Output improvement techniques 🔄 Fine-tuning complexities 📊 Output format considerations Full chat link in 🧵! #GenerativeAI #MachineLearning

17

1 Comment

LLMs can take gigabytes of memory to store, which limits what can be run on consumer hardware. But quantization can dramatically compress models, making a wider selection of models available to developers. You can often reduce model size by 4x or more while maintaining reasonable performance. In our new short course Quantization Fundamentals taught by Hugging Face's Younes Belkada and Marc Sun, you'll: - Learn how to quantize nearly any open source model - Use int8 and bfloat16 (Brain float 16) data types to load and run LLMs using PyTorch and the Hugging Face Transformers library - Dive into the technical details of linear quantization to map 32-bit floats to 8-bit integers As models get bigger and bigger, quantization becomes more important for making models practical and accessible. Please check out the course here: https://lnkd.in/g66yNW8W

10,591

144 Comments

Do LLMs really have "Emergent Properties"? That one single feature that is used to market and over-hype LLMs is actually a big debate topic! In this length medium article “A Sanity Check on Emergent Properties in Large Language Models” the author, Anna Roger, critically examines the often-cited phenomenon of emergent properties in LLMs. She highlights the lack of clarity surrounding what constitutes “emergence” and presents empirical data that challenges its validity. One of the most striking findings is that many NLP researchers are skeptical of emergent properties themselves—only 39% believe they exist, yet 70% think that most of their peers believe in them, reflecting a sociological misconception in the field. Furthermore, the article argues that what is often called emergence could be an artifact of in-context learning or data contamination, suggesting that these properties are not as “unexpected” as believed. Perosnally, I think that while LLMs undoubtedly show impressive capabilities, we should remain cautious in ascribing phenomena like “emergence” to them without more rigorous definitions and investigations, especially with what this promise brings to business stakeholders and how it sets the expectations of what LLMs can do.. if overdone, it may put us again into another AI winter. Article Link : https://lnkd.in/eKQCdcN2

6

Yann LeCun: The current state of AI is limited, but useful, and has likely hit a ceiling. We need to continue to innovate to break through. WSJ: AI is no smarter than a cat Ignore the headline and listen to the substance. The problem is that, despite being limited, the current state can be used to do things faster, better, and differently than before. We don't need to wait for AGI to leverage the power of the tool. https://lnkd.in/g64T-ykZ

13

1 Comment

Had a blast hosting a dinner in SF with founding teams of Luma AI, Glean, Pika, Decagon, Meshy, researchers / engineers at Harvey, OpenAI, Google DeepMind, Tavus, and pre-seed AI founders! Some of my favorite hot takes / predictions 👀: - A new paper has found that in areas where automated verification is available (math / coding), increase in # of model calls leads to improved results; for example, Llama-3-8B can solve more CodeContests problems than a single-attempt GPT-4o, if sampled 10,000 times 🎯 - Chess is an under-rated way to measure model capabilities ♟️ - Logo churn isn’t inherently bad - churn is concerning when a good product does not lead to high retention / usage; however, if customer demand is real, you can reverse churn when you push out a better product 🧾 - Many chips companies will go to zero 🍪 - Techniques like distributed inference drives efficiency by leveraging local intelligence paired with larger models (e.g., Samsung Galaxy AI) 🌌 - Symbolic system is under-invested but could have powerful impact when paired with LLMs ⚖️ - Many enterprises are undercutting their general SaaS budget to make room for AI budget, most of which ends up going to consulting firms - over time, as AI becomes more powerful and popularized, the budget split will reverse 💵 - We are in the early innings of multimodal learning, and much work is left to be done around how to incorporate audio / video data besides large corpses of text data to achieve smart models - there are learnings from biology 📽️ - Healthcare AI will first touch the under-developed international markets, and then the U.S. 🏥 - We already have core tools to achieve AGI - just need to figure out the pieces around and scaling law 🤖 cc: Denise T., Michael Boyers

143

5 Comments

AI memory is fundamental for conversational AI and AI agents. It is critical for building personalized experiences and improving reasoning. Check out Letta’s course to understand how to use AI memory to enhance the customization and quality of your AI app.

15

"Introducing Multimodal Llama 3.2": As promised two weeks ago, here's the short course on Meta's latest open model!  This short course is created with Meta, and taught by Amit Sangani, Director of AI Partner Engineering at Meta. Meta’s Llama family of models is leading the way in open models, allowing anyone to download, customize, fine-tune, or build new applications on top of them. Learn about the vision capabilities of the Llama 3.2, and use it for image classification, prompting, tokenization, tool-calling. You'll also learn about the open-source Llama stack, which gives building blocks for many different stages of the LLM application life cycle. In detail, you’ll:  - Learn what are the features of Meta's four newest models, and when to use which Llama model. - Learn best practices for multimodal prompting, with applications to advanced image reasoning, illustrated by many examples: Understanding errors on a car dashboard, adding up the total of photographed restaurant receipts, grading written math homework. - Use different roles—system, user, assistant, ipython—in the Llama 3.1 and 3.2 models  and the prompt format that identifies those roles. - Understand how Llama uses the tiktoken tokenizer, and how it has expanded to a 128k vocabulary size that improves encoding efficiency and multilingual support. - Learn how to prompt Llama to call built-in and custom tools (functions) with examples for web search and solving math equations. - Learn about Llama Stack, a standardized interface for common toolchain components like fine-tuning or synthetic data generation, useful for building agentic applications. By the end of this course, you’ll be equipped to build out new applications with the new Llama 3.2. Thanks to Ahmad Al-Dahle, Amit Sangani, and the whole AI at Meta team at for all the hard work on Llama 3.2 — we’re excited to make these open models even more accessible to more developers with this new course! Please sign up here!  https://lnkd.in/gRjSXjkA

4,830

115 Comments

From human, to Karpathy, and to Neural Programming 🤯👩🏼‍💻 A great short talk from Andrej Karpathy on the story and code behind llm.c - the bare-metal c implementation of ChatGPT 2 training that he wrote, and outperforms (on this specific model) the leading Python libraries like PyTorch. It was overall a fun talk, but I think the key highlight and insight was in the last minute or so. It's a no brainer that he managed to significantly beat PyTorch and other Python libraries - Karpathy wrote super efficient code using lower level languages (C and below) for a very specific task. PyTorch is intended to be a widely used platform, using a higher level language (though not all of it) and provides higher level abstractions - all in the purpose of being human friendly, and generally useful across tasks, use cases, models, hardware, etc. Now comes the leap of faith - what if we move away from these human friendly general platforms, to highly optimized task specific machine written code?! machine2machine 💡 I.e as the cost of writing such code will go down to 0~ (most probably by distilled smaller models). Most of the current coding agents are still emulating human programmers, and programming processes in order to allow bootstrapping, and collaborate with humans. But at what point should we do that leap of faith? What are the required datasets and examples to train models for that? How will the underlying databases look like? How will debugging, compliance, and overall transparency look like? Though I guess you can always ask your friendly neighborhood LLM to explain it to a human programmers like to a 3 years old... 🐣 I personally believe it's not that far away - starting with small tasks and gradually expanding. Last ramble: Funny enough, it reminds me a bit of pre-transformer AutoML - using models to design highly optimized per-task model architectures, which kinda faded away. Maybe it will come back - whether in distillation, RL, or some other form. 🔮 #ai #softwaredevelopment #llm #genai #coding #karpathy

11

2 Comments