Steve Teig

Learn a development pattern to systematically improve the accuracy and reliability of LLM applications in our new short course, Improving Accuracy of LLM Applications, built in partnership with Lamini and Meta, and taught by Lamini’s CEO, Sharon Zhou, PhD, and Meta’s Senior Director of Partner Engineering, Amit Sangani. (Disclosure: I am an investor in Lamini.) The path to tuning an LLM application can be complex. In this course, you'll learn a systematic sequence of steps for improving accuracy by reducing hallucinations:  - Create an evaluation dataset to measure model accuracy - Add prompt engineering and self-reflection - Fine-tune your model including "memory-tuning" which is a new method of embedding facts in an LLM Using the Llama 3-8B parameter model, you will: - Build a text-to-SQL agent with a custom schema and simulate situations where it hallucinates - Understand the difference between instruction fine-tuning, which gives pre-trained LLMs instructions to follow, and memory fine-tuning - See how Performance-Efficient Fine-tuning (PEFT) techniques like Low-Rank Adaptation (LoRA) reduce training time by 100x and Mixture of Memory Experts (MoME) reduces it even further I appreciate Meta releasing the Llama's family of open models -- this course gives an example of the unique type of work that developers can do with such models. Please sign up here: https://lnkd.in/gyxHwPkp

3,932

72 Comments

Did you know that this company that has surpassed Apple was once near bankruptcy? The story of NVIDIA's ascent is one for the ages: 🔎 In 1993, Nvidia was founded by dreamers Jensen Huang, Curtis Priem, and Chris Malachowsky. Their quixotic quest? Crafting processors to supercharge video games. But their 1995 chip NV1 was a brutal reality check - a flop that nearly bankrupted them. 🔎 Salvation arrived in 1997 with NV3. Outperforming rivals, it landed Nvidia deals with PC titans Dell Technologies and Gateway. They'd been granted a seat at the 3D graphics table. 🔎 1999's GeForce 256 GPU was Nvidia's big bang. Bringing massive parallel processing to PCs, it revolutionized gaming visuals overnight. The world took notice of these graphics gurus.  🔎 Nvidia charted its own path, unconventionally championing gaming as the next cultural wave before it crested. Jensen Huang's foresight cemented their iconic brand. 🔎 Restlessly innovating, they expanded GPUs to mobile, automotive, and kickstarted the AI/deep learning era. Nvidia emerged as a multidimensional computing force. 🔎 Today's climax? Nvidia's stock skyrocketed past $AAPL to become the 2nd largest $SPX component at $1164 - an all-time high with a 1:10 split looming. The former underdog has outgrown the giants. 🔎 From near-bankruptcy to industry dominance, NVIDIA's rise is a GOAT tale of perseverance, vision, and constant reinvention. They dreamed of changing graphics computing forever, and delivered that in spades.

11

LLMs and Optimization: An Unbeatable Combo!

19

Do LLMs really have "Emergent Properties"? That one single feature that is used to market and over-hype LLMs is actually a big debate topic! In this length medium article “A Sanity Check on Emergent Properties in Large Language Models” the author, Anna Roger, critically examines the often-cited phenomenon of emergent properties in LLMs. She highlights the lack of clarity surrounding what constitutes “emergence” and presents empirical data that challenges its validity. One of the most striking findings is that many NLP researchers are skeptical of emergent properties themselves—only 39% believe they exist, yet 70% think that most of their peers believe in them, reflecting a sociological misconception in the field. Furthermore, the article argues that what is often called emergence could be an artifact of in-context learning or data contamination, suggesting that these properties are not as “unexpected” as believed. Perosnally, I think that while LLMs undoubtedly show impressive capabilities, we should remain cautious in ascribing phenomena like “emergence” to them without more rigorous definitions and investigations, especially with what this promise brings to business stakeholders and how it sets the expectations of what LLMs can do.. if overdone, it may put us again into another AI winter. Article Link : https://lnkd.in/eKQCdcN2

6

And just like that, Waymo announced they have an end-to-end model for autonomous driving. Former CEO John Krafcik states: 1. there is long-term promise for an e2e approach (e.g., interpretable decision rationale is possible), but 2. big challenges remain (insufficient long-term memory to solve complex long-tail challenges, sensor fusion), so 3. it's going to be super hard for e2e-only companies (Wayve, Waabi, Tesla) to catch Waymo, which has the only safely working AI approach in the space & a huge on-going advantage: Alphabet AI adjacency/talent

120

11 Comments

CoreWeave and Lambda are growing 1,000% year-over-year reselling NVIDIA's compute to LLMs, AI startups, and anyone building AI features into their product. Why is NVIDIA sweet on CoreWeave and Lambda? - NVIDIA is arming these GPU cloud rebels with priority access to advanced chips to conduct a proxy war through them against cloud giants Amazon Web Services (AWS), Google Cloud and Microsoft Azure, which are all developing their own custom silicon. - AWS and other clouds have become GPU-constrained as a result. For example, even though Azure competes with CoreWeave on cloud GPU, it has signed a $2B+ contract with CoreWeave out of necessity to ensure that it has enough compute to meet its growing demand from OpenAI. - Around the wedge of the GPU, the GPU clouds (CoreWeave, Lambda and Together AI) are now building production-grade, AWS-like tooling for autoscaling, networking, and fine-tuning models to enable them to win on developer experience long after the GPU shortage is over. Check the comments for links to our full memos on CoreWeave, Lambda Labs, and Together AI.

27

6 Comments

Much has been said about many companies’ desire for more compute (and data) to train large foundation models. I think it’s under-appreciated that we also have nowhere near enough compute available for inference on foundation models. Years ago, when I was leading teams at Google, Baidu, and Stanford that focused on scaling up deep learning, many semiconductor makers, data center operators, and researchers asked me if AI would continue to make good use of more compute if they kept delivering it. For many desktop workloads, like running a web browser, a faster CPU doesn’t help much beyond a certain point. So do we really need faster and faster AI processors? Each time, I confidently replied “yes!” and encouraged them to keep scaling up compute. (Sometimes, I added half-jokingly that I had never met a machine learning engineer who felt like they had enough compute. 😀) Fortunately, this prediction has been right so far. However, beyond training, we are also far from exhausting the benefits of faster and higher volumes of inference. Today, a lot of LLM output is for human consumption. A human might read around 250 words per minute, which is around 6 tokens per second (250 words/min / (0.75 words/token) / (60 secs/min)). So it might seem there’s little value to generating tokens much faster than this. But in an agentic workflow, an LLM might be prompted repeatedly to reflect on and improve its output, use tools, plan and execute multiple steps, or implement multiple agents that collaborate. So, we might generate hundreds of thousands of tokens or more before showing any output to a user. This makes fast token generation very desirable and makes slower generation a bottleneck to taking better advantage of existing models. That’s why I’m excited about the work of companies like Groq, which can generate hundreds of tokens per second. Recently, SambaNova also showed it can hit hundreds of tokens per second. Incidentally, faster, cheaper token generation will also help make running evaluations (evals), which can be slow and expensive today since it involves iterating over many examples, more palatable. Fortunately, both training and inference are rapidly becoming cheaper. I spoke with Cathie Wood and Charles Roberts of the investment firm ARK, which is famous for its bullish predictions on tech. They estimate that AI training costs are falling 75% a year. If they are right, a foundation model that costs $100M to train this year might cost $25M to train next year. Further, they report that for “enterprise scale use cases, inference costs seem to be falling at an annual rate of ~86%, even faster than training costs.” I don’t know how accurate these specific predictions will turn out to be, but with progress in both semiconductors and algorithms, I do see training and inference costs falling rapidly. This will be good for application builders and help AI agentic workflows lift off! [Original text: https://lnkd.in/dJ9tVGh7 ]

2,900

80 Comments

I love Nvidia CEO Jensen Huang's unique perspective on dismissing 1:1s. "The problem with one-on-ones is you deprive a whole bunch of other people in the organization to learn from the same mistakes" Over the years, 1:1s have gotten this isolated connotation which can only happen behind closed doors. Jensen’s stance is contrarian and raises a great point - a culture of openness makes it easier to solve challenging problems. By sharing information openly with everyone, you break down silos and ensure that every team member has equal access to insights and knowledge. It may not be a “conventional” practice but it's the "best practice" for Nvidia. Wonder if anyone else has tried this and how it's worked?

26

1 Comment

Hallucinations in LLMs are inevitable, no seriously! Josh Fritzsche and I have worked hard at Voze to find a way to tune, prompt and even plead 🙏 with LLMs to prevent them from the occasional 🍄 hallucination. It seemed like the better we have done the harder the edge cases have been to solve and eventually we realized that they tend to frequently happen when LLMs predict or make anticipations on "nothing". I remember at one point nearly going insane with frustration because when LLMs had a hallucination, they were EPIC! 😱 Then we pushed through it and benefited for the effort! How did we solve this common problem? We didn't (Not completely), we got better at identifying the issues as they happened. Once we identified in the data we were able to start looking for patterns. The same catalysts that cause it to happen once if left unchanged caused it over and over again. Our solution is astoundingly simple. Once you know that it's likely to happen..... let it! Accept it will 🍄, but also be preemptive in not focusing on it and use alternative data sources, generators or providers. Change the conditions that cause it but continue to monitor it as it happens. As Dan Caffee is fond of saying to me on a regular bases. "Dusty we need the system to be adaptive and provide mechanisms to learn from it's past." Well he is right! All we needed was data and the people that contribute to it's quality. Huge props 👏 to strong support from Kathy and Janelles audit teams. We were armed with a knowledge and a desire to learn from past mistakes. Armed with confidence that we could correct these problems we could focus on the data to prevent their impact going forward. When doing millions of LLM (Generative AI) calls regularly Josh and I have learned that hallucinations will happen. We also know exactly how we will handle and prevent it going forward! Failure is not doing anything and accepting defeat.

4

2 Comments

The 2024 edition of the Stanford AI Index is out, and highlights PaLM-E and RT-2 as this past year's research spotlights in robotics: https://lnkd.in/gemZZzXJ

48

1 Comment

Loved bringing together a group of research-centric, product-minded technologists building the future of AI tonight in the 🅱️🍎, incl. product / eng leadership @ NVIDIA, Pinecone, Datadog; founding eng @ Abnormal Security, Modal, Counsel Health; applied AI @ OpenAI, Anthropic; research eng @ Meta (FAIR), Harvey, Google DeepMind; founders @ Sapien and some of NYC's greatest stealth AI co's ☄ We covered a lot of ground over 4 hours - some of the recurring themes in our conversations: - Agents:🕴Model inference is getting so much faster, which will unlock AI agents' potential. That said, agents should be trained to handle very narrow tasks, reliably, first vs. an open-ended approach many startups are attempting today - Robotics: 🤖 While the science is not there yet, the robotics market and its impact will be much bigger than LLM. The introduction of humanoids will fundamentally change how we live. That said, the U.S. is in a vulnerable position due to lack of supply chain independence, compared to rivals like China - Value creation: ✨ AI personalization is overrated as consumers mostly care about status (i.e., ‘whatever is the best’) vs. what's tailored to them. Delivering great UX is still hugely undervalued, for both enterprise and consumer use cases. Cost and reliability remain enterprise buyers' front-and-center priorities. AI can preserve endangered rare languages by documenting and creating digital archives / databases cc: our amazing sponsor Arc (Brendan Bettencourt, Erika Bricky, Eric Lo), Visible Ventures

136

5 Comments

I continue to be alarmed at the progress of proposed California regulation SB 1047 and the attack it represents on open source and more broadly on AI innovation. As I wrote previously, this proposed law makes a fundamental mistake of regulating AI technology instead of AI applications, and thus would fail to make AI meaningfully safer. I’d like to explain why the specific mechanisms of SB 1047 are so pernicious to open source. To be clear, there are routes that regulators should pursue to improve safety. For example, I would welcome outlawing nonconsensual deepfake pornography, standardizing watermarking and fingerprinting to identify generated content, and investing more in red teaming and other safety research. Unfortunately, the proposed bill pursues a less beneficial and more harmful path. SB 1047’s purported goal is to ensure safety of AI models. It puts in place complex reporting requirements for developers who fine-tune models or develop models that cost more than $100 million to train. It is a vague, ambiguous law that imposes significant penalties for violations, creating a huge gray zone in which developers can’t be sure how to avoid breaking the law. This will paralyze many teams. You can read the latest draft of the law online. I’ve read through it carefully, and I find it ambiguous and very hard to follow. Developers who try to navigate the law’s complex requirements face what feels like a huge personal risk. It requires that developers submit, under penalty of perjury, a certification of compliance with the requirements of the law. But when the requirements are complex, hard to understand, and can even shift according to the whims of an unelected body (more on this below), how do we ensure we are in compliance? For example, the certification must include many different sections. One is an analysis of “the nature and magnitude of critical harms … the model might reasonably cause or enable.” But given that even leading AI researchers aren’t sure what harms models might cause or enable, how is a team of developers supposed to figure this out and declare — under penalty of perjury — that they meet this requirement? Further, some developers will be required to implement “protections to prevent … misuse of, or unsafe post-training modifications of, the covered model and all covered model derivatives … that are appropriate in light of the risks associated with the covered model, including from advanced persistent threats or other sophisticated actors.” Even leading AI researchers don’t agree on how best to “protect” AI models against these supposed risks, or what would be “appropriate.” So how are developers supposed to figure out how to comply with this requirement? [Reached LinkedIn length limit. Rest of text here: https://lnkd.in/g_NhQpni ]

2,723

203 Comments

New Tesla Patent Reveals Clues to an Exciting Future of Humanoid Robots and AI Integration In a interesting discussion, Herbert Ong and Scott Walter dive deep into Tesla's recently published patent application WO2024072984A1 (link in comments). Entitled "Actuator and actuator design methodology," the patent primarily focuses on controlling robot movement through strategic actuator placement. However, the document also unveils important clues about the potential for end-to-end neural net training of humanoid robots, reminiscent of Tesla's approach to training its vehicles with FSD. Ong and Walter explore the concept of personalized humanoid skills. Imagine purchasing a bot with built-in capabilities, then having the option to augment and customize those skills through a "Skills Store" akin to Apple's App Store. This opens up a world of possibilities for tailoring robots to specific tasks and preferences. The discussion also touches on the potential for various robot form factors, each designed to excel at particular skills. Picture a tennis-playing robot that comes pre-trained with basic volleying instructions, but requires a "Skills Upgrade" to perform at a master level. This raises questions about the need to manufacture and train a diverse range of robot body types and sizes, mirroring how humans possess ideal physiological characteristics for different activities. Later, Ong and Walter discuss a scenario in which Tesla builds the hardware – including automobiles, humanoids, and other bot form factors – while xAI develops the brains. This arrangement opens up opportunities for licensing agreements between the two companies, that may help the companies navigate future antitrust concerns as these technologies are deployed at scale. The convergence of advanced robotics and artificial intelligence holds the potential to catalyze a profound societal transformation. As Tesla and xAI continue to push the boundaries of what's possible, I can envision a future where intelligent, adaptable robots become integral to various aspects of our lives – from manufacturing and transportation to healthcare, recreation, and beyond. The seamless integration of cutting-edge hardware and software will revolutionize the way we live and work, ushering in an era of unprecedented efficiency, productivity, and innovation. While there are still challenges to overcome and ethical considerations to address, the prospect of these combined technologies driving positive change on a global scale is awe-inspiring. #FrontierFrontier #Humanoids #AI #Robots

7

1 Comment

if you know, you know NVIDIA annual shareholder meeting today: june 26 2024 09:00 pst in about an hour we'll have a chance to listen to new developments. it's breath-taking. the speed of not only NVIDIA's ability to navigate (well i guess they must be using their own (ai) to decide what where how to do and then some. huh?) more importantly just as Apple Microsoft defined the pervasiveness of computers. their applications. we believe NVIDIA holds that position currently. they have been handsomely rewarded in the markets. a valuation as the most valuable in the world. 3.3 trillys. and going. so every and all announcements are truly where the centre of the universe of chip design, manufacturing, exists. it's more than simply offering state of the art chip design for (ai) modelling. where Alphabet Inc. organizes the world's information. NVIDIA makes sense of it. and everyone else applies it, to their subject matter expertise. this is the change that is afoot. at this moment. where it was a race to offer advertisers of services/products a better understanding of their clientele, now the lens is sharper. there is no need to suggest age/sex/ethnicity/generational differences. there are behaviours. theses behaviours are independent of what data was used to point adverts to. to get these people to buy more sh^t. that's the base of the u.s. economy. it's apparently consumer driven. and so in this post pandemic world where it literally stopped. and we were forced to sit still. wait it out as the storm clouds passed. in hopes no one. not us. or loved ones would perish. that time was a generational shift. we all felt. it. no matter what class you were. NVIDIA is the fulcrum Google is the waterfall or Google is the head NVIDIA the neck wherever the neck turns that is where the head has their attention. the neck keeps the attention of the head there. so it appears as if in many ways Google has in essence offered the search part of it's business up for grabs. this is why Perplexity exists, and others. a new dynamic pricing auction model is about to emerge. it will as it is by nature adversarial. let's not fool ourselves. we are designed to be adversarial. it's our nature. and in coveting $$$$$ as the prime objective. money for the sake of money this will destroy a lot of thoughtful, mindful, intentionally beneficial innovation. why? well. it's not playing according to the whims of a recommendation machine that rewards those that are part of it's invisible army. today is a crowning. it's a crown. upon the head of jensen. this is akin to king charles and his crowning. we wonder who will be at the crowning. to recognize this eventful day. it's good to be king. ;) < . > source: https://lnkd.in/eumX4num

1

Discover how Waymo's autonomous ride-hailing service is revolutionizing independence for people with disabilities in this sponsored article. Jessie Wolinsky, aka 'Badass Blind Babe,' shares her journey navigating San Francisco with ease, thanks to Waymo. Her story highlights the impact of accessible technology & the importance of embracing new experiences. #Accessibility #AutonomousVehicles #Inclusion https://lnkd.in/g_F2Bd57

3

Robotics and AI will touch all aspects of life and society. This one “could” be a path in reducing or eliminating the herbicides used on crops. On the technical side, from the article, this is astounding. Who would have thought even 10 years ago GPUs would be utilized in this manner. “Powered by 24 NVIDIA GPUs, Carbon Robotics’ vehicles use deep-learning-based computer vision models to autonomously identify and eliminate weeds via CO2 lasers. LaserWeeder, as the product is called, processes 4.7 million high-resolution images per hour, zapping 5,000 weeds per minute, according to the company. To date, its dataset includes 25 million labeled plants and more than 30,000 crop and weed models” #AI #agritech #robotics #nvidia

3

1 Comment