Paul Birkmeyer

Bridging the gap from a working software prototype to a production-grade product is often more challenging—and costly—than it appears, especially when explaining this to non-technical partners. Tesla’s recent “We Robot” event (ICMYI: https://lnkd.in/g5X7fihH) serves as a compelling example. The event showcased impressive prototypes of robotaxis, autonomous robots, and Full Self-Driving (FSD) technology. Yet, without concrete timelines (setting aside speculative ones), it’s evident that these innovations are still far from mass production. According to AMCI’s latest report(https://lnkd.in/gD8-MMzY), Tesla’s FSD currently requires 75 interventions per 1,000 miles of real-world driving—that’s an intervention every 13 miles. To put this into perspective, consider the “Autoland” system for airliners developed in the mid-1970s for zero-visibility conditions. Certification for this technology demanded a failure probability of less than 1 in 150,000 occurrences. The disparity between an intervention every 13 miles and a failure probability of less than 1 in 150,000 highlights the significant journey from prototype to production-ready status. As a long term Tesla investor and car owner, it's one more added disappointment. If you watched the event, what are your thoughts on the challenges of bringing such advanced technologies to market?

8

https://lnkd.in/eewVd6Jw? Last week, a Waymo got stuck in front of a Vice Presidential motorcade and had to be manually driven away by a police officer. Which is mostly one of those stories that's only interesting because an AI did it rather than a person—it's definitely not the first time someone's been flummoxed by an unusual traffic pattern, and consequently had an emotional breakdown that made them incapable of getting out of the way. But it's also in a category of stories that will get more common over time: you'll hear a disproportionate amount about models behaving badly because 1) that will happen more often when it involves famous people, who are outliers in lots of ways, and 2) it's a bigger story because there's a famous person's name in the headline, even if they have nothing to do with it. Like many other models, Waymo's training tells it to memorize and then generalize. At first it's identifying special cases, and reacting to each of them, but over time it's finding connections between those cases—learning that the important thing about a car swerving into the lane in front of it is the swerving, not the make and model. The process of training a model that gently slows to a stop at a stop sign is more straightforward than training one that figures out how to anticipate how other cars will navigate around an accident or respond to an ambulance.

2

Optimize for 🚀 Bio Cores First, Silicon 🤖 Cores Second David Heinemeier Hansson makes a compelling case: prioritize human programmers (bio cores) over computer processing power (silicon cores). Here’s why it matters: 👉 Investing in programmer efficiency is far more valuable than solely optimizing for computer speed. 👉 Human labor costs far exceed computing costs. Boosting human productivity should be the main focus. 👉 In the age of AI and high-level programming languages, the bio cores drive your real gains. Bottom line: Make your programmers more productive, and watch your team (and tech) thrive. Read the full article here: https://lnkd.in/gK8HEGtk P.S. : The image below is brilliantly summarized using Napkin.ai for a quick visual breakdown.

6

AI startup founders discuss what it's like to be a part of Nvidia's sprawling startup empire Nvidia, led by CEO Jensen Huang, has expanded its role in the AI ecosystem significantly by investing heavily in AI startups through its corporate development team and NVentures. The company’s investment portfolio grew from $300 million at the start of 2023 to over $1.5 billion, encompassing 24 investments in various AI-related fields such as foundation models, cloud providers, and humanoid robots. Notable investments include Cohere, Mistral, Coreweave, and Hugging Face. Nvidia also supports over 20,000 early-stage companies through its Inception program, which provides resources and computing power without direct investment. Huang personally approves every investment, ensuring alignment with Nvidia’s strategic vision. As competition from companies like Google and AMD intensifies, Nvidia's venture investments aim to maintain its dominance in the AI chip market by fostering a loyal ecosystem around its technology. https://lnkd.in/eDbmV2-w

2

I spent all of last week talking about how RAG patterns and Prompt Engineering continues to fail in production- this week, I’m going to be about fixing it. Through the power of simulation environments, IsoZero gives users the ability to get high zero shot performance on math, reasoning and q&a tasks. - No More Semantic Dissonance: Instead of fetching static documents that may or may not be relevant, we use AI-driven reasoning simulations. Whether you bring your own AI or connect to Claude, OpenAI, or Hugging Face agents, our system generates insights directly from the task description. This approach eliminates the misalignments that often plague RAG systems. - Iterative Reasoning, Not Retrieval: Our unique simulation environment allows AI agents to step through complex tasks iteratively, refining their understanding and outputs without relying on external retrieval. This ensures the information is not only accurate but also contextually relevant—every time. - Streamlined and Scalable: Without the burden of overwhelming retrieval systems or the risk of using outdated information, we deliver consistent performance at scale. Our solution offers real-time, up-to-date insights tailored to your specific needs, no matter how complex the task. Dump your RAGs and use our QA simulation instead. Try it on in our notebook. 

23

1 Comment

During the opening keynote at Realize LIVE, Tony Hemmelgarn focused on AI's imminent transformative effect on how we create as engineers. The point that stood out to me is that "AI feels different".  There's many promises of Industry 4.0, but we've all seen the deep commitment and investment required to truly leverage its value.  AI though, in just a short time, has given immediate value to many aspects of businesses. AI feels like it will be revolutionary and those organizations that don't start understanding now where to incorporate it will slide behind the curve. #ai #industry40 #software #newtechnology #realizeLIVE

14

4 Comments

We see a need for inference chips in our progression of building a multi-agentic interaction model. Engineering of processes is the amalgamation of systems that need to be aggregated, integrated and intelligently synthesized to achieve actionable results. An orchestration of multiple agents is therefore essential for future interaction between coordinating systems. At Numorpho Cybernetic Systems (NUMO), we are building a process engineering platform based on composable, modular and agentic constructs based on actionable intelligence - the ability to convert meaning to motion. For this, New processors enabling Neuromorphic Engineering and having Neural Processing Units (NPU) may be the next step in our evolution of computing and AI.

4

1 Comment

Experiments are critical for rigorous interpretability methods making causally valid predictions: enjoyed Atticus's talk on limitations of current methods and ways to improve them.

7

Wake up - new scaling laws paper just dropped! "Scaling Laws for Precision" Multiple interesting findings: * Training larger models in lower precision can be compute optimal - i.e. for a given compute budget you want to train larger models in lower precision rather than smaller models in higher precision in order to achieve the optimal loss * Too much pretraining data can actually be harmful as you quantize such models to lower precision for inference. Llama models popularized the data saturation regime (training on many more tokens / param than recommended by Chinchilla scaling laws (20 tok/param)) as their goal was not to train a compute optimal model but instead to optimize for inference. These models are more suceptible to degradations due to aggressive quantization. * When number of params is held fixed, compute optimal precision increases approximately logarithmically with data, i.e. for larger models you probably don't want to train in low precision even if your GPUs/training stack supports it! We're hitting the quantization limits that have historically brought us a lot of speed improvements (going from fp32 -> bf16 -> fp8). Lots more details in the paper: https://lnkd.in/dC3_EVrs --- 🔔 If you're looking to hire best ML/tech talent check out my platform here: https://lnkd.in/duJZ3D9T post a job here: https://lnkd.in/df-Ukm8E

169

4 Comments

Thinking about a paper for conf arith. "Towards a modular implementation of libm for CUDA mixed precision." A bit more than one month to write it. Some people interested in AI ? Core idea is to have the same code for double and float16, and fastmath/accurate modes both accessible as compile time, alongside fonction manipulation in exact arithmetic (integers). Would need some funding. Here an assembly sample of the fastmath mode for half2.

9

And just like that, Waymo announced they have an end-to-end model for autonomous driving. Former CEO John Krafcik states: 1. there is long-term promise for an e2e approach (e.g., interpretable decision rationale is possible), but 2. big challenges remain (insufficient long-term memory to solve complex long-tail challenges, sensor fusion), so 3. it's going to be super hard for e2e-only companies (Wayve, Waabi, Tesla) to catch Waymo, which has the only safely working AI approach in the space & a huge on-going advantage: Alphabet AI adjacency/talent

120

11 Comments

After reading Andrej Karpathy’s post on X, I’m increasingly convinced that AGI will only be complete when it includes some form of physical systems. Integrating physical components is key to unlocking the full potential of AGI. Check out his thoughts: https://lnkd.in/eRDXsnZT

8

What makes NVidia chips so valuable: computing power. By removing the computer workload from the GPU into a specialized chip, it makes AI run faster, to the point that NVidia outperforms Microsoft and Apple. It simply would not work otherwise. We're about to do the same with imagery. By moving image correction in a specialized chip (FPGA), we will cut down 90% of image computing. It even improves the computing speed for the rest if image analysis. #digitalimaging is the second most deployed digital technology after the microchip, and we're in control... We're making cameras for high end uses (3D scanning, telemetry, fusion, added compression, SLAM... ) as easy to use as a printer !

8

3 Comments

More robot stories from Urban Machine! See below for a peek into how we evolved the vision system across generations of our robots. Lots of lessons learned here! Sometimes when your solution isn’t working, the answer is to change the problem…

7

What a time to be alive. Yes, LLMs are reaching initial plateaus. But then again, the underlying compute is getting 2-3x cheaper and more energy efficient every year. Our models today, are around 40.000x more compute efficient than comparable numerical models. Already today, we can build with 5M or 10M investment, what used to take 200M+ just in infrastructure before. In one year, we can build the same for 3M. In two years for 1.5M. In three years for 0.75M (or less). In other words, a global, high resolution, high precision world model, that would have taken billions to build just 5 years ago, will be built with a few million of investment in the next 5 years. This is our exponential journey towards climate sentience.

48

4 Comments

One of the best things about being in San Francisco 🌉 is the ability to drop in on great talks. Last week I had a chance to see NVIDIA's CEO Jensen Huang (JH) being interviewed by Stripe's CEO Patrick Collison. Jensen is among the longest-serving founder-CEOs in Silicon Valley at the moment (30+ years). With that comes a special clarity of mind and some hot takes. 🌶👇 🗣 Jensen doesn't believe in 'praise in public and criticism in private'. Instead, he reasons that most information in the organization should not be privy to certain people. By giving feedback in public, you allow everyone to learn. 'What is better than learning from your own mistakes? It's learning from someone else's mistakes!!' 🔥 JH also doesn't like to fire people. He would rather "torture them into greatness". He invests in people and he doesn't like to write off his investments. 👨‍💻 Asked whether he likes his job every day, he answers "no". The perfect job doesn't give you happiness 100% of the time. "No great things were done, that were easy all the way." ... "Even if I come home from a day where I did things I didn't enjoy, it could still be have been a great day." 👩‍👧‍👦 Jensen famously has 60 direct reports. "This allowed us to cut out probably 7 levels of management". His subordinates don't necessarily have such large teams. "It doesn't scale down, less senior people need more guidance". Looking at the numbers, this approach allowed NVidia to reach ~2T market cap with ~28.000 people vs. Microsoft's 3T with ~220.000 people and Apple's 2.6T with 160.000 people. ❔ Jensen answers almost every question in the same way: "Let's break it down, let's reason about it". This is a great response on so many levels. First, it allows him to not be put on the spot. Second, it allows him to teach his workforce how he reasons through a problem so that they can emulate it and create alignment. Third, it gives people a chance to point out flaws in his reasoning. He admits he is not right all the time. ON AI 👨‍🔧 "You will not lose your job to AI, your company will go out of business because another company used AI." "If you're not using AI you're doing it wrong." ⏱ The extent to which AI will enter many processes is largely underestimated. Right now we are focused on language - sounds in a time sequence. "Any time sequence can be tokenized." This will include robotic movement. 🥛 JH also expects that we won't be satisfied with foundation models "You're going to want to tune it into perfection because you care so deeply. Say if it's already at 99% you'll want to get it to say 99.3%" This last piece is also why we created Query Vary (YC W22). AI will never feel truly intelligent until it completely understands your data and context. It needs to be taught, regardless of how powerful the foundation model is. Finally, besides insights, it was remarkable how funny Jensen was, cracking a joke at roughly a 1 per 3 min rate. It was a very entertaining talk. #ai #sf

61

8 Comments

Andrew Ng, you are absolutely right! Protecting innovation and open source is crucial for AI progress. California's SB-1047 could stifle this by imposing unreasonable liabilities on AI builders, potentially leading to the closure of many innovative projects. Rather than regulating the technology itself, it's more effective to focus on specific applications and their safety. Let's advocate against such restrictive legislation to ensure continued innovation. #AI #OpenSource #Innovation #Legislation #California #TechRegulation #AIResearch

1

This is why we can't have autonomous driving made in Germany 🇩🇪 Three days ago, Waymo announced their third round of investments of $5.6 billion - that's $𝟓,𝟔𝟎𝟎,𝟎𝟎𝟎,𝟎𝟎𝟎: https://lnkd.in/getVNB8f In total, they have raised over $11 billion, led by their parent company Alphabet Inc. Waymo was founded in 2009 within Google and is known as Waymo since late 2016 (https://lnkd.in/gDDCY2Yb). In the early days, Google invested over $1 billion dollar into the project before it officially became Waymo. Waymo is now part of the 𝐎𝐭𝐡𝐞𝐫 𝐁𝐞𝐭𝐬 category within Alphabet and probably the most expensive one. The yearly annual losses are in the billions, with just over $2 billion in Q1-2 of 2024. The patience, persistence, and ability to invest and raise such sums are simply impressive to me. Especially since they are driven by a vision that has hardly generated any revenue to date. But things are starting to change, just in October Waymo announced 100,000 paid rides per week: https://lnkd.in/gCdQFU6p Obviously, there are more reasons for Waymo's success so far, but money is a key aspect of it. For me, it seemed particularly helpful to innovate from within Google, as opposed to many German companies, which are experienced in complex safety-critical (mechanical) systems, leading to different corporate cultures. Germany is not stopping innovating, but it has become quite clear that we will not be the ones building the software systems that drive SAE Level 4 autonomous vehicles. For example, Mercedes-Benz AG, a leader in SAE Level 3 for private vehicles, just announced an increased top speed of 95 km/h (~ 60 mp/h) on German highways: https://lnkd.in/g_f6AzWf I am curious to see what role we will play in the changing mobility landscape of the future! #research #ai #innovation #invest #autonomousdriving #autonomousvehicles #safety #usa #germany #china Sources: Google 2009 -2015: https://lnkd.in/gcmDFBAr Other Bets 2016: https://lnkd.in/g2-aZ2_q Other Bets 2017 - 2023: https://lnkd.in/gZfNxVM6 Other Bets 2024: https://lnkd.in/gkvPasNs & https://lnkd.in/gck38wR9

17

1 Comment

Lawrence Berkeley National Laboratory used an algorithm called Word2Vec sift through scientific papers for connections humans had missed. Their algorithm then spit out predictions for possible thermoelectric materials, which convert heat to energy and are used in many heating and cooling applications. The algorithm didn’t know the definition of thermoelectric, though. It received no training in materials science. Using only word associations, the algorithm was able to provide candidates for future thermoelectric materials, some of which may be better than those we currently use.

6