Amer Ather

One of the best things about being in San Francisco 🌉 is the ability to drop in on great talks. Last week I had a chance to see NVIDIA's CEO Jensen Huang (JH) being interviewed by Stripe's CEO Patrick Collison. Jensen is among the longest-serving founder-CEOs in Silicon Valley at the moment (30+ years). With that comes a special clarity of mind and some hot takes. 🌶👇 🗣 Jensen doesn't believe in 'praise in public and criticism in private'. Instead, he reasons that most information in the organization should not be privy to certain people. By giving feedback in public, you allow everyone to learn. 'What is better than learning from your own mistakes? It's learning from someone else's mistakes!!' 🔥 JH also doesn't like to fire people. He would rather "torture them into greatness". He invests in people and he doesn't like to write off his investments. 👨‍💻 Asked whether he likes his job every day, he answers "no". The perfect job doesn't give you happiness 100% of the time. "No great things were done, that were easy all the way." ... "Even if I come home from a day where I did things I didn't enjoy, it could still be have been a great day." 👩‍👧‍👦 Jensen famously has 60 direct reports. "This allowed us to cut out probably 7 levels of management". His subordinates don't necessarily have such large teams. "It doesn't scale down, less senior people need more guidance". Looking at the numbers, this approach allowed NVidia to reach ~2T market cap with ~28.000 people vs. Microsoft's 3T with ~220.000 people and Apple's 2.6T with 160.000 people. ❔ Jensen answers almost every question in the same way: "Let's break it down, let's reason about it". This is a great response on so many levels. First, it allows him to not be put on the spot. Second, it allows him to teach his workforce how he reasons through a problem so that they can emulate it and create alignment. Third, it gives people a chance to point out flaws in his reasoning. He admits he is not right all the time. ON AI 👨‍🔧 "You will not lose your job to AI, your company will go out of business because another company used AI." "If you're not using AI you're doing it wrong." ⏱ The extent to which AI will enter many processes is largely underestimated. Right now we are focused on language - sounds in a time sequence. "Any time sequence can be tokenized." This will include robotic movement. 🥛 JH also expects that we won't be satisfied with foundation models "You're going to want to tune it into perfection because you care so deeply. Say if it's already at 99% you'll want to get it to say 99.3%" This last piece is also why we created Query Vary (YC W22). AI will never feel truly intelligent until it completely understands your data and context. It needs to be taught, regardless of how powerful the foundation model is. Finally, besides insights, it was remarkable how funny Jensen was, cracking a joke at roughly a 1 per 3 min rate. It was a very entertaining talk. #ai #sf

61

8 Comments

Life comes full circle. We started with mainframes, transitioned to desktop computing & then eventually moving cloud computing with browsers/apps providing hybrid experiences. The AI started with a fully-on-centralized model but soon environmental issues of concentrated power consumptions and economics of interconnect costs, latencies and compliances are going drive this towards a more hybrid approach in future. I truly believe that this could be the path of how AI-Federation would look like as technology evolves....

9

2 Comments

The high-end graphics cards from Nvidia, such as the H100, have catalyzed the emergence of dedicated Rent-A-GPU farms, expanding beyond traditional hyperscale providers like AWS, GCP, and Azure. Initially, supply chain limitations coupled with high demand enabled these specialized farms to purchase these cards—each costing nearly $40,000—in bulk and offer them for rent. By late 2023, rates were around $8 per hour under commitments ranging from one to three years. Today, the landscape has shifted significantly. Some of the largest Rent-A-GPU providers have adapted to changing market dynamics by offering GPU spot markets with commitments as short as 48 hours, with rates as low as $1.8 per hour or even less. This suggests that the supply chain for high-end GPUs has finally caught up with demand, potentially heralding a substantial decrease in costs not only in the rental market but also in capital expenditures for owning these GPUs. For the first time, there is downward pressure on the prices of high-end GPUs. This development is particularly promising for real-world applications of Generation AI. The significant reduction in inference costs is crucial for the viability and value creation of GenAI within enterprises. It represents the first of many indicators that we are approaching the rapid realization of game-changing use cases, transforming how businesses leverage advanced AI capabilities. At KamiwazaAI we believe the start of an 5th Industrial Revolution Enterprise #5IR is around #1trillionInferences a day or in the range of ~$3b dollars a day utilizing chat GPT4 at .003 per Inference (100 tokens). A 1000x decrease in the cost of Inference is only 18 months away or so and its time NOW to begin implementing the technology to keep up!

21

5 Comments