Fred Esere

Kyle Poyar and Lenny Rachitsky just dropped one of the most tactical "how to build products with embedded AI" guides I've seen. We have the sample size now of successful AI features to learn what separates useful from gimmicky. Really cool to see the width of advice here. Including my and Elad Gil's extremely-easy-to-apply tactic 😇 I also extremely vibe with: - Claire Vo's point about incrementalism. Sprinkling AI fairy dust around your product in small-but-cumulatively-meanignfully ways is the new tactic to make your product feel luxurious and modern. - Rahul Vohra's point about speed. We see this in our data too. Lots of user drop-off when the model is too slow to respond (which has made us loath to use slow models except for really tricky questions) - Cameron Adams's point about onboarding. ChatGPT has onboarded hundreds of millions of users to prompt engineering, but that doesn't mean they will just figure out the nuance of your AI feature. Full post here: https://lnkd.in/gkfkQubh

34

3 Comments

Interesting perspective on using Frameworks for building Generative AI applications. Frameworks can abstract away a lot of details, but they can also speed up your path to production. Choosing one vs others is an 'it depends' situation. If you are in for the learning, go the hard route. If you are in for fast iteration, frameworks could help. In either case though, once you have developed the understanding, frameworks will help. Remember spring boot?

3

🛝Have you recently wondered which #LLM is best for your agentic #GenAI application and how to refine the prompt to get best results? 💥Dynamiq Prompt Playground makes it easy to quickly prototype and test, selecting from broad set of #IBM, #Microsoft, #Anthropic, #Google, #OpenAI, #Meta and other models.

6

On GPT-4o performance after additional tests: I'm still seeing accuracy on par with GPT-4-Turbo and Opus, BUT you have to be aware in prompting that this thing is a jabber-mouth. After a couple of days of additional testing on various tasks, I'm realizing that this model WANTS TO TALK. It just cannot shut up. For tasks where you either want verbose performance or you can control for it in prompting, then it's just as good as 4-Turbo or Opus (actually better in most cases given its blazing speed and lower costs). BUT, for tasks where you need precision and not over-production, Opus or GPT-4-Turbo may end up being the better choice. For example, you may have a task like "Take this list, use your judgment and pare the list down to just the items that would be important for X" (just an example, obviously you can prompt better than that 😉 ). You can get good results in a task like this with GPT-4-Turbo and Claude (I would lean toward 4-Turbo for this generally, as its freewheeling "judgment" seems to usually be closer to what I'd expect from a human). But if you give this task to GPT-4o, it may very well just spit back the whole list because IT CANNOT SHUT UP. Its "judgment" in these cases generally won't be wrong - like, if you ask it to pick all the things in the list related to cats it won't give you back dogs - but if there's any connection to be made it will make it because that will let it spit back more text and IT LOVES SPITTING BACK TEXT. Still early on, of course, and I suspect that with more prompting work some of this behavior can be controlled, but wanted to pass along the observation. What's everyone else seeing?

4

As academic based LLM benchmarks are losing their relevance -- there are 2 types of Benchmarks that will matter more in the B2B space: 👉 Crowdsourced user votes to rank LLM output (like Chatbot Arena) OR where the test DATASET is kept secret by the evaluator (e.g GSM1k from scale_AI) 👉  Enterprise use case specific LLM benchmarks (e.g produced by CRM, ITSM, HR SaaS companies) evaluated on real data. Because you can not game real customer use cases. #llm #benchmarks #llmeval #AI

11

1 Comment

Coding with Claude 3.5 Sonnet is ~10x more efficient than GPT-4o or any other LLMs out there. The new Artifacts feature enhances user experience by generating and running code next to your chat. Wild - Try it for free: claude.ai #ai #anthropic #claudeai #llms #coding

6

1 Comment

AI isn't just about replacing humans—it's about empowering them. 💪 AssuranceIQ, an innovative insurance startup, is pioneering a groundbreaking approach to leveraging generative AI: 1. They're using Large Language Models (LLMs) to score customer conversations. 2. This AI-powered system structures unstructured data from interactions. 3. The goal? Enhance the performance of human sales and support teams. 4. It's not about AI vs. humans, but AI amplifying human capabilities. 5. This could revolutionize how we train and develop customer-facing teams. This use case showcases the true potential of AI in business: not as a replacement, but as a powerful tool to augment human skills and drive performance. What other innovative ways could we use AI to empower rather than replace human workers? #AIInnovation #CustomerExperience #FutureOfWork #BusinessAI #TechTrends Share your thoughts on AI augmentation in the comments below!

10

1 Comment

Chegg's Market Value Plunge: A Lesson in Innovation and Adaptation in the AI Era The shocking drop in Chegg's market value speaks volumes about the tumultuous intersection of education technology and the aggressive march of AI. While CEO Dan Rosensweig places the blame squarely on Google and AI, one must question whether this is merely a scapegoat for deeper systemic failures in their business strategy. The reality is, Chegg’s trajectory underscores a critical lesson: in a world increasingly dominated by AI, relying on outdated models without innovation is a recipe for obsolescence. Companies must evolve or risk extinction—it's that simple. Furthermore, this dramatic decline serves as a wake-up call for all edtech entities. The market doesn’t just reward technology; it demands a profound commitment to educational utility and authentic engagement. Let's challenge the prevailing assumption that merely adopting AI guarantees success. It doesn’t. The new frontier is not just about integration but also about fostering a genuine connection with learners. As we navigate this rapidly shifting landscape, the question remains: will we adapt or fall behind? #EdTech #AI #Innovation Link to original: https://lnkd.in/eBMh5P7y

3

If you use AI in cases where precision is critical, nothing is more dangerous than a model that gets it right 90% of the time. The more I test models like ChatGPT and Gemini on REAL analytics use cases (not the clean, cherry-picked demos that product marketers use), the more warning flags I'm beginning to see... 🚩 They often completely miss the mark when tasks require a certain level of context, nuance, or domain knowledge 🚩 They give you the right answer just often enough to feel like magic, and lure you into a state of implicit trust 🚩 They "speak" with complete conviction, mirroring the same language we might hear from a confident, seasoned professional (even when the output is wildly inaccurate) And perhaps the biggest issue of all is that content marketers and sales people continue to make wild, unfounded claims and push clickbait headlines pitching AI as some sort of "magic bullet". Don't get me wrong – I think the progress we've seen in the AI space is aboslutely incredible, and could list a hundred ways that data professionals can (and should) use it to work smarter and faster. But in cases where the stakes are high or the margin of error is razor thin, we need to get real about how to leverage these tools responsibly. That means: 👉 Continuing to build strong, foundational skills so that you can ask smart questions and anticipate potential mistakes or blind spots 👉 Incorporating a layer of human review and QA before any decisions are made based on AI-generated outputs 👉 Tuning out the hype and maintaining a healthy level of skepticism 👉 Leveraging AI primarily for clearly defined, repeatable tasks that don't require additional context or nuance P.S. We're working on a new video series to take an unbiased look at how AI performs on tasks that real data professionals encounter on the job. Drop me a comment if there are any specific topics or use cases you'd like to see!

52

14 Comments

🌟 How AI is Shaping a New Era of Product Building for Founders 🌟 In the past, founders focused on building one, single monolithic product — a giant intended to conquer a specific market. But the landscape is evolving. With the power of AI, founders are shifting from monolithic giants to agile, interconnected product ecosystems. This isn’t just a trend; it’s a strategic evolution. Here’s what today’s AI-powered, multi-product strategy looks like — and why founders should embrace it: ✨ Diverse Revenue Streams — Multiple products can tap into different market segments, creating stable and diverse revenue sources. ✨ Enhanced Flexibility — Small, agile products allow quick pivots based on user feedback and market shifts. ✨ Synergy Across Products — A suite of products can create network effects; using one tool enhances the others, making the whole ecosystem stronger. Imagine a suite of productivity tools — a scheduling app, a task manager, and a note-taking tool. Each is valuable alone, but together they create an integrated experience that makes work easier and faster. This isn’t a collection of tools; it’s a seamless, strategic ecosystem. 🚀 The Rise of the Product Studio Model 🚀 In an AI-driven world, a product studio approach offers key advantages: 🔹 Shared Tech Stack — Build AI models and integrations once, and reuse them across products. 🔹 Centralized Data — Pool insights from across the ecosystem for sharper development decisions. 🔹 Unified Design — Ensure users feel the connection between products with a cohesive experience. 🔹 Modular Architecture — Microservices make it easy to share components across products, accelerating innovation. With AI-driven product studios, today’s founders can build faster, test hypotheses in real-time, and deliver interconnected solutions. Founders aren’t just solving isolated problems anymore — they’re creating ecosystems of value. This new era belongs to founders who don’t build in silos but in synergy — creating value through interconnected, AI-powered experiences. 💡 #AI #ProductBuilding #Founders #ProductStudio #Innovation #Entrepreneurship #NetworkEffects #FutureOfWork

3

Get a professional level prompts from a breif description.

9

Totally agree! This happens for both new hardware and software. Take a look at some of the big players. The initial versions are always buggy but they are released to the consumers anyway because it solves a few key problems and captures the attention of the target market. Feedback from early adopters are then used to improve subsequent versions.

2

If you're more than just a casual ChatGPT user, you've likely encountered the situation illustrated in the image. At its surface, it looks like regular A/B testing, but it's much more than that. The technique is called RLHF (Reinforcement Learning from Human Feedback), and it originated from a collaboration between OpenAI & Google DeepMind. So, what exactly is RLHF? We know that large language models (LLMs) are trained to predict the next word in a sentence, and once trained, they become really good at it. But the path from a great LLM to a product like ChatGPT isn't straightforward. Here's what it takes: 1. Making LLMs follow user instructions: Base LLMs are great at predicting the next word in a sentence, but they won't necessarily follow user instructions. For example, when prompted with "Explain gravity", a base LLM might respond with "Explain the solar system" because it saw similar lists of questions in its training data and learnt that the most likely sentence after a question is another similar question. Not very useful, right? To address this, OpenAI gathers several prompts like "Explain gravity", some handcrafted and others sampled from real user prompts sent to ChatGPT. Then, their moderators writes desired responses to these prompts. The pairs of prompts and moderator-written responses are fed back into the LLM, so it learns how to diligently follow user instructions. This step is called Supervised Fine-Tuning (SFT). The SFT model when prompted with "Explain gravity" tries to explain gravity, instead of responding with another question. 2. Making LLMs generate responses that are useful, truthful and non-toxic: Even when LLMs start following instructions, they can still generate useless responses, make false statements (known as "hallucinating") and produce toxic content. It's not easy to define what makes an LLM useful, especially for tasks like creative writing or summarization. This is where RLHF comes in. LLMs are non-deterministic and can generate a variety of responses to the same prompt. As part of RLHF, LLMs are used to generate multiple responses for the same prompt with moderators ranking each response from best to worst. This process is repeated over a large number of prompts, and the LLM eventually learns the preferences of moderators, based on how they rank responses. This produces a version of the LLM that's far more useful, truthful and non-toxic. Since ChatGPT's release, OpenAI has expanded RLHF even further, by nudging users to choose between two responses to certain prompts - all in an effort to make its LLMs even better. **** Book a 1:1 with me https://lnkd.in/gdQ7mY5f #chatgpt #llm #rlhf #finetuning #openai #deepmind

5

1 Comment

Are Generative AI Drawbacks Your Big Differentiation Opportunity? Struggling to find your differentiation in the generative AI space? What if the so-called 'flaws' in AI are actually your secret weapon for differentiation? Let's flip the script on three common AI challenges: 1. Prompt Engineering: Embrace the User’s Burden Ever feel like you're playing a guessing game with AI, needing to experiment with the inputs till you get a satisfying response? That's prompt engineering at work. While essential for quality outputs from your LLMs, it can be incredibly frustrating for users. Your move? Develop smart systems that intuitively decode user intent, even from the vaguest prompts. Think multi-prompt strategies and context-aware responses that learn from user history. Suddenly, you're not just another AI tool – you're the one that "gets" your users. 2. Hallucination: From Glitch to Unlocking Creativity AI hallucinations – those intriguing yet sometimes annoying fabrications – are usually viewed as liabilities. But what if they could be a hidden wellspring of creativity? For example, imagine you tapped into your model’s hallucination to develop an Ethical Dilemma Simulator for workplace training, or a Strategy Game with "what-if" market scenarios. The key? Clear labeling AI generated content, user controls  for adjusting the level of "creativity" or unexpectedness, and rock-solid ethical guidelines. You're not just sidestepping pitfalls; you're advancing interactive learning by embracing AI hallucinations. 3. Small Language Models: Less Can Be More In AI, bigger models isn't always better. Small Language Models (SLMs) with the right pruning and quantization, can provide superior efficiency, privacy, and specialized performance without sacrificing on accuracy. An AI that runs locally on devices offers ironclad data security and lightning-fast responses. For enterprises paranoid about data privacy (aren't we all?), this could be the clincher. Invest in optimizing these compact powerhouses to meet critical user considerations like latency and battery performance, and you might just corner the market with your privacy friendly, "snappy and secure" AI solutions. The bottom line? In the AI gold rush, your differentiator might just be hiding in plain sight – in the very challenges others are scrambling to overcome. I'd love to hear from you: How are you planning to differentiate your product or service with generative AI?

10

1 Comment

NVIDIA DIDN'T DISAPPOINT Nvidia stock pops 7% to top $1,000 after earnings beat forecasts, it announces stock split and dividend hike Here's a short summary: 💲 Earnings  - Nvidia (NVDA) reported Q1 earnings that exceeded expectations.  - Adjusted earnings per share (EPS): $6.12 on revenue of $26 billion.  - Year-over-year increase: EPS up 461%, revenue up 262%. 🔍 Analyst Expectations  - Expected adjusted EPS: $5.65 on revenue of $24.69 billion.  - Previous year’s Q1: Adjusted EPS of $1.09 on revenue of $7.19 billion. 🔭 Future Outlook  - Nvidia expects Q2 revenue of $28 billion (± 2%).  - Analyst expectation for Q2 revenue: $26.6 billion. 💹 Stock Performance  - Nvidia stock rose nearly 7% in pre-market trading, reaching above $1,000. 🌐 Key Business Segments  - Data Center revenue: $22.6 billion, up 427% year over year, representing 86% of total revenue.  - Gaming segment revenue: $2.6 billion.  - Significant growth driven by demand for generative AI training and inference on the Hopper platform. 🛠 Market Dynamics  - Non-hyperscaler use of Nvidia chips is growing.  - Large cloud providers account for around 45% of Data Center revenue.  - Revenue from China down due to halted shipments of powerful chips, with a competitive market expected ahead. ✂ Stock Split and Dividend  - Announced a 10-for-1 stock split, effective June 7.  - Increased dividend, to be paid on June 28 to shareholders as of June 11.  - Post-split stock expected to trade at around $98 per share. ↔ Industry Trends  - Nvidia’s increased dividend follows similar moves by Meta, Alphabet, and Apple.  - Speculation that Nvidia could be added to the Dow Jones Industrial Average. #nvidia

14

I compared the PRDs from the top LLMs and ChatPRD. ChatPRD emerged as a clear favorite. Well, technically, it tied with a 4o Mega-Prompt—both getting B+. But here's why I prefer ChatPRD overall. (You can see all the PRDs for yourself here: https://lnkd.in/eT6GXCQB) SIMPLE PROMPT RESULTS It's common knowledge at this point: You can't prompt a generic LLM with something simple like 'I work on Gmail. Create a PRD for Cmd+K functionality.' The resulting PRD scores anywhere from a C to an F: • GPT-4o: Much better than GPT-4, scored a solid C • Gemini Advanced: Disappointingly poor, expected more from the #2 on LLMSys, scored a D • Claude Opus: Slightly better than Gemini, but still lacking in content quality, scored a D+ • LLama-3: The worst performer, with truly garbage content, scored an F Basically, if you just simply prompt LLMs you're going to think they're garbage for PRDs. MEGA-PROMPT RESULTS But you can definitely optimize these LLMs. Since ChatGPT-4o came out on top, I focused on optimizing that. The secret is: you need to create a massive mega-prompt with stylistic advice, detailed PRD instructions, and specific section guidance. It works. The result is: • Quality of Content: Rivals ChatPRD's, both scoring B- • Usability of Writing: Below ChatPRD, scoring B compared to ChatPRD's A- • Completeness: Can be elite, scoring A- versus ChatPRD's B But that's with a heavily tuned mega-prompt just for the task. It takes significant time and effort to craft these prompts. And they're not easily reusable for other PRD tasks. CHATPRD CAN TAKE SIMPLE This is where ChatPRD shines. You can create a similar quality PRD overall in much less time: 1. You start with the same simple prompt we started with for all LLMs, a single line 2. Then you just answer the 3-5 followup questions ChatPRD has 3. And, optionally, further fine-tune (which I recommend) Like with any AI generated content, you need to edit it to sound human and ensure it didn't hallucinate. But the amount of editing is much less for ChatPRD. The usability of the writing is that much higher. As Alisa Haman said, "it actually sounds like a PM." So that's where, in the end, you end up saving a lot of time. TIME SAVED IS WORTH IT FOR PMs That's why I recommend ChatPRD. And also worked with the team to build an ultimate guide to using it: https://lnkd.in/e6nsDWKt As a 3x CPO herself, Claire Vo understands that for PMs, time saved writing PRDs is: • Time available for building relationships • The option to talk to more customers • Or simply not burning out AI isn't replacing PMs anytime soon. But PMs who embrace AI will certainly replace those who don't.

628

57 Comments

7 RECOMMENDATIONS FOR HARDWARE STARTUPS It’s hard to understate the difficulty of the v1 launch of a consumer hardware company. I see a ton of new HW startups coming out. Here are some counterintuitive recommendations… 1) Target a small market for v1, not a large market. You need a small number of people to love your product and this will earn you the right to build a v2. You have to understand your v1 will not make you a successful company. Your v1 needs to prove that you deserve to exist in the first place. In the case of WHOOP, we started with pro athletes who were willing to put up with our v1’s short battery life and flaky Bluetooth connections because they so desperately needed Sleep and Recovery data. A broad consumer base would have rejected it. 2) Be great at no more than 1-3 things. If you try to be great at more than that, you will build a hardware that is mediocre at a wide range of things. You won’t think it’s going to be mediocre, but it will be. 3) You want a small launch. Not because you want small sales but because you want to have a steady ramp. It’s much better to do 100k of sales steadily over 12 months than in 2 months with a huge drop off. Keep in mind you need to manage your manufacturing partner and that there’s nothing better or more efficient than steady and predictable. 4) CAC is your enemy. Referrals are gold. The magic of referrals is that they signal high NPS and they help you grow steadily. Again everything that helps you have a steady HW manufacturing plan is good. Building a referral program for v1 launch is a good use of your time. 5) Be excellent at Software. Turns out being good at both Hardware and Software is a hard thing and something that your BIG COMPANY competitor is actually not great at (excluding Apple here). For some reason, there’s such a magnifying glass on the Hardware development, that Software is then overlooked by consumer startups. Don’t make that mistake. You can be great at both. 6) You should spend an unreasonable amount of time and money on ONE aspect of your Hardware. Could be the design or a technical breakthrough. But something needs to be SPECIAL. In our case, we designed a modular battery pack that allowed Whoop to be charged without taking it off your body. This made our development timeline much longer, but it allowed us to achieve something that no other wearable has achieved since: 24/7 health data. 7) Finally, have an unrealistic belief in yourself and your team. You are not supposed to succeed and the odds are stacked against you. Building hardware takes confidence and you’re up against big players who will rip you off. We went so far as to leave a message for our competitors on our circuit board (pictured below). In the case of Amazon and others, the prophecy came true. If you’re building consumer hardware, I wish you nothing but success. It will be super hard, but super rewarding. Good luck 👊🏼 #consumer #hardware #wearable #startup #whoop

4,788

214 Comments