Sam Fleming

Meet Basil - LeavePlus' own Product Designer and Dad Joke extraordinaire.   Check out this video to hear how... → ...LeavePlus' improvements are guided by user insights. → ...larger organisations can still act like startups. → ...we rapidly transition from ideas to market. ...as a bonus, you'll also get one awful joke about Adele. #productdesign #transformation #startup

57

9 Comments

How much AB testing should you do? Booking.com believe you should do a LOT. “𝘐𝘧 𝘪𝘵 𝘤𝘢𝘯 𝘣𝘦 𝘢 𝘵𝘦𝘴𝘵, 𝘵𝘦𝘴𝘵 𝘪𝘵. 𝘐𝘧 𝘸𝘦 𝘤𝘢𝘯’𝘵 𝘵𝘦𝘴𝘵 𝘪𝘵, 𝘸𝘦 𝘱𝘳𝘰𝘣𝘢𝘣𝘭𝘺 𝘥𝘰𝘯’𝘵 𝘥𝘰 𝘪𝘵.” Stuart Frisby (ex Design Director) AB testing has faced a bit of backlash recently: • "Founder mode" / design-led approaches emphasising product vision • Acknowledgement of the cost (time + effort) of testing • Fears over incremental rather than step-change improvements • Recognition that results don't stack or often persist That said, Booking[.]com makes a compelling case for it: • 2-3x the industry conversion rate • Continued growth at >25% y-o-y And most companies will aim for the happy medium: Strong product vision + AB testing to learn and verify So there's a lot to learn from Booking[.]com's test-driven approach ... 𝗪𝗛𝗬 𝗧𝗘𝗦𝗧? • It builds an evidence based culture - some of the biggest wins are the most surprising • Spots pockets of value sooner - helps you focus on high impact areas • Harnesses the power of it’s teams - democratises decisions away from senior stakeholders • Prevents cost mistakes - you can measure exactly how changes perform 𝗛𝗢𝗪 𝗧𝗢 𝗧𝗘𝗦𝗧? 1. 𝗖𝗿𝗲𝗮𝘁𝗲 𝗮 𝗰𝗹𝗲𝗮𝗿 𝗵𝘆𝗽𝗼𝘁𝗵𝗲𝘀𝗶𝘀 𝘉𝘢𝘴𝘦𝘥 𝘰𝘯 [𝘦𝘷𝘪𝘥𝘦𝘯𝘤𝘦]. 𝘞𝘦 𝘣𝘦𝘭𝘪𝘦𝘷𝘦 [𝘟] 𝘸𝘪𝘭𝘭 𝘦𝘯𝘤𝘰𝘶𝘳𝘢𝘨𝘦 [𝘵𝘩𝘦𝘴𝘦 𝘶𝘴𝘦𝘳𝘴] 𝘵𝘰 [𝘤𝘩𝘢𝘯𝘨𝘦 𝘣𝘦𝘩𝘢𝘷𝘪𝘰𝘶𝘳]. 𝘞𝘦 𝘸𝘪𝘭𝘭 𝘬𝘯𝘰𝘸 𝘵𝘩𝘪𝘴 𝘸𝘩𝘦𝘯 𝘸𝘦 𝘴𝘦𝘦 [𝘦𝘧𝘧𝘦𝘤𝘵] 𝘩𝘢𝘱𝘱𝘦𝘯 𝘵𝘰 [𝘮𝘦𝘵𝘳𝘪𝘤]. 2. 𝗖𝗵𝗼𝗼𝘀𝗲 𝘁𝗵𝗲 𝗿𝗶𝗴𝗵𝘁 𝘀𝘂𝗰𝗰𝗲𝘀𝘀 𝗺𝗲𝘁𝗿𝗶𝗰𝘀 • Fully aligned with the outcome you want • Something you can measure • Easy to see impact in a short timeframe 3. 𝗨𝗻𝗱𝗲𝗿𝘀𝘁𝗮𝗻𝗱 𝘆𝗼𝘂𝗿 𝗲𝘅𝗽𝗲𝗿𝗶𝗺𝗲𝗻𝘁 𝗰𝗼𝗻𝘁𝗿𝗼𝗹𝘀 • Traffic split: how many users do you throw at the test and control variants • Confidence interval: 95% means  1 in 20 times you'll get this result by chance. A good start point, but you can flex this • Run time: how long it takes to reach your confidence interval. A function of confidence level and traffic 4. 𝗔𝗻𝗮𝗹𝘆𝘀𝗶𝘀 𝗮𝗻𝗱 𝗶𝘁𝗲𝗿𝗮𝘁𝗶𝗼𝗻 • Not every iteration will be successful • But you should learn with every test • So iterating and continued learning are critical 𝗙𝗢𝗨𝗡𝗗𝗔𝗧𝗜𝗢𝗡𝗔𝗟 𝗖𝗨𝗟𝗧𝗨𝗥𝗘 Booking[.]com run 1000s of tests a year That’s a volume that most tech companies can only dream of They’ve managed to achieve that success by: • Making experimentation a foundation of their culture • Always testing new features. Rigorously, at scale and cross-platform • They’ve invested significantly in hiring, training and tooling to support this For more details and worked examples check out the full article here by the awesome Evie Brockwell : https://lnkd.in/eirC28Uc

2

Product people always talk about failing fast, but what if we look at failure differently? 🤷 In today's episode of the Product Leaders Podcast, our host Ben Moulton sits down with Michelle Reiman Wright, Chief Product Officer at Kurtosys. 🎙️ They chat about the taboo topic of failure! Michelle's refreshing voice on the topic is a reminder of why failure is crucial for innovation and how to foster psychological safety within product teams. Tune in to gain valuable insights into the power of innovation and leveraging failure for growth. Head to the links in the comments to give it a listen 🔗 #weareorbisgroup #podcast #productleaders

44

2 Comments

How much impact has your product team had? 🤷‍♂️ That’s a surprisingly tricky question to answer. Sure, AB tests are great … … assuming you’ve got the traffic ... ... and you’re measuring a single feature. But AB tests aren’t great for measuring performance across multiple initiatives or several teams. AB test results do NOT stack up. • 5x 2% uplift in AB tests does not lead to a 10% increase overall. • Double counting is difficult to exclude. So how to we attribute product impact? Some thoughts below, but first: • Why do you need to know? • What decisions will this inform? It’s easy to get sucked into a laborious process of attribution seeking the Truth. In most cases, an approximation will do. Seeking a perfect answer vs. a "good enough" answer is busywork. 𝟱 𝗟𝗘𝗩𝗘𝗟𝗦 𝗢𝗙 𝗔𝗧𝗧𝗥𝗜𝗕𝗨𝗧𝗜𝗢𝗡 Ideally you build up analysis linking leading product metrics in your control to lagging business critical metrics like revenue. This is done in stages: 𝟭 - 𝗩𝗜𝗕𝗘𝗦 You don’t attribute. “It’s a team sport, who cares?” If you’re very early stage and things are going well, do you even need to attribute? 𝟮 - 𝗡𝗔𝗥𝗥𝗔𝗧𝗜𝗩𝗘 You can describe the mechanism, but not put numbers to it. “Commenting drives retention, retention drives revenue” 𝟯 - 𝗠𝗘𝗧𝗥𝗜𝗖𝗦 You have a driver tree with specific metrics, but you don’t know how much moving one moves another. “# comments a day drives D30 retention. D30 retention drives revenue per active user” 𝟰 - 𝗖𝗢𝗥𝗥𝗘𝗟𝗔𝗧𝗜𝗢𝗡 Most companies aim here. You know the link between your metrics. “10% increase in comments a day equals 3% increase in D30 retention, and 2% increase in revenue per active user” 𝟱 - 𝗛𝗢𝗟𝗗 𝗢𝗨𝗧 𝗚𝗥𝗢𝗨𝗣 You hold back all features from a subset of users for several months. (Like a prolonged AB test) This is very expensive, but very rigorous. 𝗚𝗘𝗡𝗘𝗥𝗔𝗟 𝗧𝗜𝗣𝗦 • Work with an analyst / the CFO to come up with a simple model that everyone can buy into. • It's more important to have political buy-in than an absolutely scientifically correct answer. • Standardise the baseline so you can compare across teams and initiatives (e.g. last year’s actual revenue / traffic) • Be conscious of when you’ll actually release features, and how many months contribution they’ll deliver this year. Often shipping something in Q1 means you'll see impact from Q2. • Agree how you'll count future years. Product changes are often small but permanent, affecting all future revenue. More on different types of quantitative testing here on Hustle Badger: https://lnkd.in/eSaJ9jt8

2

It's finally here... Your own Product Marketing Sidekick inside Miro. 🥷 Taking the infinite wisdom of our PMM-GPT, you can access this in real-time for instant feedback as you're planning your next product launch, market research project, or setting 2025 OKRs. (It's pretty cool if we say so ourselves). You never need to be an island again. 🏝️ Kudos to the team at Miro who have helped us incorporate this and been working hard over the last few months. 👏 If you use Miro, go check it out and give us lots of feedback on what you think. https://lnkd.in/eJFbY657

47

4 Comments

🚨Episode 10 of the Crave Connection In Product podcast is LIVE! In this episode, we are joined by Áine McKay, Head of Product at AutogenAI. 🚩 Áine recently introduced the topic of ‘The evolution of the Product Trio’ at our In Product Meetup. 🫶This podcast focuses on the emergence of new roles, practices, and value drivers due to the rapid expansion of Generative AI. Sharing her firsthand experience and learnings, Áine will also share some predictions about the future when integrating AI into established practices. In this episode, Aine brings her opinion and insight from a Product Leadership perspective and discusses 👇 ✅The blending of roles in the Product Trio. ✅Developing skills, and navigating career progression in an increasingly hybrid landscape. ✅The introduction of new roles to the Product Trio and specifically Prompt Engineering. We hope you leave this episode with an insight into the traditional Product Trio, and insights from the conversation as to how this has evolved.  You can listen in or watch the full episode through the below links: 🎙️ https://lnkd.in/e5m9v_MU 🍿 https://lnkd.in/eWbawKut #podcast #ai #ml #productmanagement #productcommunity #productrio

18

🌟 Your Competitive Edge! The 2024 #SaaS Product Metrics Benchmark Report is out! Learn from the best and see how your #productmetrics compare. Get expert advice to enhance your strategy: https://lnkd.in/eqQdDjsg

4

If you missed our Crafting Effective OKRs for Product Teams webinar, you can now watch a snippet here and join us next time for another amazing session. Product NBO Serah Lois Wawira

10

Love this framework from Marcus Andrews. 🦑⚡ The Jellyfish Product Marketing model: "A simple, visual way to think about and remember how great product marketing works. 1. How do Positioning > Story > Marketing tactics fit together 2. How great product marketing work comes together. Great product marketing isn't just made up. It's built on top of great positioning. And the marketing tactics that bring it to life erupt from a single, exciting narrative." 👋 Tag a Product Marketer you think would like this! Interesting? ♻ Repost to share. P.S. Make sure to give Marcus a follow. #productmarketing

760

42 Comments

Too often we're asked to execute an X....(email, social post, web page, etc...). The tactic only works when you put in the time to inform to effort and create the story. #productmarketing #marketingprocess

3

Here we are for round #2 with Onset’s Product & Design consultant, Alex Robertson, as he dives into the challenges businesses face when hiring in-demand talent and the key skills needed for Product Managers looking to step up. Alex shares his insights on: 🔍 Navigating hiring challenges in a competitive market 📈 Identifying and nurturing future-proof skills 💼 Adapting to evolving industry demands Connect with Alex for more insights and stay informed with the latest trends in Product 🎙️ #TechRecruitment #Careers #CareerAdvice #Product #ProductDesign #ProductManagement #CareerDevelopment

19

Why is collaboration so important for engineering teams? Marcel Weekes @ Figma sums it up with this answer from his #ELCAnnual2024 session. “Ultimately, we’re all trying to do the same thing,” he says. “We’re trying to ship the best product we can as quickly as we can.” With shared goals in mind, working together is the only thing that makes sense. Listen to the podcast for the full session with Marcel Weekes & Arquay Harris below. 🎙 https://lnkd.in/gBBHPfc9 #EngineeringLeadership #ELC #ELCAnnual2024 #Figma

4

Levelling up your product from demo viable to operational viable is tough. You're hammered from all sides, but once you get past it, your product is more robust. Key lesson: #staycalm, avoid emotional reactions, and focus on the hurdles. If you are a founder, this video is for you!! We would love to hear your stories of growing your business. Watch now: https://lnkd.in/dR-2driF Ryan Purvis #Founder #StartupChallenges #ProductDevelopment #BusinessGrowth #Innovation #valuuai #ai #startups #OKRs

8

Thanks to the Intelligent People team for keeping this amazing schema running. Working with a diverse palette of product leaders over the past year has been extremely enriching. I'm eagerly looking forward to continuing this journey and meeting more inspiring leaders! Are you a product person (IC, leaders...) looking for some support, a sparring partner or just an external view to help you in your job or product career? The Intelligent People Mentorship program hosts a large number of top-notch CPOs who're ready to help ! #productmanagement #mentorship #productleadership

3

🤔 Is your team struggling with role clarity? 🚀 At Product People, we've tackled this challenge head-on, helping Product teams redefine their roles for maximum impact. ⬇️ Swipe to read some key insights to help you do the same! #TeamGrowth #ProductLeadership #Collaboration

6

Product Sphere members are getting more interviews than before. How are they doing it? It starts with a well optimised CV! Members improve their CV with video feedback, coaching, and collaboration on content. Cold applications aren't dead. They require a tightly optimised CV to convert. If you're not achieving at least a 5% application conversion rate, it's probably your CV. Some of our members are seeing 10% or greater conversion rate with their CVs after working with our Founder who has reviewed 1000's of CVs as a Product Leader, Hiring Manager, and Career Coach.

23

1 Comment

NEW EPISODE 📢 Kevin Gentry chats with the amazing ✨Jason Fraser✨, a well-known author and advisor, about his journey from linguistics to product management. Grab your cup of ☕️ and listen now! #podcast #productmanager #productmanagement

2

Have you been trying to drive more impact in your product org, but it's just not working? We're joined by impact-guru Matt LeMay on our latest episode. Matt unpacks exactly what product organisations need to change to drive more impact, and why this is so important, & why it's not about having dead certainty on your goals. He also throws in so many gems: 💎 What are soft skills in product, and why are they important 💎How can you detach yourself from your work and stay sane 💎Approaching conversations about the uncomfortable thing

9