DoubleLoop

Most teams struggle with predicting the impact of future bets because it’s too complex and labor-intensive. As a result, they miss out on continuously growing their impact through data-driven learning loops. So I'm trying to figure out a lightweight workflow for teams to simulate the quantitative impact of their future bets. To be practical, the workflow must be conceptually sound while not requiring an onerous amount of data collection or ad hoc data science. The attached gif shows a tool prototype I'm playing with to power this workflow. Here's how I'm thinking this works: (1) Start by building an algebraic KPI tree for your business—this simplifies the impact of various factors into a clear model. An algebraic KPI tree breaks down your primary metric (could be revenue or a customer-oriented north star) into logical components (e.g., Revenue = Visitors * Revenue per visitor). (At DoubleLoop we have AI that helps with fast creation of algebraic KPI trees.) Note: algebraic KPI trees are a good place to start because the relationships are deterministic. While some teams want to create probabilistic models with soft influencer relationships between metrics, it requires more data science resources to get insight from these models. We're working on making this easier with DoubleLoop. (2) For a future period of work (e.g., Q1 2025) plug baseline values into the KPI tree. You could use a previous period's values or just use your judgment to pick something reasonable. It doesn't need to be perfect. (3) Based on the above, you can immediately do sensitivity analysis on the KPI tree to see where 1% changes to metrics will have the highest impact on your primary metric. This helps inform which levers to target with your bets. (4) Add your planned future bets to the canvas and connect each one to the input KPI you think that bet will influence. (5) Add other factors to the KPI tree; e.g., holidays, seasonal influences, or anything external that might impact your metrics. (6) At each connector between bet/factor and KPI, estimate how much you think that bet/factor will change the metric with a percentage. For example, a marketing campaign might both increase the # of new visitors and decrease conversion given lower intent. (7) Based on the formulas of the KPI tree, you will now be able to see the total predicted impact to your primary KPI across your whole portfolio of bets. (8) You will also have a framework to quantify the impact of each of your bets, even when external factors add noise. For example, sales might be down YoY, but you could still show how your bets had a positive impact in the face of headwinds. The first time you try this, your predictions will probably be far off. Your goal is to make better predictions with each cycle. The is unlimited potential to make your predictions more accurate, but this shouldn't stop you from getting started. Would you want to try this workflow for simulating bet impact? Why or why not?

We've been user testing our new AI agent for North Star workshops. Encouraged to be getting feedback like this: "Took a previous 5-6 hour (valuable) process down to 45 minutes." "It revolutionizes the approach to the workshops. Once it's framed, you don't have to be as nuanced of a facilitator. You're not going to be constantly coralling people because it's doing that for you." "I'm re-imagining the the data workshop we just had. In 5 minutes [with the North Star AI] we got to a better place than what we did in 2 days." "It's pretty fun to be honest. I didn't expect it to give me any ideas that I haven't thought about before, but it did." You can watch demo vid of it here: https://lnkd.in/gcsPww-w Not entirely sure how we're going to bring this to market yet, but I'm excited with where this going.

Looking backward to examine the outcomes of our previous work is vital for learning. A desire to help teams look backward more effectively was the origin of DoubleLoop. But, of course, the reason why we look backward is to learn how to make better decisions moving forward. At DoubleLoop I think we've neglected providing teams tools more acutely designed for estimating the impact of future bets. So I've prototyped a tool for running simulations in DoubleLoop to help teams predict/compare the impact of future bets to inform prioritization and goals. We're starting pretty simple with simulations based on algebraic KPI trees. See the attached screenshot or play with the functional prototype with Replit here: https://lnkd.in/gepbfgXy I'm thinking a KPI tree simulation tool like this could be the missing link in how many data teams can make metrics more actionable to the rest of the business. What do you think? Here's how it works: 1. Use DoubleLoop’s AI agent to generate a KPI tree tailored to your product or initiative, or build your own from scratch. 2. Experiment with potential targets for input metrics based on future bets. Set the baselines based on your historical data or just make your best guess. 3. Instantly see how changes to input metrics cascade through the tree, revealing their impact on your primary metric. Note: While we're starting simple with determistic algebraic relationships, the system will evolve to: - expand to incorporate probabilistic metric relationships, and - go beyond KPI trees to other models like growth loops.

Product/eng leaders - Imagine you had an AI assistant that every week analyzes all of your work data (e.g., Jira data). What question would you want this AI assistant to answer?

Marketplace builders: How do you like "Scaled Marketplace Liquidity" as a North Star metric? Scaled Marketplace Liquidity is a measure of a marketplace's ability to efficiently match supply and demand at scale, combining the total economic activity (gross marketplace sales) with the success rate of transactions (match rate). Scaled Marketplace Liquidity = Gross Marketplace Sales * Match rate Match Rate = Successful Transactions / Total Transaction Attempts While some focus solely on Match Rate to measure liquidity, we multiply it by Gross Marketplace Sales to incorporate the concept of scale; I.e., having a shrinking marketplace with a high match rate is not the goal. See the full KPI tree here: https://lnkd.in/gFRasdkN Marketplaces have been one of the best domains for DoubleLoop given their complexity. We're exploring how this KPI tree structure can power an AI assistant that will: (1) customize the KPI tree to reflect the unique dynamics of your marketplace, and (2) suggest OKRs based on industry benchmarks, helping you focus on metrics that matter most for growth, efficiency, and winning the land grab. For example, "Transaction Speed" matters for real-time marketplaces (e.g., ride-hailing, food delivery, and gig economy platforms) but less for low-frequency, high-value marketplaces (e.g., real estate, B2B services, and collectibles)."

We're developing AI at DoubleLoop for identifying North Star metrics (and input metrics) for any product. I'm looking for a handful of folks to do user 30-min user testing sessions with us on Monday, November 11th. If you'd like to help out, please leave a comment or DM me. Here's what you'll get out of it: - A high-level overview of the North Star framework and how to operationalize it - Suggestions for North Star metrics and input metrics tailored to your product The main requirement for participation is that your company must be fairly well known. As part of the process, our AI performs an outside-in analysis of your company based on publicly available info.

Most companies have a "Value Gap" — a disconnect between employees' day-to-day work and the company’s overall value creation. When this happens, product teams struggle to prioritize, GTM teams miss the mark on positioning, and executives make investment decisions in the dark. Using analysis I did for Google Cloud with DoubleLoop's AI business mapping tools, my latest blog post explains how "Value Architecture" bridges the Value Gap. The post includes real example models for how a product delivers value for customers, monetizes that value, and creates long-term competitive advantage, using Hamilton Helmer's "7 Powers" concept. Check out the post: https://lnkd.in/gwXEExtx A few quick thanks: - Marchelle Varamini for suggesting the term "Value Architecture" as a way to summarize a lot of this stuff we do at DoubleLoop. - Sachin Agarwal 🤙🏽 for the opportunity to present at Google Cloud's internal PLG summit, which provoked this analysis. - John Cutler for some important feedback on the post.

For a long time, I had very naive ideas about prioritization. Here's what I've learned over the years. 1. Unless you are a tiny startup, your company is designed to do multiple high-priority things at once. The argument, "Well, if everything is high priority, nothing is high priority," almost never works (while rational). The 🐘 in the room is that the mix of high-priority things has changed, and the org design is no longer fit for purpose. What seems rational to you—oh, obviously that team is overwhelmed and we need to set priorities—is, behind the scenes, "Crap, we got the budget wrong for that group. Anyone want to contribute? No? Hmmm. Surely they are inefficient and can juggle more than one thing, right?" See below. 2. Most senior execs know that the plans will change. They know that by March, there will be a new set of priorities. So they focus on budgets, not priorities. So when you see lackluster strategies and vague priorities, remember that there is a whole other prioritization activity that centers around the allocation of money, how people get bonuses, very back-of-the-napkin revenue attribution models, etc. 3. Humans, in general, are not great at building a mental model for capacity. Capacity in software is hard. People aren't fungible. The architecture is what it is. You can't have huge teams. You are building the factory (to meet the needs of a strategy) AND the factory is building "the product." Oh, it isn't a factory: it is more like gardening. This is important because our mental models for what we are prioritizing (e.g., focus, capacity, typing, bandwidth, etc.) are often all over the place. 4. Most companies are not progressive when it comes to thoughtful re-orgs, org redesign, reallocating capacity, etc. The simple thinking is: "If we stop doing X, then we should let those people go because they fit in this cell on the spreadsheet." People too heavily couple the finance spreadsheets with one-row-per-person, and the (most often useless and made-up) % allocations to things. This means that basic things like having priorities and reducing work in progress are fraught with very justified fears around layoffs. To some, a pristine list of force-ranked priorities looks like heaven. FINALLY, we can make decisions. To others, it is NOT GOOD. This is why efforts to visualize WIP often turn into Game of Thrones. There's a silver lining with all this! If you are aware of these tendencies, you can figure out approaches to prioritization that can nudge the org in the right direction. Knowing that the theoretically pristine force-ranked list of efforts is likely not possible, you can devise other strategies like building more realistic models with finance and being more open to "everything in high priority" yet focusing on the actual conflicts/trade-off decisions, etc. And embrace products like Runway and DoubleLoop that engage people in tangible discussions about how $s flow into outcomes and then back into your core models.

Sometimes when we identify the Movable Metrics that Matter, we're able to target metrics that have a mathematical relationship with the business KPIs we want to influence. For example, increasing average order value increases sales when all things are equal. Other times, our efforts must use metrics that we hypothesize will have a positive influence on a business KPI, but we might be wrong. Most companies can't afford to allocate data scientists to analyze every theoretical connection between metrics, so we're often left placing bets in the dark. At DoubleLoop we've seen companies set OKRs for metrics that they don't know are *hurting* their business. To help with this, we've shipped a "Metric relationship analyzer" AI assistant in DoubleLoop. In the screenshot, the AI assistant has assessed whether reducing page load time has a positive impact on the conversion rate from product views to add-to-cart, using correlation analysis, regression analysis, and other methods. When sharing its findings, it provides the usual context around correlation ≠ causation. While with everything AI-related, the results should be scrutinized. We aim to make it more efficient for teams to feel confident that their bets are targeting metrics that will lead to the downstream impact they're striving for.

Working on an algebraic KPI tree template for eCommerce. I thought it would be straightforward to make it, but there were some puzzles. Any feedback? See the bottom of this post for the criteria. == eCommerce algebraic KPI tree == View in DoubleLoop: https://lnkd.in/gfzeKTWg > Digital Sales = RPV * Visits --> Revenue Per Visit (RPV) = AOV * CR from Visits to Purchases -----> Average Order Value (AOV) = AUR * UPT ---------> Average Unit Retail (AUR) ---------> Units per Transaction (UPT) -----> Conversion Rate (CR from Visits to Purchases) = CR from Visits to Product Views * CR from Product Views to Add to Cart * CR from Add to Cart to Checkout Page View * CR from Checkout Page View to Purchase ---------> CR from Visits to Product Views ---------> CR from Product Views to Add to Cart ---------> CR from Add to Cart to Checkout Page View ---------> CR from Checkout Page View to Purchase --> Visits = Direct + Referral + Organic Search + Paid Ads -----> Direct Visits -----> Referral Visits -----> Organic Search Visits -----> Paid Ads Visits == Criterial for a "good" algebraic KPI tree == * Every parent must be derived from its children with an algebraic equation. * No KPIs can be duplicated in the tree. * Every metric must be quantifiable with time series data to judge performance over time. * As you move down the tree, the KPIs should get more actionable; i.e., they should be possible to influence with product or marketing work. For context on how to use a KPI tree like this, see my last post: https://lnkd.in/gGX5pBUj