James McHenry

Why historical data is not enough for Logistics? Just as humans are inherently unpredictable, AI models are never completely predictable. The world throws curveballs and just relying on historical data to predict future is not enough. For new generational models, we need to accept the challenge of adaptability and generalization. Why? ... Imagine training a chess program to be a champion - it may excel under standard rules, but if the game's rules or the board change slightly, you would notice that it may struggle to adapt. Another example: Remember in 2021, the Ever Given (a.k.a. EVERGREEN) container ship (image below) created a logistics nightmare by blocking the Suez Canal for nearly a week. Many business lost orders due to delays and late shipments. This disruption exposed a critical flaw in many supply chain systems: an over-reliance on historical data. Many logistics algorithms were trained on "business as usual" data, leaving them ill-equipped for such an anomaly. The Ever Given incident highlighted the need for algorithms that can handle the unexpected. In AI world, it was a classic case of overfitting. When reality deviates from the trained "norm", the overfitted models break down like a chess engine that underperforms when the rules change. Here are some ways to improve AI model robustness: ° Consider potential failure scenarios during training with test cases and training data that addresses these weaknesses. ° Generate synthetic data for failure scenarios. ° Provide transparency to stakeholders through clear metrics (like confidence score etc.) so that they understand the model's limitations and avoid over-reliance ° Implement techniques like cross-validation and regularization with your data scientists ° Implement mechanisms for human oversight. Trust me you can't go wrong with this one :-) Please share what else would you do to avoid this over-reliance on historical data. #supplychain #logistics #aiml #overfitting #data #aiengineering #predictions Photo Credit: Suez Canal Authority Via AP

17

1 Comment

Suresh Kumar, Global CTO for Walmart -> “A standard search bar is no longer the fastest path to purchase. We must use technology to adapt to customers’ individual preferences and needs.” Walmart’s AI platform strategy aligns AI products with use cases and workflows to support internal user and customer needs. First-party data is positioned as a core competitive advantage. “At the heart of our platform strategy is developing core capabilities built once and deployed across Walmart." If the AI platform doesn’t align with the business and operating models, it won’t succeed. Data and AI strategy must enforce that alignment or value never materializes. Goldman, Meta, SAP, Salesforce, and now Walmart have moved their AI platforms in the same direction. https://lnkd.in/gz6Bv3ai #AIStrategy #DataStrategy

64

4 Comments

📊 ARIMA for Smarter Forecasting in Retail 📊 In retail, accurate forecasting can transform decision-making. ARIMA (AutoRegressive Integrated Moving Average) is a powerful statistical ML model that helps analyze time series data, making it easier to identify patterns and project future trends. ARIMA is especially useful for: Inventory Planning: Anticipate stock needs based on historical demand patterns. Demand Forecasting: Capture seasonal trends and forecast demand shifts. Staff Scheduling: Predict peak times to optimize staff allocation. Promotion Planning: Align promotions with high-traffic periods. ARIMA’s approach to time series forecasting offers retailers a data-driven way to manage resources effectively. Learn how to make the most of it here 👉 https://lnkd.in/gmr5hTYP #ARIMA, #DemandForecasting, #MachineLearning, #RetailAI

4

Have you tried Snowflake's Artic? Artic is the latest open sourced LLM launched by Snowflake. It's amazing to observe the innovation driving training and operating efficiency for LLMs. Some Specs about Artic: - Dense-MoE Hybrid transformer architecture - 480B parameters spread across 128 fine-grained experts - 17B Active Parameters - Apache 2.0 licensed The main use cases of Arctic are SQL, coding, and instruction following for enterprise applications. Generating SQL code presents an important challenge, demanding a deep understanding of both natural language and programming syntax. It involves crafting precise outputs that faithfully reflect the user's intentions. Artic is available on Hugging Face, Together AI, Microsoft Azure, NVIDIA API Catalog and Replicate. It's also coming soon on Amazon Web Services (AWS), Lamini and Perplexity.

26

6 Comments

The Information has a great write-up about the intense battle for foundational data storage companies namely Snowflake and Databricks and who really has the pole position. It is a great read for a part of the technology stack that frequently does not get the headlines they deserve. What do you think? #mangement #venturecapital #privateequity #cloudcomputing #capitalmarkets

1

A few words about forecasting ... Effective forecasting is essential for strategic planning and decision-making across industries. Accurate forecasting requires analyzing historical data, identifying trends, and using predictive models to anticipate future outcomes. A good approach often combines quantitative methods like time series analysis and qualitative methods such as expert judgment. Flexibility is also key—forecasts should be regularly updated to reflect new data and changing market conditions. According to research, organizations that integrate data analytics into their forecasting processes are 33% more likely to achieve financial targets​ (Hometown Locator). By leveraging these strategies, companies can improve decision-making, optimize resources, and mitigate risks effectively.

1

The proof is always in the pudding. While many players claim 'talk to your data' capabilities, very few actually deliver. 🤷‍♂️ We recently put Lumi AI up against a notable vendor in a brutally honest, simple, and fair head-to-head comparison: same dataset, same questions. The difference in output quality and accuracy was stark. Lumi: 95% Notable Vendor: 33% If you're searching for a solution that truly helps you uncover insights hidden in your operational datasets, Lumi is quantifiably best in class. Link to full results: https://lnkd.in/dK7JbHeZ

40

2 Comments

I read this great blog post by Mikhail S., if you are interested in how to price your products using Data Science here you go: https://lnkd.in/eVtwenHJ

2

There are rumors coming out of Snowflake Summit this week that Snowflake’s launching a semantic layer. Standardizing key metrics like revenue is crucial for streamlining analytics and reporting. DBT and Snowflake's semantic layer products will make this concept mainstream. But metrics aren't the only thing companies lack a standard definition for. Companies care about customers (hopefully) but often don't have a consistent customer definition. Consider a B2C company with customer interactions across multiple channels: 1. Anonymous (guest) visits from multiple devices 2. Logged-in users 3. Physical store interactions Are these interactions considered as one customer or multiple? Is there alignment on this definition between different teams? Is product measuring the funnels for the same set of customers that marketing is targeting with email campaigns? Is data science building life-time-value or churn models for the same customers who are receiving marketing emails? Without a consistent customer definition, teams can't have a shared understanding of the customer's journey. This results in a disconnected experience across product and marketing channels. There's a pressing need for a customer 360 semantic layer. That's what we're building with RudderStack Profiles.

132

1 Comment

Data lineage is one of the most overlooked components in building reliable AI—here’s why: Lineage is the foundation of data observability. ▪️ It’s the first thing we check when something breaks. ▪️ It maps our complex data pipelines. ▪️ It connects upstream data to real business outcomes. But without it, we’re blind. Today, getting a full lineage view is tough. It’s mostly solved for SQL— where tools can automatically map SQL logs to lineage. But the world is bigger than SQL. What about streaming pipelines, raw code, upstream systems? Stitching these together is endless work. With AI, data pipelines only get more complex, moving far beyond SQL. And without a lineage solution, we’re in the dark. To build reliable AI, we have to solve this. 

47

5 Comments

Your data isn't valuable if you can't centralize it and look at the big picture. That's where the data warehouse comes in and how Mozart Data helps our customers get to insights quickly. See how you can deploy a data warehouse without a data engineer: https://lnkd.in/gCTqDMtD #snowflake #fivetran #datawarehouse #dataengineer

13

1 Comment

I've been in the syndication space for a while, and understanding why syndication is so difficult is important to understanding how your company can get better at product syndication. Educating yourself can avoid choosing the wrong tool or strategy that can cost you sales and resources. Reach out to Trailbreakers.ai if you want an analysis of your Syndication needs.

6

Exciting news! Sigma has integrated with Glean, making it easier than ever to find and leverage your most valuable data. With this powerful new integration, you can search for datasets, dashboards, and insights within Sigma right from Glean’s intuitive search experience. 🚀 Streamline your data workflows and get the insights you need faster. Check out how this integration is taking data discovery to the next level!

20

1 Comment

Looking to deploy predictive models faster, more accurately, and with greater transparency? With Causify you can: • Deploy models in 50-80% less time, even with data teams of just 1-2 people. • Achieve 30-50% higher prediction accuracy. • Leverage causal inference for actionable insights. • Build probablistic AI solutions with transparent reasoning streams, ensuring your predictions are not only accurate but also explainable and trustworthy. Our automated MLOps platform simplifies the entire machine learning pipeline, empowering teams to focus on delivering real value without getting stuck in complexity. Want to learn more? If you’re at AWS re:Invent 2024, let’s connect on the expo floor, or drop me a message. I’d love to show you how Causify is transforming predictive modeling and decision-making for businesses. #MLOps #CausalInference #ExplainableAI #ProhibitiveAI #PredictiveAnalytics #MachineLearning #DataScience #AWSreInvent

17

4 Comments

🔥 David vs Goliath: Sulie's foundation model crushes retail demand forecasting (and yes, beats Nixtla's TimeGPT in accuracy). Exciting news from our ML kitchen! We've just wrapped up a juicy comparison between our time series foundation model Mimosa and Nixtla's TimeGPT. And spoiler alert - sometimes the underdog has a few tricks up its sleeve! 📊 Let's dive into the raw data: * Battle arena: Nixtla's own retail dataset (we love fair fights!) * 50 products in the ring (25 bestsellers, 25... not so best sellers) * 7-day forecast horizon * Zero fine-tuning - because real models don't need makeup 💅 **Update** The Results? Our Mimosa served up a crisp median MAE of 13.77, while TimeGPT landed at 15.56. Not exactly a photo finish, is it? But here comes the humbling part... 🐌 TimeGPT absolutely smokes us in the speed department - beating us by 700ms. I guess while we're still doing our calculations, they're already planning their next forecast. 🥁 Stay tuned for a more comprehensive benchmark in the following week or two, across more datasets and with more foundation models. 🚨 A quick disclaimer: the comparison we posted was by no means a comprehensive benchmark or study. It was more of an early indication of our model’s performance, and we’re working on a more thorough analysis. 📚 Dataset: https://lnkd.in/dHZ_XXzY 🐍 Benchmark notebook: https://lnkd.in/djbHDMFf 🌐 Website: https://sulie.co

59

15 Comments

🤖 Analytics Agents in Snowflake ❄ Product Managers are incredibly busy, the last thing they want to do is pull up more charts to analyze more metrics to create more decks. In the demo video, Aryan shows how: 👉 An AI agent in Snowflake can breakdown rather nuanced questions into simpler instructions and pass them on to SQL agents. 👉 These SQL agents then use Snowflake APIs as "tools" to generate the relevant charts and pass them on to a "summarizer agent". 👉 Orchestrating multiple such agents is now possible with Langgraph, something we’ve used to power this execution pattern.   We’ll be open-sourcing this repo soon, and in the future, we’ll be baking in learning capabilities for these systems. This would enable end users and AI engineers to feed selective feedback on individual agents and improve their robustness. Imagine. A future where multiple agents literally live inside my data warehouse, roaming around, fetching insights, surfacing them to the relevant humans! 🔜 More agents coming soon! Dhruv Abiola Aryan Houseware

62

14 Comments

In the race to deploy AI, I'm increasingly seeing leaders ask for AI without fully understanding the need to integrate disparate data sources. This is a critical step—without it, even the most advanced AI won’t deliver meaningful ROI. Data management vendors seem to get it...highlighted by last week's acquisition of Datavolo by Snowflake (along with their partnership with Anthropic). While AI via LLMs can be sufficient for individuals (e.g., “write an itinerary for my 5-day St. Lucia vacation”), businesses require much more. AI for enterprises isn’t just about generating responses (e.g. chatbots, co-pilots)...it’s about optimizing complex processes, like a 300-person P&C Claims handling operation (which is a real use case at Base64), by connecting workflows, automating decision-making, and leveraging real-time insights from diverse data sources (i.e. content reasoning). If your AI POC doesn’t start with a clear strategy to address disparate data, you’re not setting up for success. AI is a transformative tool, but only when the foundation is built to support it. #AI #DigitalTransformation #LLMs #DataIntegration #EnterpriseAI

28

6 Comments

With catalogs being such a hot topic, I wrote this quick article highlighting some key points I review almost every week and the slide I use to show it! Check it out and hope it helps! https://lnkd.in/e6wKnzy3 #iceberg #datacatalog #MDM

29

Today, I want to dive into a topic that is at the core of our business success - the power of data. The power of data is what ultimately drove me to start OptiFunder in the first place. The mass amounts of warehouse and loan data were not being effectively consumed and analyzed by #IMB’s to make the most optimal funding decisions. In today's digital age, data has become more than just numbers and statistics. It is an asset that holds the key to unlocking insights, making informed decisions, and driving innovation. As a #SAAS company, we rely on data to understand our customers, improve our products, and stay ahead of the competition. Data (the right data) allows us to: 1. Gain valuable insights into customer behavior and preferences 2. Identify trends and patterns that can inform strategic decisions 3. Measure the success of our initiatives and campaigns 4. Personalize the customer experience and drive engagement 5. Optimize our processes and drive efficiency I encourage you to harness the power of data. It is important to understand its significance and leverage it effectively in your day-to-day operations. Whether you are in product development, marketing, sales, or customer support, data plays a crucial role in driving success. Data is not just numbers on a spreadsheet - it is a powerful tool that can help you achieve your goals.

20

1 Comment