“If you are struggling with performance on the platform, the first answer is not to upsize your warehouse. The first answer is … how have you modeled your data?” - Patrick Cuba 💡 I spent a big portion of my career focusing on performance tuning and optimization of database systems. Throwing hardware at a performance problem might seem like a good solution, but it’s very expensive, and sometimes there's just no “server” that is big enough. 🏃♀️ A well-performing database system starts with a thoughtfully designed data model and ends with an understanding of how a data platform itself works under the hood. Check out this presentation from Patrick where he explains how the #Snowflake optimizer works and how it impacts #DataVault, as well as various other data modeling patterns. https://lnkd.in/eR-UCfNy ❗ Friends don't let friends throw hardware at a data performance problem.❗ Snowflake #DataVault #Datamodeling #DataModelingIsBeautiful
Data Vault Performance & Constraints on Snowflake
https://www.youtube.com/
In my recent experience, I've observed that the performance of a platform is closely linked to its cost optimization. In cloud workloads, data teams are increasingly adopting on-demand pipelines and utilizing resources as needed. However, the focus on modeling and optimizing data pipelines often becomes a priority only when costs escalate and there is a pressing need for cost-efficient processes.
Nicolas Manzo
5mo
It was great dropping into todays meeting! Thanks for sharing the link so I can revisit it!
Holistic, passionate and creative Data & AI Professional and Educator. When I can spread joy, when I can empower, I do. Otherwise, I remain silent. I embrace every day knowing that life is not a highly curated dataset.™
6moFor folks who never had to face the "maybe this query will finish executing by the time we're back from launch," this is hard to comprehend. The enormous tool ecosystem coupled with lack of proper foundations/fundamentals training have contributed to the state of affairs.