Christian Minich

EK's Lulit Tesfaye and Jeff Jonas of Senzing teamed up on EK's latest blog, in which they discuss typical problems that would necessitate Entity Resolution, how to unify enterprise data with a Semantic Layer, and the role of entity-resolved knowledge graphs. #semanticlayer #knowledgegraph #entityresolution https://lnkd.in/ePujrbve

9

What's in a name? Falko Schulz creates a SAS Visual Analytics infographic on the popular names in the US collected by the Social Security Administration from 1920 to 2022. There's even a link to the interactive report if you want to delve into the data. #infographic #dataviz #SAS #SASviya #SASusers #SAStip #data https://lnkd.in/guhBVUPU

22

1 Comment

Want to learn more about how to get started using Snowflake Cortex LLM Functions? Check out this post from Jon Coulter detailing how to use these tools to quickly classify data, empowering Data Engineers with Data Science tools! https://lnkd.in/gMTFT5xQ

14

Are you interested in... - Data viz tepid takes? - Slightly humorous example graphs? - Pie chart hate? If you answered "yes" (or even "maybe") to any of the above, then you should check out my latest GovLoop article on data viz!

6

How does an organization benefit from Strategic Data Modeling? 💥 Consensus and visibility of the broader business model 💥More productive conversations with business teams 💥Better quality of requirements 💥Higher signal, lower noise in technical conversations 💥Cross-platform, cross-domain, and widely understood conventions 💥Big-picture visual overview of the business and its database footprint 💥Preliminary designs become implementation blueprints 💥Accelerating onboarding of new team members 💥Making data more accessible and unlocking self-service within organizations 💥Keeping the database landscape manageable at scale 💥Getting a handle on complex data pipelines Check out Serge Gershkovich's book, 𝐷𝑎𝑡𝑎 𝑀𝑜𝑑𝑒𝑙𝑖𝑛𝑔 𝑤𝑖𝑡ℎ 𝑆𝑛𝑜𝑤𝑓𝑙𝑎𝑘𝑒, to learn more! #DataModeling #SqlDBM #Cloud #Snowflake https://lnkd.in/gsQsJ_yb

14

1 Comment

Databricks Apps is in public preview, and it's awesome. Here are 5 things we love about it...and why 👇 𝗨𝗻𝗽𝗮𝗿𝗮𝗹𝗹𝗲𝗹𝗲𝗱 𝗗𝗲𝘃𝗲𝗹𝗼𝗽𝗺𝗲𝗻𝘁 𝗦𝗶𝗺𝗽𝗹𝗶𝗰𝗶𝘁𝘆 • Leverage familiar Python frameworks: Build with Dash, Shiny, Gradio, Streamlit, and Flask. • Seamless IDE integration: Develop in Visual Studio Code or PyCharm, your choice. • Pre-built templates: Jumpstart your app development with ready-to-use Python templates. • Local or cloud development: Work in your preferred environment, then deploy effortlessly. 𝗖𝘂𝘁𝘁𝗶𝗻𝗴-𝗘𝗱𝗴𝗲 𝗔𝘂𝘁𝗼𝗺𝗮𝘁𝗲𝗱 𝗗𝗲𝗽𝗹𝗼𝘆𝗺𝗲𝗻𝘁 • Serverless compute: Apps run on auto-provisioned infrastructure, eliminating manual setup. • Git integration: Benefit from version control for collaborative, trackable development. • CI/CD pipeline support: Streamline your workflow with integrated continuous integration and deployment. • Scalability: Automatically handle varying workloads without manual intervention. 𝗘𝗻𝘁𝗲𝗿𝗽𝗿𝗶𝘀𝗲-𝗚𝗿𝗮𝗱𝗲 𝗚𝗼𝘃𝗲𝗿𝗻𝗮𝗻𝗰𝗲 𝗮𝗻𝗱 𝗦𝗲𝗰𝘂𝗿𝗶𝘁𝘆 • Unity Catalog integration: Centralized data governance across your entire Databricks ecosystem. • Granular access controls: Ensure data is accessed only by authorized personnel. • Automated service principals: Simplify and secure app-to-service authentication. • SSO and OIDC/OAuth 2.0: Robust user authentication mechanisms built-in. 𝗦𝗲𝗮𝗺𝗹𝗲𝘀𝘀 𝗣𝗹𝗮𝘁𝗳𝗼𝗿𝗺 𝗜𝗻𝘁𝗲𝗴𝗿𝗮𝘁𝗶𝗼𝗻 • Access to Databricks SQL: Harness the power of SQL queries directly within your apps. • AI model serving: Integrate machine learning models seamlessly into your applications. • ETL job integration: Connect your apps to data pipelines for real-time insights. • Unified data access: Leverage Delta Lake, data warehouses, and data lakes effortlessly. 𝗘𝗰𝗼𝘀𝘆𝘀𝘁𝗲𝗺 𝗘𝘅𝗽𝗮𝗻𝘀𝗶𝗼𝗻 𝗮𝗻𝗱 𝗩𝗮𝗹𝘂𝗲 𝗖𝗿𝗲𝗮𝘁𝗶𝗼𝗻 • Databricks Marketplace: Partners can now build, package, and sell data products and AI assets. • Self-service analytics: Empower business users with custom, governed data applications. • Rapid prototyping: Quickly iterate on ideas and bring concepts to life. • Cross-functional collaboration: Bridge the gap between data teams and business stakeholders. Yes. We partnered up with Databricks. But we didn't 𝗵𝗮𝘃𝗲 to. Innovations like Databricks Apps make us just 𝘄𝗮𝗻𝘁 to. 🙂

14

🚨CODE DROP🚨 I created a Baseball Savant data downloader that fetches and processes Statcast data from Baseball Savant into a Polars DataFrame. It handles retries, uses concurrent requests for efficiency, and filters out empty rows. https://lnkd.in/g7MhNNCQ

36

This week on the blog, Ami Ikanovic discusses the "DIY is better" mentality for #datamasking and how it might seem like a quick win initially, but the long-term costs – time, complexity, and risk – are too high. https://lnkd.in/eUXyXKQf #datasecurity #datasecops #datagovernance #Snowflake

19

Have you ever wondered about the differences between Datafold's Data Diff and #dbt tests? Do data teams really need both? In this post, Elliot G. and Leo Folsom explain: ❓ Why dbt tests prevent some data quality issues, but not all 🤔 How the two tests answer fundamentally different data quality questions 🤝 And why having both in your CI pipeline is essential for complete data quality coverage https://lnkd.in/ggmC97hn

13

1 Comment

108 Recipies Snowflake Build: Just moments ago (Nov 12th, 12:30 PM CST), SF presented the Snowflake Dynamic Iceberg table using Polaris External CatalogSync." We have 20+ recipes in this e-book. Below is how to build summary tables as new data streaming into the base table. Snowflake

6

There was a whole lot of talk about GenAI at the Snowflake and Databricks Summits this year, but don't be fooled – the star of both shows wasn't actually the flashy LLM demos. It was data enablement. The announcements that made the biggest splash this year were the ones that focused on the new ways Snowflake and Databricks customers can enable data for a given use-case. The features that will help them get the right tools in place to deliver structured, governed, and reliable data into a modern data platform. So what were these announcements? We compare and contrast the events to understand the biggest news, what it says about the state of the data industry, and why data will always be the star of AI. Check it out: https://lnkd.in/eYA3kCEb #GenAI #DataCloudSummit #DataAISummit #dataquality #dataenablement

23

1 Comment

❄Snowflake Arctic: The Coolest Open-Source LLM on the Block?❄ Breaking New Ground: Snowflake Arctic is a cutting-edge, enterprise-focused Large Language Model (LLM) that is setting new standards in efficiency and accessibility within the AI landscape. Known for its ability to excel at complex tasks like SQL generation, coding, and instruction following, Arctic represents a leap forward in cost-effective AI training. With its unmatched capabilities, it empowers enterprises to create custom models that are not only high in quality but also economical, making advanced AI more accessible than ever before. Blueprint of Brilliance: At its core, Snowflake Arctic utilizes a pioneering architecture called the Dense-MoE Hybrid transformer. This structure combines a 10B dense transformer model with a residual configuration of 128×3.66B Mixture of Experts (MoE) MLP. This results in a robust model featuring 480 billion total parameters and 17 billion actively chosen parameters through a sophisticated top-2 gating mechanism. The training of Arctic is divided into three stages, each focusing on different data compositions: the initial phase emphasizes generic skills, while the subsequent phases concentrate on enterprise-specific abilities. This strategic curriculum is designed to fine-tune Arctic's proficiency in key business areas, enhancing its effectiveness and efficiency in real-world applications. Building the Future: The significance of Snowflake Arctic extends beyond its technical achievements. With an Apache 2.0 license, Arctic offers complete openness by providing ungated access to its weights, code, data recipes, and research insights. This level of transparency fosters a collaborative environment where the community can contribute to and benefit from the Arctic's advancements. Additionally, by performing comparably or better than established models while consuming significantly less computational resources, Arctic sets a new benchmark for sustainable AI development. It meets and exceeds the diverse needs of enterprises, driving them towards more sustainable and responsible AI usage, illustrating a future where technology and environmental considerations go hand in hand. 🔶 Taken from Yesterday's Newsletter 🔶 Link in Comments ⤵

4

Exciting News, LinkedIn Family! I just published another article to my snowflake series: "An Interactive Guide to Snowflake Stages and Storage Integration". In this article, I dive deep into: 1. The nuances of Snowflake Stages and their role in streamlining data movement 2. How to leverage COPY commands for bulk loading and the power of Snowpipe for continuous data ingestion 3. Practical examples of integrating storage solutions with Snowflake for optimal performance But wait, it’s more than just theory! I’ve included interactive examples and code snippets to help you apply these concepts in real time! Curious to enhance your Snowflake skills? Check it out and let me know what you think! #SnowflakeData #DataEngineering #CloudComputing #ETL #DataLoading #DataOptimization #RealTimeData #Snowpipe #CloudData #BigData

9

🎉🧐 The Technician's MBA is Live! 🧐🎉 The first post of "The Technician's MBA" is live on Substack - https://lnkd.in/eG5vC4wp. It's more outlining what content to expect, but if you want to have read you can do so with the link above. If you want to subscribe its free to do so and you will get each new post direct to your inbox every Wednesday. I will also cross post here for reach purposes. #TheTechniciansMBA #ThoughtLeadership #SoftSkills #TechnicalPractitioners #Coders #Coding #Leadership #Mentoring.

3

2 Comments

How can Tempered AI help you with adopting Databricks? 🤔 We have certified consultants that can deliver or speed up any project: ➡ Migrating to data and pipelines 🔄 Set up CI/CD best practices 🍱 MLOps (Yes, it's a bento box representing the varied components) 🤖 Create a RAG based chatbot? We are a small, newly-established and boutique firm of enthusiastic consultants focused on Databricks. We don't bloat costs as we are light on management and heavy on technical ability. We solve problems because we enjoy solving them. Got burnt working with a consulting firm that was all fluff? Experience what's it's like working with a firm whose vision is #TechFirst! The #Databricks professional services team trusts us to deliver projects, and so can you!

9

1 Comment

There has been a lot of hype around a semantic layer or metrics layer as an independent and separate part of the analytics stack. We've been hearing about this for years, but I think nobody in the data ecosystem has really delivered on the promise of an independent semantic layer outside of our friends at Cube. I'm excited that we're launching our Hashboard partnership with Cube today. Now you can easily explore and find insights in the metrics you define in Cube. We'll be hosting a webinar tomorrow to show off how the products work together - link below Hashboard and Cube have a strong shared vision for governed and consistent metrics. While Hashboard has an integrated semantic layer (Hashquery), the purpose is just to drive consistent metrics and exploration inside of Hashboard and to make the most compelling reporting system we can. There is still a missing link for plugging metrics into other tools in the ecosystem. I see two basic needs for a *separate semantic layer: 1️⃣ SQL is not particularly composable, it's hard to share calculations (I've written a lot about this) 2️⃣ The analytics stack has a bunch of missing links: tools not being able to interoperate. Power BI, probably one of the most popular BI tools doesn't work particularly well with Snowflake for example. Hashboard is certainly not going to help you connect Power BI and Snowflake anytime soon. There are a few requirements for a semantic layer to be useful as a separate part of the data stack (these problems are *hard): 1️⃣ It's ultimately a network effects business: you need lots of integrations both upstream and downstream for all of this to work. 2️⃣ It's hard to get visualization tools to want to adopt you until you have lots of upstream integrations and customers. So you have to make integration really easy. I think Cube's secret sauce is its sql api: you can just write sql against Cube as a visualization tool which means that you don't have to do a highly customized integration. Their semantic layer speaks sql which is pretty familiar for all of the downstream tools in the ecosystem It has been delightful to get to know and work with the whole cube team Brian Bickell Artyom Keydunov and Tony Kau!

50

4 Comments