Jake Peterson

JSON Handling and Type Management with Large Datasets in Go -- Ep. 4 is out now! In this episode, Miki Tebeka tackles the complexities of working with JSON data, especially when dealing with extensive datasets and optimizing type management. Here, Miki introduces key strategies for managing large JSON documents, avoiding unnecessary type definitions, and using Go’s encoding/json package to handle custom serialization needs effectively. Things you will learn in this video: ✅ Streamlining JSON Handling: How to avoid excessive type declarations by using anonymous structures and selective decoding for large JSON datasets. ✅Implementing Custom Serialization: Techniques for defining custom JSON serialization in Go with the Marshaler and Unmarshaler interfaces. ✅Managing Mixed-Type Arrays: Strategies to work effectively with JSON arrays containing various data types, ensuring smooth integration with Go’s type system. 📰Watch Now: https://lnkd.in/ghjz58pC Stay tuned for more episodes from Miki Tebeka in this mini series, highlighting Intro to #JSON for Engineers & Managers.🤓 #golang

23

Have you tried Snowflake's Artic? Artic is the latest open sourced LLM launched by Snowflake. It's amazing to observe the innovation driving training and operating efficiency for LLMs. Some Specs about Artic: - Dense-MoE Hybrid transformer architecture - 480B parameters spread across 128 fine-grained experts - 17B Active Parameters - Apache 2.0 licensed The main use cases of Arctic are SQL, coding, and instruction following for enterprise applications. Generating SQL code presents an important challenge, demanding a deep understanding of both natural language and programming syntax. It involves crafting precise outputs that faithfully reflect the user's intentions. Artic is available on Hugging Face, Together AI, Microsoft Azure, NVIDIA API Catalog and Replicate. It's also coming soon on Amazon Web Services (AWS), Lamini and Perplexity.

26

6 Comments

Our yearly report on the State of Spatial Data Science is out! Co-authored with our partners at Databricks, it will give you endless conversation topics for your summer barbecues on how the adoption of Spatial Data Science is growing across different industries. Check it out here: https://lnkd.in/dU4bFVdi

45

2 Comments

You know a Semantic Layer would be hugely valuable, but how do you actually build such a thing? Pro tip: Crafting a Semantic Layer is about building iterative velocity alongside accuracy, so that when your stakeholders ask about Revenue MoM grouped by Attribution Channel, you can answer instead of adding a ticket to the backlog. Start with these four steps: 1. Identify a Data Product that is impactful: Find something that is in heavy use and high value, but fairly narrow scope. Don’t start with a broad executive dashboard that shows metrics from across the company because you’re looking to optimize for migrating the smallest amount of modeling for the highest amount of impact that you can. For example, a good starting place would be a dashboard focused on Customer Acquisition Cost (CAC) that relies on a narrow set of metrics and underlying tables that are nonetheless critical for your company. 2. Catalog the models and their columns that service the Data Product, both in dbt and the BI tool, including rollups, metrics tables, and marts that support those. Pay special attention to aggregations as these will constitute metrics. 3. Melt the frozen rollups in your dbt project, as well as variations modeled in your BI tool, into Semantic Layer code. 4. Create a parallel version of your data product that points to Semantic Layer artifacts, audit, and then publish. Creating in parallel takes the pressure off, allowing you to fix any issues and publish gracefully. You’ll keep the existing Data Product as-is while swapping the clone to be supplied with data from the Semantic Layer. Dig deeper into the step-by-step process of how to ship a Semantic Layer in pieces at our link in the comments.

371

26 Comments

Navigating the Databricks landscape can be challenging, even for seasoned data engineers. Our resident expert Robert Leach wrote about the common mistakes people make, and offers practical advice on how to: 👉 Tailor your clusters to drive for performance and cost efficiency 👉 Ensure consistency, compliance, and reliability with proper data governance policies 👉 Boost your productivity by mastering the Databricks Command Line Interface 👉 Implement robust access controls, encryption, and regular security reviews 👉 Make the most of the platform’s capabilities You’ll want to bookmark this post! 🔖 https://bit.ly/45DZqhY

15

What does marketing in a post-cookies world look like? https://lnkd.in/gN7BPfjQ #datascientist #dataanalyst #cfo #cmo #productmanagers #founders #cofounders #cxo #ceo #cpo #bizops #revops #operations #b2bsaas #marketing #marketingmanagers

1

There was a whole lot of talk about GenAI at the Snowflake and Databricks Summits this year, but don't be fooled – the star of both shows wasn't actually the flashy LLM demos. It was data enablement. The announcements that made the biggest splash this year were the ones that focused on the new ways Snowflake and Databricks customers can enable data for a given use-case. The features that will help them get the right tools in place to deliver structured, governed, and reliable data into a modern data platform. So what were these announcements? We compare and contrast the events to understand the biggest news, what it says about the state of the data industry, and why data will always be the star of AI. Check it out: https://lnkd.in/eYA3kCEb #GenAI #DataCloudSummit #DataAISummit #dataquality #dataenablement

23

1 Comment

There have been huge waves recently after #Databricks announced their new open-source fine-grained MoE (mixture of experts) large language model... but what does that actually mean? If you're already happily chatting away with your favorite LLM, why would you care? And how do you even get started with using it? In this video, Simon takes a look through the major points of the DBRX announcement and what the sheer focus on speed tells us about LLM maturity. We then take a dive into the new Databricks AI Playground to test out DBRX Instruct and show you how you get can get started with it. https://hubs.la/Q02trdVk0 #dataengineering #datalakehouse #datascience #machinelearning #llm #DBRX #genai

21

1 Comment

System slowdowns don’t happen in isolation—there’s always a trail of breadcrumbs waiting to be found. In just two minutes, using anomaly monitoring tools, we uncovered the real issue – and it wasn’t what anyone expected. Here’s what we found: 🔍 The slowdown wasn’t tied to SQL queries. Instead, buffer I/O problems were causing severe disk performance degradation. 📊 By analyzing performance trends and historical data, we tracked the issue back to spiking write and read operations between the 24th and 31st. ✅ With this clarity, we pinpointed the root cause: poor disk performance, not query inefficiency or system changes. This quick, data-driven approach saved hours of guesswork and got the client back on track in no time. 👀 Watch the full video to see how we diagnosed and resolved the issue – and how you can do the same for your systems.

4

You can’t properly treat a symptom without understanding its cause. Like a data doctor, you need to look at your data issues from all angles—identifying both the symptoms (what’s visibly wrong in your pipeline) and the causes (the underlying issues leading to those problems). That’s why we launched Fivetran observability to begin with: so you could start understanding which connectors were upstream of data quality issues. Today, we’re rolling out another huge enhancement as part of our Summer of Data Observability: observability on Fivetran metrics like connector duration and freshness. Our new dashboard puts connector performance front and center, so you can catch ETL issues as they happen. Now, you can: • Explore how long each connector takes • See how many rows Fivetran synced • Check for unusual connector performance spikes • Consolidate alerts in one place Check out the blog post in the comments. Can you guess what we’re launching next? 👀 #dataengineering #dataobservability #data

23

1 Comment

"Only data teams need AI training..." "AI will replace jobs in my firm..." "My business needs a Chief AI Officer..." The Harnham Group's latest whitepaper, Demystifying AI: Bringing Order to Chaos, comes off of the back of our event last month with Tableau. From a comprehensive history of AI to exclusive commentary from industry experts, this report aims to dispel some of the biggest myths surrounding artificial intelligence right now. 🎯 Download the whitepaper here: https://lnkd.in/drGJVNSC

8

The latest update for #SnowflakeDB includes "Retail Media's Business Case for Data Clean Rooms Part 2: Commercial Models". #analytics #BusinessIntelligence #BI #DataDriven https://lnkd.in/dH8h5ftY

2

Check out HealthDataViz's latest blog article "Improve the UX of Tableau Filter Selections with a Dynamic Error Message" https://hubs.li/Q02Bm-ml0. In this blog Katie Bueno shares a trick for handling user messaging when a chart goes blank due to advanced filtering. #Tableau #dataviz #BlogPost

3

Life is full of surprises. That's why Nue offers easy-to-update forecast assumptions at lightning speed. 🏎️💨 Our scalable forecasting platform lets you run an unlimited number of scenarios – helping you prepare for whatever life sends your way. See how: https://lnkd.in/evjUzNVH #FinancialPlanning

4

Evaluating prompts at scale with Prompt Management and Prompt Flows for Amazon Bedrock In this post, we demonstrate how to implement an automated prompt evaluation system using Amazon Bedrock so you can streamline your prompt development process and improve the overall quality of your AI-generated content.... https://lnkd.in/eNpp_-QJ #AI #ML #Automation

1

Tableau dashboards are powerful tools, but ensuring their performance can be time-consuming and complex. With Datagaps BI Validator, you can automate performance testing to ensure your dashboards are always running at their best. https://hubs.la/Q02PVdpB0 #TableauDashboards #DatagapsBIValidator  #Tableau

14