Dana Forsberg

🚀 Transform Your SQL Development Experience with SDF Lint & Format Hygiene, consistency, and readability in SQL are essential for scaling data teams and enabling self-serve analytics. That’s why we’re thrilled to introduce SDF Lint and Format: 💨 Up to 1000X faster than the current standard SQLFluff ✅ Built-in support for Jinja macros and deep semantic SQL understanding ⚡ Zero-cost linting in daily development and CI/CD (because it’s so fast!) 🎯 Designed for developer productivity with sensible defaults and easy configuration Our Rust-powered SQL Linter and Formatter brings industry-leading speed and accuracy to SQL workflows, making it easier than ever to enforce standards and maintain clean, consistent code. Ready to take your SQL workflows to the next level? 👉 Check out our blog post in the comment below to learn more and get started.

33

6 Comments

Today we're taking the powerful data tools we've built for technical practitioners, and opening them up to everyone! We're releasing end-to-end no-code workflows with: ∑ 𝐒𝐩𝐫𝐞𝐚𝐝𝐬𝐡𝐞𝐞𝐭-𝐬𝐭𝐲𝐥𝐞 𝐜𝐚𝐥𝐜𝐮𝐥𝐚𝐭𝐢𝐨𝐧𝐬 🧮 𝐃𝐚𝐭𝐚 𝐛𝐫𝐨𝐰𝐬𝐢𝐧𝐠 🔏𝐔𝐩𝐝𝐚𝐭𝐞𝐝 𝐠𝐨𝐯𝐞𝐫𝐧𝐚𝐧𝐜𝐞 𝐟𝐞𝐚𝐭𝐮𝐫𝐞𝐬 🔮𝐀𝐈 𝐬𝐩𝐞𝐞𝐝 𝐚𝐧𝐝 𝐚𝐜𝐜𝐮𝐫𝐚𝐜𝐲 𝐢𝐦𝐩𝐫𝐨𝐯𝐞𝐦𝐞𝐧𝐭𝐬 These new workflows live right alongside our existing SQL, Python, and AI tools, so everyone— regardless of technical experience— can come together in Hex to ask and answer questions with data. Watch the launch video below, and check out the blog in the comments for all the details! https://lnkd.in/gdpHuAYb

88

9 Comments

If you're tired of expensive data lakehouse/analytics solutions and are considering open-source technologies, you definitely would like to read this: Dien, Tran Thanh shares his firsthand experience, proven best practices, and insightful suggestions for migrating from BigQuery to Apache Doris. They are saving $4,500 per month. https://lnkd.in/ggVXgRaB #datalakehouse #BigQuery #dataanalytics #dataengineering #dataplatform #opensource

50

2 Comments

Legacy TMS vs. TrueLiquid: See the difference! TrueLiquid outperforms legacy systems in: - Database Technology - Trailer Management - Real-Time Tracking - Implementation Time - And more... Our blog, "10 Clear Signs You've Outgrown Your Legacy TMS," dives deeper into why it might be time for an upgrade (https://hubs.ly/Q02MLm3n0) Learn how modern solutions like TrueLiquid can offer: ✅ Regular updates and advanced features ✅ Superior flexibility and scalability ✅ Efficient load matching and commodity assignment ✅ Enhanced operational efficiency and accuracy Don't let outdated technology hold your fleet back. Ready to make the switch? We're offering zero implementation fees and a 90-day satisfaction guarantee!* #FleetManagement #TMS #TruckingTechnology #TrueLiquid *Terms and conditions apply

8

🚀 Automate Your CI/CD Pipeline with Snowflake and GitHub Actions 🚀 Our Senior Data Engineer, Jaime Salas, shares an insightful guide on integrating GitHub Actions with Snowflake to streamline your CI/CD pipeline. Learn how to: 🔹 Set up key pair authentication 🔹 Create and manage Snowflake users and schemas 🔹 Automate tests and merges using GitHub Actions Enhance your development workflow with these powerful automation techniques! https://hubs.ly/Q02BWXC10 👉 Read the full article #DataEngineering #Snowflake #GitHubActions #CICD #Automation

15

Despite the very large number of data lineage solutions out there, getting *full* visibility to your data pipelines remains a common challenge for growing data platforms. And if you have some Spark-based pipelines in your data stack, whether it's Databricks, Amazon EMR or others, you should prepare to work a lot harder... ! However, there's a class of new solutions, such as OpenLineage and Foundational that go beyond traditional SQL parsing. Read more on our blog: https://lnkd.in/gCEdAXSE #datalineage #spark #datagovernance

12

💡 Optimizing Costs with Dragonfly: A Technical Breakdown 💻 For technical teams looking to maximize performance while minimizing cloud infrastructure costs, Dragonfly offers significant advantages. By cutting down memory overhead, leveraging CPU-specific optimizations, and using a shared-nothing architecture, Dragonfly is designed to be resource-efficient while maintaining high throughput. Whether you’re scaling horizontally or aiming to optimize performance, Dragonfly can help you reduce operational costs without sacrificing speed. In this blog, we break down the technical strategies that Dragonfly uses to cut costs and improve efficiency. 🔗 Read more here: https://hubs.la/Q02RDNVr0 #CloudOptimization #TechEfficiency #DevOps #DatabasePerformance #Dragonfly#CostReduction

3

Question: Considering migrating to Unity Catalog? Curious about potential errors or the rollback process? Watch this quick clip of Blueprint's Databricks Champion, Shannon Lowder, explaining just how easy it is to manage and reverse a migration if needed. #unitycatalogmigration #ucmigration #unitycatalog #databricks

8

Databricks is best known for its data engineering, processing, and data science capabilities that make it easy for enterprise data teams to ingest, process, and store large amounts of data before operationalizing it in AI/ML applications or building data warehouses to enable SQL querying. All of these capabilities can be leveraged inside the customer’s #cloudobjectstorage without moving the data into a proprietary storage platform, which would result in high data egress fees, high storage costs, and undesirable data retention trade-offs at scale. ☁️ But when it comes to using Databricks for log and event analytics, customers who want to leverage the deeper capabilities of specialized third-party tools will often need to ship data outside of their cloud object storage and into those platforms. This can result in high data egress and storage costs, especially for organizations that generate large volumes of log data. Enterprises may attempt to reduce short-term data storage costs by restricting the data retention window in third-party tools, but this also reduces the viability of long-term log analytics use cases like application performance trend analysis or security incident root cause investigation. Power log analytics use cases with ChaosSearch on Databricks. ChaosSearch can now be deployed in Databricks to deliver our simple, powerful, and cost-effective log analytics capabilities on the Databricks platform. Learn more: https://bit.ly/4eXFHxU

1

🚀 Tired of manual ETL processes slowing down your data flow? Stop dealing with the hassle of tedious data extraction and transformation errors — automation makes it seamless, reliable, and scalable. Redwood Software’s latest guide breaks down the ETL automation process and how RunMyJobs by Redwood can revolutionize your data workflows by: ✅ Simplifying cloud data warehousing with low-code integration ✅ Automating ETL testing and workflows using no-code templates ✅ Centralizing control over data across ERP, CRM, and more #ETL #Automation #DataIntegration #RunMyJobs #BusinessGrowth Read more 👇

12

Tired of manual ETL processes slowing down your data flow? Stop dealing with the hassle of tedious data extraction and transformation errors — automation makes it seamless, reliable, and scalable. Redwood Software’s recent guide breaks down the ETL automation process and how RunMyJobs by Redwood can revolutionize your data workflows by: ✅ Simplifying cloud data warehousing with low-code integration ✅ Automating ETL testing and workflows using no-code templates ✅ Centralizing control over data across ERP, CRM, and more #ETL #Automation #DataIntegration #RunMyJobs #BusinessGrowth Read more 👇

5

1 Comment

Running heavy workloads with Databricks? You can use Apache Airflow to integrate your workflows with other tools in your data stack and manage your pipelines as code. Learn how to orchestrate Databricks jobs using #Airflow at our live session on Thursday, December 5 at 2:00pm EST. You’ll learn: 📶 How to connect Airflow to Databricks 👀 Monitor your Databricks jobs from Airflow ✍️ Write tasks that run Databricks Jobs using familiar DAG code https://bit.ly/3ZnUGvm

46

New feature release - Job metrics timeline view! Ever wonder why your Databricks job performance changes over time? You may ask yourself questions such as: “Why is my runtime increasing since last week?” “Is the cost of this job also increasing?” “Is the input data size changing?” “Is my job in danger of crashing?” Understanding why your job performance is changing is critical to both optimize costs and prevent crashes. In this post we show a new visualization timeline features to help engineers find root causes quickly. Try it out today with Gradient! #dataengineering https://lnkd.in/ebPjtU7y

7

5 Comments

Materialize’s Chief Scientist Frank McSherry discusses how to create live data by just using SQL in our latest blog post. In the post, Frank builds a recipe for a generic live data source using standard SQL primitives and some Materialize functionality. Then he adds in various additional flavors: distributions over keys, irregular validity, foreign key relationships. It’s all based off of Materialize’s own auction load generator, but it’s written entirely in SQL and can be customized as your needs evolve. By the end of the blog, you’ll discover that the gap between your idea for live data and making it happen is just typing some SQL. READ BLOG -> https://lnkd.in/eFJcB_un

10

The team at Decodable have created your blueprint to success with real-time data. 📐🏗️✏️ Dive deep into real-time data success with this technical guide with the ultimate guide to accelerating ETL, ELT, and stream processing with Decodable. Download now and transform your data strategy. 👉 https://hubs.ly/Q02FW7S10

6

Unlock the Power of Data Lake Table Formats with Alex Merced, Developer Advocate at Dremio! 📊 Join Alex Merced as he delves into "The Who, What, and Why of Data Lake Table Formats." 📅 Save the Date: https://lnkd.in/eFZ6m5HV 🔍 Key Learnings: - Explore how data lake table formats handle large datasets. - Learn how these formats optimize query performance on extensive data. - Understand the importance of data durability and recoverability. - Discover how these formats ensure data is auditable and compliant. Don't miss out on this opportunity to level up your data analytics game! 📈 #DataAnalytics #DataLake #Dremio #TechTalk

4

Let’s dive headfirst into the Advanced Observability Suite (AOS) in the DuploCloud Portal. 👀 DuploCloud’s AOS lets you integrate, migrate, and scale observability data across platforms seamlessly. This is at a fraction of the cost and headache of other observability tools. 😵‍💫 Get actionable insights across your cloud infrastructure — securely, transparently, and without the BS. Learn more about DuploCloud AOS - https://lnkd.in/gD6QKuKQ #DuploCloudAOS #OpenTelemetry #Observability #CloudMonitoring

44

🚀 𝐀𝐧𝐚𝐥𝐲𝐭𝐢𝐜𝐬 𝐇𝐞𝐫𝐨𝐞𝐬 𝐭𝐨 𝐭𝐡𝐞 𝐑𝐞𝐬𝐜𝐮𝐞! 🦸 Featuring Ryan Brown, Sr. Data Engineer Architect at Trace3. 🙎‍♂️ Ryan’s an experienced data & analytics engineering expert, having worn multiple hats—analyst, engineer, consultant, and now architect. With over five years of experience building optimized ETL pipelines and data infrastructures, he’s seen it all. And in our latest episode, he’s spilling the secrets! 🎤💡 In Episode 2, we chopped it up with Ryan about the big stuff shaking up the analytics world. Here’s a quick teaser of what’s coming your way: 💠 The transition from 𝐅𝐚𝐛𝐫𝐢𝐜 𝐭𝐨 𝐃𝐚𝐭𝐚𝐛𝐫𝐢𝐜𝐤𝐬—and why it’s a game-changer 💠 𝐂𝐨𝐧𝐧𝐞𝐜𝐭𝐢𝐧𝐠 𝐭𝐡𝐞 𝐃𝐨𝐭𝐬 to ROI—where data meets $$$ 💠 Turning 𝐃𝐚𝐭𝐚 𝐑𝐞𝐜𝐨𝐫𝐝𝐬 𝐢𝐧𝐭𝐨 𝐅𝐢𝐧𝐚𝐧𝐜𝐢𝐚𝐥 𝐂𝐥𝐚𝐫𝐢𝐭𝐲—making sense of it all 💠 Why 𝐈𝐧𝐧𝐨𝐯𝐚𝐭𝐢𝐨𝐧 𝐄𝐪𝐮𝐚𝐥𝐬 𝐄𝐟𝐟𝐢𝐜𝐢𝐞𝐧𝐜𝐲—and how to stay ahead 💠 The 𝐃𝐨𝐮𝐛𝐥𝐞-𝐄𝐝𝐠𝐞𝐝 𝐒𝐰𝐨𝐫𝐝 𝐨𝐟 𝐆𝐨𝐯𝐞𝐫𝐧𝐚𝐧𝐜𝐞—balancing control with speed 💠 Why 𝐌𝐨𝐝𝐮𝐥𝐚𝐫𝐢𝐭𝐲 & 𝐓𝐞𝐬𝐭𝐢𝐧𝐠 are key to a rock-solid data strategy Sound like your kind of conversation? Want to catch all of Ryan’s insights? Hit that 𝐒𝐮𝐛𝐬𝐜𝐫𝐢𝐛𝐞 link and get the episode sent straight to your inbox as soon as it drops! 🔔📬 𝐒𝐮𝐛𝐬𝐜𝐫𝐢𝐛𝐞: https://lnkd.in/dywmwe7T This one’s packed with real-world takeaways and industry trends that’ll have your team talking. Don’t miss out!

11

2 Comments