Piethein Strengholt

Eraneos Netherlands 𝗽𝗿𝗲𝘀𝗲𝗻𝘁𝘀: 𝗗𝗲𝗲𝗽 𝗱𝗶𝘃𝗲 𝗶𝗻𝘁𝗼 𝗔𝘇𝘂𝗿𝗲 𝗗𝗮𝘁𝗮 𝗘𝘅𝗽𝗹𝗼𝗿𝗲𝗿 | 𝗙𝗮𝗯𝗿𝗶𝗰 𝗥𝗲𝗮𝗹-𝘁𝗶𝗺𝗲 𝗔𝗻𝗮𝗹𝘆𝘁𝗶𝗰𝘀 𝗠𝗲𝗲𝘁𝗨𝗽 𝗼𝗻 𝗝𝘂𝗹𝘆 𝟭𝟬𝘁𝗵 Spark, Databricks and Snowflake are typical technologies that come to mind when talking about ‘Big Data’. However, Azure has a fully managed analytics solution few people know about even though it handles Petabytes every day... The technology we are talking about is Microsoft Fabric’s Real-Time Analytics which is also known as Azure Data Explorer (ADX). It is purposefully built to provide real-time insights and analytics on large streaming datasets and typically used in monitoring, IoT / OT and web analytics scenarios. It’s ease of use, blazing performance and rich set of features give rise to its growing popularity. Microsoft uses it for all Azure monitoring, ingesting an estimated 18 PB of data and executing 30 Billion queries every day! During this session we will introduce Azure Data Explorer, talk about its value in real customer cases and get our hands dirty solving some ‘data puzzles’ using this awesome technology! Sign up via 👉 https://okt.to/MOhy1I and see you there! #MeetUp #Azure #DataExplorer #ADX #BigData

3

Why does Stijn (Stan) Christiaens the Co-Founder of Collibra have the title 'Chief Data Citizen'? He explained all on Driven By Data: The Podcast a couple of weeks ago! A great episode which you can listen to below ⬇️ #data #value

38

5 commentaren

Again as an independent senior EA and data architect consultant that loves looking at solutions offered to the area of expertise that I am working in. This morning I stumbled upon Ellie Technologies (a Dutch company) that, by looking at youtube, and official website seem to offer one thing that I still lack in many solution offered which is, high user friendliness, ease of collaboration, and AI (scary for a datanerd like me that still need to learn quality trust in AI for real data and process work) . Ellie.ai is a cloud platform designed to help data teams manage and collaborate on data projects efficiently. It focuses on simplifying data modeling, governance, and teamwork to get the most out of data initiatives. In short: -Comprehensive Data Management: Offers tools for both business and data modeling, making it easier to visualize and align data with business goals. -Collaboration Tools: Enables real-time teamwork across locations, improving communication and workflow. -Business Glossary: Helps standardize data definitions, promoting clarity across departments. -Scalable & Easy to Use: Works well with various platforms, making it adaptable to different organizational needs. It seems clever! I dont know pricing structure yet though. I probably ask for a proper demo to satisfy my curiosity 😁I hope they will give it. Does anyone n my network have hands on experience with Ellie? Let me know! If you have the time check it out: https://www.ellie.ai/ https://lnkd.in/dYVWN-yy

3

Recently, I finished reading 'Data Management at Scale, 2nd Edition' by Piethein Strengholt. It deconstructs established thoughts, adds new perspectives and guides through the journey from data-driven aspirations to a fully realized data-driven culture. A great read for those looking for a Decentralized approach for managing data at scale using Centralized alignment of organization, governance, technology, and architecture. Personally, I benefitted immensely from the Data Domain and Data Product topics with a focus on practical strategies for modern data engineering pipelines. https://lnkd.in/eFVgWt9V

24

🔥 Struggling to monitor thousands of Spark applications? It’s a common challenge! The diversity of Spark users often leads to inefficiencies, soaring costs, and endless frustration. In our latest blog “𝐌𝐨𝐧𝐢𝐭𝐨𝐫𝐢𝐧𝐠 𝐭𝐡𝐨𝐮𝐬𝐚𝐧𝐝𝐬 𝐨𝐟 𝐒𝐩𝐚𝐫𝐤 𝐚𝐩𝐩𝐥𝐢𝐜𝐚𝐭𝐢𝐨𝐧𝐬 𝐰𝐢𝐭𝐡𝐨𝐮𝐭 𝐥𝐨𝐬𝐢𝐧𝐠 𝐲𝐨𝐮𝐫 𝐜𝐨𝐨𝐥” Lead Data Engineer Niels Claeys shares expertise to help you take control. Here's what you’ll learn: 🚀 Tools to analyze Spark applications (and when to use them). 🔧 How Dataflint streamlines issue detection. 💸 Pro tips to save resources while scaling Spark efficiently. Don’t let Spark inefficiencies burn your budget. Read the blog now and take charge of your Spark landscape! 👉 Read the full article here: https://hubs.li/Q02_n6Hd0

15

Avoid these 7 Common Business-related Mistakes On Data Projects https://lnkd.in/eV_WGAbh #kamphuisgroep #caronne # #ArtificialIntelligence #MachineLearning #bigdata #businessintelligence

3

Your contribution for the evening 🌖: sodadata/soda-core🥤 Co-founded by Maarten Masschelein and Tom Baeyens, Soda is a data quality platform built for the modern data stack. Here’s why you should contribute to this repo: 1️⃣ With a growing repo now at ~1,700 stars, contributing to the soda-core repo allows you to refine your skills in data quality management, gain hands-on experience with the Soda Checks Language (SodaCL) and learn how to implement practical data quality tests across various data sources. You also have the opportunity to connect with other data professionals, expanding your network and opening doors to working opportunities. 2️⃣ Maintainers ensure a response rate of 100%, with typically a median PR merge time of around 3 hours. This guarantees that your contributions receive the attention they deserve. 🩶 3️⃣ The repo is more accessible. At Quira, we rank Developers by their DevRank (more on this here: https://lnkd.in/eNtrs97F). We found that the median developer working on the repo belongs to the 90th percentile of GitHub developers. For reference, many of the open-source companies we reported on have a median contributor level upwards of the 95th percentile. This means that you have a real opportunity to make an impact. ⚡️ If you want to help improve a pivotal tool for many organisations in ensuring data integrity, you can check more of their open-source stats on Quira (https://lnkd.in/egxYTC4N) or access the GitHub page: https://lnkd.in/e2r3iazu 📖

6

Eraneos Netherlands 𝗽𝗿𝗲𝘀𝗲𝗻𝘁𝘀: 𝗗𝗲𝗲𝗽 𝗱𝗶𝘃𝗲 𝗶𝗻𝘁𝗼 𝗔𝘇𝘂𝗿𝗲 𝗗𝗮𝘁𝗮 𝗘𝘅𝗽𝗹𝗼𝗿𝗲𝗿 | 𝗙𝗮𝗯𝗿𝗶𝗰 𝗥𝗲𝗮𝗹-𝘁𝗶𝗺𝗲 𝗔𝗻𝗮𝗹𝘆𝘁𝗶𝗰𝘀 𝗠𝗲𝗲𝘁𝗨𝗽 𝗼𝗻 𝗝𝘂𝗹𝘆 𝟭𝟬𝘁𝗵 Spark, Databricks and Snowflake are typical technologies that come to mind when talking about ‘Big Data’. However, Azure has a fully managed analytics solution few people know about even though it handles Petabytes every day... The technology we are talking about is Microsoft Fabric’s Real-Time Analytics which is also known as Azure Data Explorer (ADX). It is purposefully built to provide real-time insights and analytics on large streaming datasets and typically used in monitoring, IoT / OT and web analytics scenarios. It’s ease of use, blazing performance and rich set of features give rise to its growing popularity. Microsoft uses it for all Azure monitoring, ingesting an estimated 18 PB of data and executing 30 Billion queries every day! During this session we will introduce Azure Data Explorer, talk about its value in real customer cases and get our hands dirty solving some ‘data puzzles’ using this awesome technology! Sign up via 👉 https://okt.to/pwqEf6 and see you there! #MeetUp #Azure #DataExplorer #ADX #BigData

3

Winsome Wim Stoop & I offer a practical guide to building your modern data architecture. Some assimilate the word #modern with #future or #emerging, but it denotes #recency, and failing to stay modern portends getting #leftbehind. #ModernDataArchitecture #DataMesh #DataFabric #OpenDataLakehouse Cloudera CDO Magazine https://lnkd.in/gzUkfj7Q

38

2 commentaren

Is there maybe a similar article, but about how to transition from data engineering to software engineering? Because I have a strong feeling that I have written more YAML/SQL than Python/Scala/Java in the last year... Maybe one finds DE interesting, but for me writing all these YAMLs/JSONs/SQLs is so boring. The idea of "let's speed up your DE development with just-another-low-code-tool" may sound cool for the business, but not for me. I still like programming, not YAMLing :) https://lnkd.in/dEjeNtgY

26

Check out my Data Story Book summary of Collibra's DATA CITIZENS 24 Show in NYC. I am always looking for the Data Stories and I found plenty! Collibra CEO & Founder Felix Van de Maele opened the day with his vision to help customers "create the unified view of their data" We all have to be on the same page!👍 McDonald's John Tucker presented his unified view in the shape of a Quarter Pounder with cheese!* 🍔 We’ll take some AI fries with that! 🍟 * No, I did not ask what they call that system in France! 😂 DATAcated’s Kate Strachnyi🤩 showed up for a group shot with John! Thomas Brence still buzzing from the amazing New York Mets victory, deftly moderated his panel through endless baseball metaphors! ⚾ David Janson led off with healthcare giant Cigna Evernorth’s "self-healing data." Then Linh Nguyen threw an imagery curveball with an ’80s pop-inspired mnemonic for financial services firm Edward Jones, where data is "T’Pau: trusted, protected, accessible, understood." Chris Tabb, COO of LEIT DATA and Mayor of #MeanDataStreets arrived in his #DATAPUPPETS T-shirt! 🐝 Michael Rahm makes the case for AI governance: “Without constant access to trustworthy data, AI projects don’t get off the ground.” There's sure a lot of traffic on that runway! ✈️ Seth Clark vows to “capture as much metadata as possible” And I believe him! He never meta data he didn’t like!😆 OMG!! Ran into Ping F. and the DQ team from NBCUniversal, including Rahel Mehreteab and Runqian (Sarah) Huang. Ping was carrying my book TELLING YOUR DATA STORY and asked “Would you sign it?” WOULD I?!?! 😎 Then CMO Denise Vu Broady wrapped up a great day of Data Stories! Thanks, Collibra! 🙏 #datastorytelling #datacatalog #datamanagement #datagovernance #aigovernace

89

18 commentaren

Eraneos Netherlands 𝗽𝗿𝗲𝘀𝗲𝗻𝘁𝘀: 𝗗𝗲𝗲𝗽 𝗱𝗶𝘃𝗲 𝗶𝗻𝘁𝗼 𝗔𝘇𝘂𝗿𝗲 𝗗𝗮𝘁𝗮 𝗘𝘅𝗽𝗹𝗼𝗿𝗲𝗿 | 𝗙𝗮𝗯𝗿𝗶𝗰 𝗥𝗲𝗮𝗹-𝘁𝗶𝗺𝗲 𝗔𝗻𝗮𝗹𝘆𝘁𝗶𝗰𝘀 𝗠𝗲𝗲𝘁𝗨𝗽 𝗼𝗻 𝗝𝘂𝗹𝘆 𝟭𝟬𝘁𝗵 Spark, Databricks and Snowflake are typical technologies that come to mind when talking about ‘Big Data’. However, Azure has a fully managed analytics solution few people know about even though it handles Petabytes every day... The technology we are talking about is Microsoft Fabric’s Real-Time Analytics which is also known as Azure Data Explorer (ADX). It is purposefully built to provide real-time insights and analytics on large streaming datasets and typically used in monitoring, IoT / OT and web analytics scenarios. It’s ease of use, blazing performance and rich set of features give rise to its growing popularity. Microsoft uses it for all Azure monitoring, ingesting an estimated 18 PB of data and executing 30 Billion queries every day! During this session we will introduce Azure Data Explorer, talk about its value in real customer cases and get our hands dirty solving some ‘data puzzles’ using this awesome technology! Sign up via 👉 https://okt.to/07wcxK and see you there! #MeetUp #Azure #DataExplorer #ADX #BigData

4

Would you like to learn from other databricks users in the Netherlands? Join the largest databricks usergroup in the Netherlands (1500+ members) on linkedin for tips, tricks, solutions, previews and be the first to know about local databricks events. Click here to join ➡️ https://buff.ly/3ShQKHz #databricks #databrickslearning #azuredatabricks #databrickspartner #dataplatform #AI #data #dataengineering #revodata #berevolutionary #databricks #databrickspartner

6

Iceberg performance on Snowflake is almost as fast as native tables & 25x faster than #external tables on parquet files. Just because both options use #parquet files as the storage method does not mean they work or perform the same

30

While we can't predict the future, we can build it! We're expanding our data team and looking for passionate professionals who want to make an impact: * Data Engineer (because future-proof pipelines don't build themselves) a) Building scalable data infrastructure b) Ensuring data quality from source to insight c) Making 'it works in production' more than just a myth * Data Analyst/Power BI Developer (turning data into decisions) a) Creating compelling data stories b) Building intuitive dashboards c) Helping stakeholders find answers before they ask questions * Data Scientist (finding tomorrow's insights in today's data) a) Uncovering patterns that matter b) Building models that drive value c) Turning business questions into data solutions Want to be part of a team that's writing the next chapter in data excellence? Let's talk! Job ads in comments. DM open for questions. #hiring #datatransformation #analytics #futureofdata #dataengineering #powerbi #datascience

24

1 commentaar

no plans yet tomorrow? do you want to learn how to #automate your #database #deployment for your #SQLServer or #AzureSQL database(s) through #AzureDevOps? come and join me tomorrow #virtually! more info via the link in the post below

5

𝐃𝐨 𝐲𝐨𝐮 𝐰𝐚𝐧𝐭 𝐭𝐨 𝐮𝐧𝐥𝐨𝐜𝐤 𝐭𝐡𝐞 𝐟𝐮𝐥𝐥 𝐩𝐨𝐰𝐞𝐫 𝐨𝐟 Databricks 𝐟𝐨𝐫 𝐲𝐨𝐮𝐫 𝐝𝐚𝐭𝐚? Then check out our new “Databricks in a Week” workshop! In this 5 day workshop we will (optionally together with you): ✅ rapidly prototype and validate your data-driven product or solution ✅ showcase the Databricks technology hands-on with your data in your Databricks account ✅ demonstrate feasibility and value to your stakeholders Sounds good? Then head on over to our website and get in touch with our team! 👉 https://lnkd.in/dxHyFgfK What is your biggest challenge with Databricks at the moment? Let us know in the comments and let’s discuss! #databricks #data #DataIntelligence #DatabricksInAWeek

36

1 commentaar

𝗝𝗼𝗶𝗻 𝘂𝘀 𝗱𝘂𝗿𝗶𝗻𝗴 𝗼𝘂𝗿 𝗱𝗲𝗲𝗽 𝗱𝗶𝘃𝗲 𝗶𝗻𝘁𝗼 𝗔𝘇𝘂𝗿𝗲 𝗗𝗮𝘁𝗮 𝗘𝘅𝗽𝗹𝗼𝗿𝗲𝗿 𝗼𝗻 𝗝𝘂𝗹𝘆 𝟭𝟬𝘁𝗵! During this session we will introduce Azure Data Explorer, talk about its value in real customer cases and get our hands dirty solving some ‘data puzzles’ using this awesome technology 🚀 Spark, Databricks and Snowflake are typical technologies that come to mind when talking about ‘Big Data’. However, Azure has a fully managed analytics solution few people know about even though it handles Petabytes every day... The technology we are talking about is Microsoft Fabric’s Real-Time Analytics which is also known as Azure Data Explorer (ADX). It is purposefully built to provide real-time insights and analytics on large streaming datasets and typically used in monitoring, IoT / OT and web analytics scenarios. It’s ease of use, blazing performance and rich set of features give rise to its growing popularity. Microsoft uses it for all Azure monitoring, ingesting an estimated 18 PB of data and executing 30 Billion queries every day! Join our session via MeetUp 👉 https://okt.to/zTj5EP #MeetUp #Azure #DataExplorer #ADX #BigData

5

I share many of Niels Cautaerts views on software versus data engineering. There are some big differences that many people (CTOs, etc), processes (scrum, agile, etc) and business strategists/planners mis-handle. Read the link in the comments and lmk your thoughts.

12

4 commentaren