Kalyan Chakravarthy Arangam

We're coming to London 🇬🇧 Join Guy Fighel at Big Data LDN on Sept 18-19 for four #dataengineering events you cannot miss: ⚙ Guy's Keynote: The Inevitable Shift to Modern Data Lake Architecture (12:40pm Sept 18) ⚙ Great British Data Founders Panel (5:40pm Sept 18) ⚙ Data Engineering Things Meetup (6pm Sept 18) ⚙ Drinks with Hetz at Big Data LDN (2pm Sept 19) Featuring Hetz data portfolio Expanso + Superduper + Upriver Sign up info in the comments 👇 #data #engineering #startups

17

2 Comments

In our latest case study, we explore how Intellinexus assisted a manufacturing company to migrate from 'On Premise IBM Netezza' to 'Snowflake Data Cloud'. The company undertook the migration in three phases. The first was focusing on data and code, followed by ETL to ELT optimisation along with data sharing and data monetisation considerations, and then concluded with integrating advanced analytics capabilities Snowflake provides for predictive workloads, forecasting and data apps. Follow this link to read the full case study: https://buff.ly/3JWNRbC #DataInsights #DataDriven #Technology #Manufacturing

13

1 Comment

🚀🎙️ Exciting News! 🎙️🚀 We're happy to announce the launch of my new podcast series: "Success in Data Analytics" 15 mins max episodes. This will interest anyone embarking on their data journey. In this weekly series, Ben Parker will sit down with David Hill, Oka-BI's CTO, who has over 20 years of experience delivering data warehouses, to explore the strategies, insights, and trends that drive success in data analytics projects. 👉 First episode is OUT NOW - Platform Architecture Fundamentals We discuss 3 topics: ➡ Objectives ➡ Data Volumetrics ➡ Single Platform Strategy You can listen to the full episode below ⬇ https://lnkd.in/eiVzgFQm #dataanalytics #datafundamentals

6

2 Comments

🚀 Simplify your data operations with Zizo Pathway. Our platform makes ELT, data modelling, and query creation effortless. Explore how at https://www.zizo.co.uk/. #SimplifyData #ZizoPathway #EffortlessOperations

1

Metadata is a key asset in your data estate, but the term is broadly used. So let’s break it down… In the latest episode of our “Data Engineering in Five Minutes" series, Harry Cummings—our Principal Technical Consultant—discusses the different categories of “data about data.” #DataManagement #MetaData #DataEngineering

32

📢 SESSION ANNOUNCEMENT 📢 Calling all Databricks Ninjas! 🥋 Azure Databricks 🌐🧱 provides a highly optimised runtime environment supporting workloads in the Lakehouse, many of these done automatically. However, what can be done when automatic optimisations are not enough and cluster scaling alone does not meet expectations? 🤔 We will look at how we can influence DBR performance with advanced configuration settings and analyse physical query plans, looking at how they can be optimised 💻💯 👨‍🏫 Databricks Ninja: Performance tuning 🗣 Jakub Wawrzyniak - MVP Data Platform, solutions architect, coordinates and is responsible for the quality of projects implemented in the area of AI, BI, Big Data and advanced analytics. 🗣 Łukasz Grala - MVP Data Platform since 2010, mentors the TIDK team, which implements innovative projects such as concepts and strategies for data-driven organisation. 📅 Wednesday, May 15 ⏰ 11:40 am - 12:40 pm 📍 Auditorium Unlock the secrets to optimizing performance in Databricks and become a true ninja in data processing 🥷 Reserve your spot for SQL Day 2024 here 👉 https://sqlday.pl/ 🤩 #SQLDay2024 #sqlday #DataCommunityPoland #datacommunity #wroclaw #poland

10

❄️ Learn how to cut Snowflake data ingestion costs by 70% or more… ⚡️ …with much faster updates… . 👆 …and a single “source of truth” data copy, for all use cases. 🗣️🗣️ Ryan Garrett and Andy Walner of Onehouse explain it in in our webinar, three short weeks away. https://lnkd.in/ghsPrMv2 #onehouse #dataengineering #nolockin #datalakehouse #apachehudi #apachextable #opensource 

12

1 Comment

Brendan Grady of Qlik has some advice for enterprise companies thinking of building their own LLMs. Please don’t! Join Brian Booden and George Beaton every Tuesday fortnight for another episode of TDM - The Data Mix where we chat with the best guest from the world of data and technology. Or catch us on our YouTube channel at https://lnkd.in/eVwkBNXW

1

Following the recent Informatica World Tour, we asked Martin Paice, Valcon UK’s Data Capability Lead, for his insights. He highlighted that CEOs are searching for AI-driven ROI, often from investments they haven’t made yet. In a landscape where everyone has similar access to AI models and supporting services, the true differentiator is the quality of your data. High-quality data is what fuels AI to deliver a competitive edge and drive real business impact. Want to make your data work harder for your business? Contact Martin Paice at [email protected] to discover how to unlock data-driven value from your AI investments. #informaticaworldtour #iwt2024 #ai #dataquality #implementingyourfuture

39

May is here, which means it is time for the next edition of our First Friday newsletter. This month: 🎯 Andy Hayler with our annual Data Quality Market Update 📚 David Norfolk reviews 'The Algebra of Data' 💻 Daniel Howard on the data classification capabilities of IRI - Total Data Management ➕ Plus loads more, including analysis of Teradata, Redgate Software, VIAVI Solutions, Mage Data and SnapLogic #MutableBusiness #newsletter #research #data #bookreview #dataclassification #datafabric #dataquality #testdatamanagement https://lnkd.in/eNERW9zy

7

1 Comment

💡 Pathway by Zizo makes data modelling and ELT effortless with its codeless approach. Embrace the future of data handling at https://www.zizo.co.uk/. #EffortlessDataModelling #CodelessELT #ZizoPathway

2

The Forecast is Looking Sunny for Snowflake ❄️ Snowflake is venturing into the generative artificial intelligence space with the introduction of its open-source large language model, Snowflake Arctic. - Open and Enterprise-Grade: Snowflake Arctic is touted as the most open, enterprise-grade large language model currently available. - Mixture of Experts Architecture: The advanced architecture enhances performance across a diverse array of enterprise applications, making it a robust solution for varied business needs. These advancements in Snowflake's generative AI capabilities present new opportunities for WhereScape's automation tools. By integrating with Snowflake Arctic, WhereScape can offer enhanced AI-driven data solutions, further optimizing enterprise data management and analytics. More Here: https://buff.ly/3UKGvga #AI #DataManagement #Snowflake #WhereScape #Automation #BusinessIntelligence #GenerativeAI

26

🚨 So, you've connected #Tableau to #Snowflake, delivered killer analytics, and now your users want more. But here's the plot twist: with the increasing demand for #dataaccess comes the challenge of safeguarding sensitive information like never before. 📈 In our latest eBook, discover the 4 steps to transforming your BI strategy from fragmented to fortified. Say goodbye to the shared service account chaos and hello to streamlined data access management. It's time to ensure every piece of the puzzle is in place for a complete view of your business. 🔒 Don't let critical data fall through the cracks. Download now and unlock the secrets to controlling and protecting your BI data with ease. 👉 👉 👉 https://lnkd.in/gMg7KiJp #BI #DataAnalytics #Snowflake #Tableau #DataSecurity

4

📈 Better data understanding = better business outcomes Nicola Askham, the Data Governance Coach, knows the secret to success lies in creating a shared language around data. When teams are aligned and everyone has a clear understanding of data definitions, magic happens. 🪄 Imagine a world where marketing, sales, and finance all speak the same data language. No more confusion, no more miscommunication, just smooth collaboration and reliable insights. That's the power of effective data governance. 💪 Don't let inconsistent data definitions hold you back any longer. Take control of your data, create clarity, and watch your organization thrive. Learn from the best – check out Nicola Askham's expert advice now via link in the comments 👇

49

3 Comments

Have you tried running the DBRX models from Databricks? In our latest blog post we provide a tutorial to get you started with the model on Ori’s public cloud. We were very impressed with DBRX’s performance on the Hugging Face Open LLM Leaderboard before the updates last week. While DBRX Instruct did not score as highly under the new scoring system, it's still one of the best open models for Language Understanding (MMLU), Programming (HumanEval), and Math (GSM8K). We also used BeFOri, the benchmarking framework from Ori to determine DBRX Base performs inference most efficiently between 40 and 256 concurrent requests on one of Ori's 8 x H100 PCIe bare metal servers. Check out the blog to learn how you can use BeFOri to optimise self-hosted LLM performance on any infrastructure! https://lnkd.in/gi55WEks

43

❄Snowflake Dynamic Tables are now officially Generally Available 🚀 That means 🏎 faster data in Snowflake is even easier and a great time to revisit Streamkap's Demo Dare from earlier this year. Load data in seconds and use Dynamic Tables to set your latency targets and you're all set for enabling near real-time customer-facing analytics and operational use cases. #snowflake #streamingdata #dynamictables #snowpipestreaming #realtimedata

98

1 Comment

🤔 Curious about the benefits of the WhereScape / Databricks integration? Join our webinar, "The Benefits of Combining WhereScape with Databricks," on Thursday, July 11th. We have two sessions available: EMEA / APAC: 10:00 CEST | 16:00 SGT - Register Here: https://buff.ly/4eEWTIy North America: 10:00 AM CDT | 4:00 PM BST - Register Here: https://buff.ly/4cBbfHM Discover how Databricks’ Medallion Architecture and WhereScape’s automation tools streamline workflows, improve data quality, and optimize the data lifecycle. Join us to elevate your data management strategy! 🌟 #DataManagement #WhereScape #Databricks #Automation #DataProcessing #MedallionArchitecture #DataQuality #BusinessIntelligence

9

Fancy a free 60-minute assessment to help you better understand Unity Catalog? 👀 Our consultation will take a look at your environment and how you work, to help us set your on the right path for your migration. Get in touch via the Azure Marketplace here: https://lnkd.in/ePEpPRga #UnityCatalog #Databricks #DataEngineering

1

Is your data team spending more time firefighting than building scalable solutions? 🚒 Are your data pipelines clogged with redundant transformations and struggling to meet evolving SLAs? The lack of alignment between data sources and user needs could be the root cause of your pipeline challenges. Animesh Kumar, CTO and co-founder, and Travis Thompson, Chief Architect of The Modern Data Company, lay out a framework for turning data into a product that actively shapes infrastructure. They share how aligning SLAs with aggregated data products creates faster, more reliable systems. By shifting from reactive customizations to a proactive data product model, teams can reduce complexity and deliver insights efficiently. If you’re looking for practical strategies to transform your data infrastructure, this guide is a must-read. Click on the link in the comments section to know more. #DataProducts #DataInfrastructure #TechDebt #SLAs #DataEngineering

18

2 Comments