Michael Down

Snowflake Cortex described in 3 words.. Fast, Easy, Secure. You can do a lot with other products out there, but no one does it like Snowflake. https://lnkd.in/eErwVa5e #Snowflake #Cortex #AI

54

Top 5 Reasons to attend the ThoughtSpot Data Renaissance event on 12th June! I'm thrilled to be hosting the event and will be joined by some of the most successful data teams in the industry, some of our most innovative technology partners and some of the most informed thought leaders as we discuss what the Data Renaissance is and why it matters for your business. Here's why attending is a must; 📊 Learn From Customers - Ovo Energy, Brambles and Vita Mojo are all sharing their unique journeys of how they have become more data driven and delivered value from their data. Learn from their challenges and understand what shortcuts can be made 📊 Bernard Marr - world renowned futurist, influencer and thought leader will be discussing how GenAI in practice will revolutionise your business 📊 See the Data Renaissance - ThoughtSpot has a differentiated vision as to how Search and AI can come together to deliver an easy and unique experience for the business user. Learn how our innovations can drive adoption, time to insight and value for your business 📊 Cindi Howson - there are few people in the world that understand the Data & Analytics market as well as Cindi. Her unique perspectives and views on how to be successful with data will inspire, inform and educate. 📊 Engage with Innovators - Google, AWS and Snowflake will all be at the event showing their latest and greatest data platforms and AI. A real chance to engage 1 on 1 to understand their product vision and alignment to your strategy The event is the launch of the Data Renaissance so don't miss out, we'd love to see you there. See the link in the comments for more info and to register.

51

1 Comment

Read this great story from hipages Group on how they overcame challenges including legacy data systems slowing productivity and insight, requiring days to transition data between multiple warehouses. Partnering with Databricks hipages overcame these hurdles by transforming their data architecture into a single lakehouse architecture, enhancing both speed and reliability.  “We were able to tackle all our platform, productivity and insights challenges by migrating to a single platform, Databricks. Our architecture is more streamlined and our data users are more autonomous.” — Minas Kamel, Senior Product Manager, hipages #data #ai #lakehouse https://lnkd.in/gfXSqNSy

72

2 Comments

Kai Waehner always has insightful, informative, and practical content - he's one of the best. But his recent series on Snowflake (and DWH integration broadly) and some of the patterns, technical and cost considerations, etc is some of the best from the best and worth a read: https://lnkd.in/gtR-6Ces

8

So you've looked at DocAI and thought it had to be a very difficult and expensive way to tackle GenAI? Well, with Snowflake Arctic-TILT think again! #ArticTILT easily extracts content, like invoice amounts or contract terms, from documents via a built-in, multimodal large language model (LLM) we call Snowflake Arctic-TILT (Text Image Layout Transformer). Users now have a way of interacting with unstructured data that simplifies the setup and deployment, enabling users without any machine learning background to effortlessly package model builds for high-value, enterprise-scale extraction tasks. #LLM #GenAI #Snowflake #AI #multimodal

16

A great read in iTnews today highlighting how Seven West Media leveraged Databricks to forecast audience view patterns on both a seven-day and 28-day horizon. The company is now scaling its use of Databricks to support additional analytics use cases. Andrew Brain, Audience Intelligence and Growth Director, emphasized the significance of this advancement, stating, "We can now understand what our audience is going to be from a value perspective on volume and also what they’re going to be watching." This insight is pivotal for their business. #databricks #ai Read more about Seven West Media's predictive analytics success here: https://lnkd.in/e4rb3_mp

174

4 Comments

My latest on how infostealers play an outsized role in breaches and what organizations can do. Snowflake is just the latest example of how this complex ecosystem is impacting the landscape.

32

BigQuery now has support for Delta Lake tables through Big Lake. #bigquery #googlecloud #databricks

32

Do you want a better understanding of Databricks Lakehouse Monitoring, how to implement it, and if it is a good fit for you? Join Anna-Maria Wykes at the next Manchester Databricks User Group Thursday 15th August - 18.00 - 20.00 Sign up in the comments below. #meetup #databricks #lakehouse #monitoring

5

3 Comments

Yesterday we published a little piece to cover something that has been the subject of A LOT of our early calls on integrating DB2 on AS/400: what journaling means for the machines, how to manage it, and how to estimate usage. If data ingestion from IBM i machines is a topic for you, then you should give this a read to level up on the topic. https://lnkd.in/dDaYZSs8 #mainframe #cdc #db2 #ibmi #changedatacapture #etl #elt

24

1 Comment

Tobias Macey and Tom Baeyens discussed the role of data contracts in today's episode of the Data Engineering Podcast and raised some good points about data testing. The rich metadata embedded in data contracts offers a unique opportunity to improve data pipeline workflows. This metadata can be utilised not only for data quality checks but also for generating production-like data, which gives you all the essential ingredients for automating integration tests. By producing input data for your pipelines and validating the output data, you can ensure more robust and reliable data processes. Currently, many data pipelines lack integration testing due to various reasons such as: Time-Consuming: Integration tests can be lengthy, delaying the development process and demotivating developers. Maintenance Difficulty: Keeping tests up-to-date and relevant is often complex and resource-intensive. Lack of Tools: Limited/no tools are available for testing data pipelines. Data Catering is one of only a few companies that have truly invested in data contracts, which helps resolve all the above pain points. It has direct support for the Bitol Open Data Contract Standard (ODCS), can be run on your local laptop or as part of your CI/CD pipeline, is data source agnostic, and offers a great developer experience. For more details on how Data Catering makes testing simple, check out this page: https://lnkd.in/gQFJG4RQ Data Engineering Podcast link: https://lnkd.in/gBWTTk-h #datacontracts #testing

21

Graph databases have emerged as a powerful tool for managing interconnected data, but not all are created equal. Join industry expert George Anadiotis and George Demarest as they uncover the key criteria for evaluating #graphdatabases in this #webinar. https://lnkd.in/eWBYYY6S

4

📣 No matter where you're at in your journey to learn about Snowflake Cortex, Mistral, TruLens, or Streamlit, there are some incredible free workshop resources available to you. 💪 Today, I attended the first of four workshops being provided for participants of the Snowflake Rag 'n' Roll Amp Up Search Hackathon. 💡 The workshop held today was the kickoff to the hackathon and included some great content about Cortex Search provided by David Taylor and Josh Reini. Don't worry if you weren't able to make it to the live session because the workshop was recorded. You can access the recording from the resources tab once you sign up for the hackathon. ▶️ Signup link: https://lnkd.in/gnQCcv9A The next workshop is about Mistral AI and will be held on December 4th. TrueLens workshop will be held December 11th. And the Streamlit workshop will be held December 18th. 🙏 Thank you Michelle Brain (Desrochers) and Devpost for making this a great experience for participants. 🏆 I'm one of the hackathon judges on the panel and I can't wait to see the creative ideas that emerge from the hackathon. #hackathon #snowflakecortex #snowflakedevelopers #datacloud #snowflakedatacloud #aidatacloud #datasuperhero #snowflakesquad #artificialintelligence #generativeai #ai #technology #cortexsearch

41

Gemini in BigQuery changes the game for data engineers and analysts! More productivity ✔️ Improved query performance ✔️ Optimized costs throughout the analytics lifecycle—from ingestion and pipeline creation to deriving insights with AI ✔️ #GoogleCloudNext

2

"Undisciplined use of Kafka leads to poor quality data." That’s 100% true - not just for Kafka but for any data movement/integration tool. Hear hear Andrew Jones Almost everywhere I go I see a mixture of home-grown CDC, SaaS tools like Fivetran, Stitch, QlikReplicate, sprinkled with cloud tools and OSS like Airbyte and Airflow or whatever to orchestrate. The problem is very rarely the tools used. The problems always stem from a mix-and-match approach. No real holistic design … as Andrew says it is a problem of discipline. I’d go as far as say beware of any tool that makes your job of moving data easy. There will always be a catch, probably technical debt to pay down as the ELT pipelines proliferate! Paraphrasing Tolstoy “all happy data pipelines are alike, all unhappy data pipelines are unhappy in their own way”

3

2 Comments

Some fun data edge cases we have come across 😅 I’m actually very curious about the use case for negative years - if anyone can share, I’d love to hear it. 👉 Timestamp value where the year exceeds the YYYY format (e.g. 20350) and causes downstream encoding issues 👉 Negative years  👉 Non JSON values in a JSONB column 👉 JSON values that are not JSON compliant, like {"foo": "bar", "foo": "bar"} #dataengineering #datareplication #data

22

4 Comments

Today we are enhancing Amazon Bedrock Knowledge Bases with structured data retrieval for RAG from Amazon SageMaker Lakehouse, including Redshift, S3 and S3 Tables. Create Knowledge Bases with support for structured data stored in the Lakehouse/Redshift. #amazon #bedrock #kb #rag #structureddata

262

1 Comment