Boris Bialek

At #T3DD24, I talked about the architecture of the mittwald cloud platform, the operational challenges that we encountered, and our future plans for ensuring long-term reliability and scalability. I've finally gotten around to posting a (two-part) write-up of said talk on the mittwald blog, brought up-to-date with everything that's changed in the past few months and a few more words on our plans for the very immediate future (in german; translation provided by the AI assistant of your choice). https://lnkd.in/gTpQBEwb

22

3 Kommentare

🤔 What are the differences between Data Fabric & Data Mesh? How does Confluent enable these architectures? 🎙️ We answer these questions and more in Episode #4 of our Confluent Capabilities 'Bite Size' podcast series with Italo Nesi and John Dee! Discover how Confluent enhances both of these models through its cloud platform, showcasing it as a versatile solution and demonstrating its crucial role in modern data strategies. 🖥️ Watch the latest episode here: https://lnkd.in/emG9jRJ2 Paul Archer Joanna Grimmond Chris Gallagher Bob Feeney #HybridCloud #MultiCloud #Confluent #DataFabric #DataMesh

10

🛠️ Build powerful applications with the integration of #AzureCosmosDB NoSQL and #OpenAI! Sandeep Nair demonstrates how to leverage vector embeddings for seamless multi-modal data operations. Watch now: https://lnkd.in/ezx6s8yu

2

The first version of the new Dynatrace Distributed Tracing app allows you to easily analyze trace data with: Enhanced visualization for deeper insights Automatic data capture for seamless analysis OpenTelemetry integration for better trace data utilization Kayla Bondy and Thomas Rothschaedl provide a walkthrough of our latest app here.

6

Tune in on December 17, as we're diving into the future of table optimization with Beyond Partitioning: Using #LiquidClustering to Escape from Hive-Style! I Say goodbye to the costly guessing game of partition optimization. With liquid clustering, your tables can adapt effortlessly to shifting data volumes—no more stress about getting your partition strategy right from day one. For years, we’ve needed to play an expensive guessing game — will my partition strategy result in too small, too big, or just right file sizes? Do the decisions today still make sense 6 months down the road? What about 1 year? What if you didn’t need to make so many decisions on day one? What if your table could instead adapt to the volume of data with minor effort on your behalf? Welcome to the world of liquid-clustering! In this session you’ll learn to harness the power of adaptive clustering and come to an understanding that today is the day you can finally escape the shackles of hive-style partitioning https://lnkd.in/g_u44AZD

2

Join CAST AI, DKB and Grafana Labs as they showcase how their solutions work together to optimize cloud costs. Network with peers and hear from industry experts. Learn how our platforms support Kubernetes to make it simpler to configure its monitoring tools, troubleshoot issues, and reduce cloud costs.

33

3 Kommentare

I’m really looking forward to this conversation! Join us for an in-depth comparison of PostgreSQL and MongoDB. AI workloads place greater demands on databases than traditional Online Transaction Processing (OLTP) systems. They present unique challenges, including massive data volumes, high throughput, and diverse data structures that exceed the capabilities of most relational database management systems (RDBMS). In this webinar, MongoDB Field CTO Rick Houlihan will present benchmark results and real-world insights, demonstrating how MongoDB’s flexible and scalable database platform compares to PostgreSQL in addressing the complex demands of AI. Don't miss out! Sachin Chawla, Stewart Garrett, Abi Nourai, Steve Su, Aditya Nanda, Vivek Ganesh, Sachin Shinde, Bhavesh Punjabi, Céline Zuberek, Gabriel Woo, Joe Shin, Yan (Sunny) Yang, Joe Toh, Ban Selvakumar, Keagan Schmitz, Shelley Sansby #ArtificialIntelligence #DataAnalytics #Database

73

RAG architectures enhance #LLMs by integrating proprietary data, but maintaining data freshness poses challenges. 🤯 Continuous updates of vector embeddings ensure accuracy in AI models’ responses. ✅ The below solution uses MongoDB Atlas Stream Processing and MongoDB Atlas Vector Search to streamline the updating and searching of embeddings, simplifying the process for developers. 👇 https://lnkd.in/eNGxSp7k

32

1 Kommentar

📷 New on YouTube: In his 2023 #SAGconf session, Matthias Niehoff delves into the latest developments in modern data architectures, including Data Mesh, Data Lakehouse, and the integration of analytics with operational systems, offering insights into their practical implementation. 👀 Watch now: https://lnkd.in/eHs3EkZa #SoftwareArchitecture #SoftwareArchitectureGathering #SoftwareArchitectureConference #DataArchitecture #DataMesh #DataLakehouse #ELTPattern #CloudDataWarehouse #ModernDataStack #DataStrategy #SoftwareEngineering

8

One of the most common questions we hear is “what is the best way to migrate from Apache Kafka to #Confluent?” There is no one answer to this question – context is crucial to guarantee your data management platform is migrated over quickly and efficiently and in the best way to suit an organization's needs.   Fortunately, we’ve created a handbook that can guide you through the best practices for the initial stages and provide you with the resources to maintain your platform post-migration. Take a look: https://lnkd.in/gVsDcZ_3   #ApacheKafka #datastreaming #datamanagement

54

Today at Microsoft Build, GitHub announced support on stage for Copilot Extensions, including DataStax. 🙌 With the new service, developers can chat with their data held on Astra DB through the familiar GitHub Copilot Chat interface. Developers can describe their desired data outcomes in natural-language prompts, and Copilot translates those requests into database queries that the Astra DB data API will execute. You can learn more about the launch onstage right here 👇 https://ow.ly/pfW450S3Pqj #MSBuild #GenAI #AstraDB #GitHubCopilot

3

Redshift Serverless just became better with the GA of AI scaling and optimizations.

64

1 Kommentar

Tomorrow is the day, the Databricks Roundtable is taking place in Cologne.🚀 Join us for an exclusive hands-on session where Philipp Schwartenbeck will present a multi-agent system and Felix Xaver Krisch will demonstrate the complete lifecycle of a data product directly in Databricks! In the live demo, we will take you through the entire journey of a data project - starting with data exploration, followed by incremental data processing, workflow scheduling and scaling. We'll also show you how to implement CI/CD pipelines and monitoring tools to ensure seamless project delivery in Databricks. In the second live demo, we'll dive into the world of LLM-based multi-agent systems. See how autonomous collaboration between agents can solve complex tasks in real time, helping organisations improve the efficiency of their data processing and decision making! Find all the event details and register now... there are only 3 places left❗ 🔗https://lnkd.in/g_EyDuHp Looking forward to seeing you tomorrow! #databricks #dataengineering #AI #multiagent #LLM #cloudmigration #datagovernance #realtimedata #dataplatform #changemanagement

6

Really interesting read on the evolution of Data Warehouses, Data Lakes and Lake Houses and how Databricks and Snowflake are battling for the top spot. https://lnkd.in/egg4P5VG This is one of the more complete analysis I've seen on the evolution of the analytics space and it's big gorilla's. In the end, it gets into the AI discussion of big LLM's vs. small models and how SLM's will have a place in the analytics world, which I completely agree with.

9

1 Kommentar

Watch "Getting Started with Apache Paimon" and unlock the potential of #ApachePaimon! Paimon is an open-source streaming lakehouse platform that is a powerful tool for building real-time data pipelines and applications. It is a good choice for use cases that require high-speed data ingestion, change data tracking, and efficient real-time analytics. Benefits of using Paimon include: ✔️Reduced complexity: Paimon provides a unified platform for batch and streaming data processing, which can reduce the complexity of your data pipelines. ✔️Real-time insights: Paimon can be used to build real-time data pipelines that provide insights into your data as soon as it is generated. ✔️Scalability: Paimon is a scalable platform that can handle large volumes of data. Watch the video now to learn more! https://bit.ly/4bxs3Px

14

Watch "Getting Started with Apache Paimon" and unlock the potential of #ApachePaimon! Paimon is an open-source streaming lakehouse platform that is a powerful tool for building real-time data pipelines and applications. It is a good choice for use cases that require high-speed data ingestion, change data tracking, and efficient real-time analytics. Benefits of using Paimon include: ✔️Reduced complexity: Paimon provides a unified platform for batch and streaming data processing, which can reduce the complexity of your data pipelines. ✔️Real-time insights: Paimon can be used to build real-time data pipelines that provide insights into your data as soon as it is generated. ✔️Scalability: Paimon is a scalable platform that can handle large volumes of data. Watch the video now to learn more! https://bit.ly/3V6BtLk

4

Since we launched Redshift Streaming Ingestion with Kinesis Data Streams (KDS) and Managed Streaming for Apache Kafka (MSK) we have seen great adoption with thousands of customers ingesting data at very high rates with low latency directly to Redshift Managed Storage. One common request was to extend the ingestion capabilities to other Kafka sources. Today we are extending Redshift's Streaming Ingestion to support self-managed Apache Kafka on EC2 as well as partners, such as Confluent Cloud. Go and Ingest! https://lnkd.in/gpzC-JD2 #amazon #redshift #streaming #ingestion #kafka #confluent

200

1 Kommentar