WarpStream’s Post

Queue support in Kafka, enabled by share groups, will be available as part of #ApacheKafka 4.0 (currently in Early Access). Not only does it give your consumers a more flexible way to cooperate (without necessarily being tied to specific partitions), it’s suitable for applications that traditionally use queues—making Kafka a one-stop shop for all of your workloads. Learn more in the latest blog from Arun Singhal. Read it here ➡️ https://cnfl.io/3P3zXXx

🚀 Kafka 3.8.1 is here, and it’s all about smoother, smarter, and future-ready performance! From seamless KRaft mode security to faster MirrorMaker2 setups, this release is packed with essentials for the Kafka pros. Dive into optimized retention, precise metrics, and clear steps toward a ZooKeeper-free future. Read the official documentation: https://lnkd.in/dnYXNaKx Ready to level up? 📈 #Kafka #DataStreaming #TechUpdates #OpenSource

🥊 Properly assessing the cost of running Kafka can be a difficult task. Join this webinar and discover how to reduce your Kafka costs up to 60% and boost development velocity 🥊

Pictured: Apache Kafka users running away with savings in H2-2024. Confluent’s founders built Kafka - we understand both what drives Kafka costs and how to reduce them. Join our webinar June 18th to learn how to target these top costs. https://lnkd.in/ebsmVGEj

I came across this nice explanation of Apache Kafka KIP-392 (Fetch from follower) from Stanislav Kozlovski recently here https://lnkd.in/gQXppC3G. This got me thinking that I had come across it before, and sure enough I had written about it in 2020 in this blog with highlights from the Kafka Summit https://lnkd.in/gs3KFaHp But I was also puzzled about the claimed apparent lack of support in the cloud. I've checked with our Kafka team, and sure enough, we do support it (maybe the table could be updated) - but it's a feature that is available on request. Part of the reasoning behind this is that as the article mentions, passing on the cost benefits (the main reason you would use it, as it reduces cross-AZ networking costs) is not necessarily automatic. For NetApp Instaclustr managed Kafka the cost benefits are substantial for RIYOA (Run In Your Own Account) customers - e.g. our pricing guide here https://lnkd.in/gKBdyghi Out techops people do some magic on the Kafka cluster side to enable fetch from follower to work, and some other magic involves the RackWareReplicaSelector https://lnkd.in/g_iqfBjR and Kafka consumers will need to set client.rack to their local data centre ID https://lnkd.in/gH37P67Q - talk to our techops people who will guide you through the process. And there would likely be improved performance fetching from nearer brokers? Watch this space!

DigitalOcean Managed Kafka removes the complexities of self-managing Kafka at a flat-rate pricing model to ensure that customers avoid unexpected costs. Try Managed Kafka today! 🔗 https://do.co/48wIivm

Is your Apache Kafka spend hard to track and optimize? Join our webinar to uncover hidden costs and learn how to save up to 60% on your Kafka expenses. 📉💰 Register now for our webinar on Sept 25→ https://cnfl.io/3RA54vR

This infographic showcases the immense scale at which Kafka operates, powering critical infrastructure for industry giants like LinkedIn, Slack, and Netflix. Kafka's ability to handle billions of messages and massive data streams is truly impressive and essential for real-time data processing and analytics. However, integrating Kafka with other streaming solutions can enhance its capabilities even further. This is where the Lightstreamer Kafka Connector comes into play. The connector seamlessly bridges Kafka with Lightstreamer's high-performance real-time data distribution, enabling low-latency data delivery to remote mobile and web applications. By combining Kafka's robust messaging platform with Lightstreamer's real-time data streaming, organizations can achieve unparalleled scalability and responsiveness in their data pipelines, encompassing front-ends. 🔗 Learn more about the Lightstreamer Kafka Connector and how it can enhance your Kafka-based infrastructure: https://lnkd.in/dsqE6NYY #Kafka #StreamingData #RealTimeData #BigData #DataIntegration #Lightstreamer

Another lecture I presented focused on message-oriented middleware, specifically covering Kafka, RabbitMQ, and NATS.io. The lecture discussed the key features of each platform and included a comparison of their throughput and latency.