In this third, extensively illustrated post of his windowing blog series, Bill Bejeck demonstrates sliding windows in Kafka Streams, and their logical equivalent in #FlinkSQL, OVER aggregations. Learn about use cases, like running average calculations, and more, here ➡️ https://cnfl.io/3W7N7qS
Confluent’s Post
More Relevant Posts
-
Lake architecture Spark Apache Delta Lake
Fabric Runtime 1.3, based on Apache Spark 3.5 and Delta Lake 3.0, has been released to all production regions as an Experimental Preview! You can now use it for early access to new features and improvements. Check the documentation: https://lnkd.in/dJFG9uE8 🙌 👏👏👏 Ashit Gosalia Justyna Lucznik Ajith S Akshay Tayal ABHISHEK UPMANYU Christopher Finlan Raghav Sidhanti Daniel Coelho Faruk Celik Mimi Gentz Andrew Fogarty
-
Looking for practical #ApacheFlink use case examples? Our go-to Flink experts, David Anderson, Martijn Visser, and Chesnay Schepler, have curated a collection of best practice solutions that showcase common patterns and use cases for Flink. These recipe examples can help you tackle a variety of common on-premise challenges such as: ⚪ Deserializing JSON from Kafka ⚪ Joining and deduplicating data ⚪ Doing complex event processing ⚪ Using session windows … and many more! Take a look and share what you are building with us!
-
You can start using Fabric Runtime 1.3 Runtime as Experimental Public Preview. Please check our documentation for the limitations : https://lnkd.in/d885PJ4d #msftadvocate #microsoftfabric #apachespark #synapsespark #spark #deltalake
Fabric Runtime 1.3, based on Apache Spark 3.5 and Delta Lake 3.0, has been released to all production regions as an Experimental Preview! You can now use it for early access to new features and improvements. Check the documentation: https://lnkd.in/dJFG9uE8 🙌 👏👏👏 Ashit Gosalia Justyna Lucznik Ajith S Akshay Tayal ABHISHEK UPMANYU Christopher Finlan Raghav Sidhanti Daniel Coelho Faruk Celik Mimi Gentz Andrew Fogarty
-
🚀 Sharing a quick-read blog post on Apache Spark Unified Memory! This post helps you understand concepts like memory pools, reserved memory, and overhead memory. Check it out for insights on optimizing memory usage and improving Spark application performance. 📝 Read more - https://lnkd.in/ejh7EBZr #ApacheSpark #DataEngineering #MemoryManagement
-
-
This article will teach you how to install and manage Apache Kafka on Kubernetes with the Strimzi operator. More: https://lnkd.in/gzGK4Xdz
-
-
Practice processing with #FlinkSQL by completing a hands-on exercise that teaches you how to deduplicate events from an Apache Kafka stream. You'll learn about sub-queries, the PARTITION BY & ORDER BY clauses, & the ROW_NUMBER() function. Full test steps are included. Check it out here ➡️ https://cnfl.io/4bjhhfr
-
-
🥳 #ApacheFlink 1.19 is here! Get a summary of the latest updates with David Anderson. AF1.19 introduces: ✔️ Legacy feature deprecations ✔️ Improved windowing support in Flink SQL ✔️ Better observability of Flink’s checkpoint and recovery mechanisms ✔️ And more! 🎬 Watch the video for release highlights:
Apache Flink 1.19 - Deprecations, New Features, and Improvements
https://www.youtube.com/
-
Sr. Software Engineer | Big Data & AWS Expert | Apache Hudi Specialist | Spark & AWS Glue| Data Lake Specialist | YouTuber
How to use Flink with Hive MetaStore and Minio and Create Iceberg tables | Docker | Hands on Labs Exercise files https://lnkd.in/eKK2HhGe To InterOperate between LakeHouse format https://xtable.apache.org/
-
This is how spark connect works! #spark #sparkconnect #clientserver #dataengineering #architecture
Spark Connect is built with a bunch of cool open technologies. * Protocol Buffers (protobuf) is used to encode unresolved logical plans that are sent to the Spark Server * Apache Arrow is used to send results as record batches from the Spark Server to the Client * gRPC is used to connect the Client and the Spark Server #apachespark Connect uses a lot of the same technologies and has a similar architecture as Apache Arrow Flight RPC.
-
More from this author
-
How are innovators across financial services turning often overwhelming volumes of data into value—both for their businesses and their customers?
-
Data in Motion Tour 2023 : Comment réinventer Apache Kafka® à l'ère du streaming de données avec Confluent
-
Digital-First Startups Have a Unique Path to Streaming Success