Consdata’s Post

Amazon Managed Streaming for Apache Kafka (Amazon MSK) is a fully managed, highly available service that makes it easy for you to build and run applications that use Apache Kafka. Here is a post on streaming data with Amazon MSK and #Couchbase Capella

Amazon MSK Replicator now supports a new configuration that enables you to preserve original Kafka topic names while replicating streaming data across Amazon Managed Streaming for Apache Kafka (Amazon MSK) clusters. Amazon MSK Replicator is a feature of Amazon MSK that lets you reliably replicate data across MSK clusters in the same or different AWS region(s) with just a few clicks.

Last week, we announced two exciting new features for Lambda event source mappings that enhance performance and observability: ESM metrics and provisioned mode for Kafka pollers. Join me at re:Invent for SVS217, where I’ll dive into strategies for improving throughput and monitoring of serverless streaming workloads. I’ll cover a range of techniques, both ESM-agnostic and ESM-specific, and explore new capabilities in detail. #lambda #kafka #streaming #throughput #observability #esm #aws #kinesis #reinvent

A really nice read from Sai Maddali on the benefits of Tiered Storage with Amazon MSK. https://lnkd.in/gGNJ5nyt #ApacheKafka #Kafka #AWS #MSK #tieredstorage #streaming #storage

What to do when Kafka generates a lot of cross-AZ traffic fees when deployed on AWS with multiple AZs? You will have the answer after reading this blog post. #automq #apachekafka #streaming #aws #s3 https://lnkd.in/dAcrSjau

The expensive cross-AZ traffic fees on AWS are essentially a flexible tool prepared for its sales. It can gain competitiveness by adjusting its discounts when competing with third-party vendors. Now, AutoMQ has helped Kafka break this shackle. #kafka #streaming #automq

In a pivotal move for the video industry, Amazon Web Services (AWS) has launched ServerlessVideo, showcasing the power and flexibility of serverless architecture for both live and on-demand video streaming. In his recent article, my colleague Max Kalmykov explores the serverless approach and how AWS is shaping the future of video streaming: https://lnkd.in/eHe8hjbu  #DataArt #DataArtMedia #videoindustry #AWS 

Finally, got this finished! What would be the potential use cases? There are a ton of requirements for messaging/queues/data-streaming. What do you think are most fitting for AWS Lambda based data streaming? What would be needed to make this AWS Lambda to AWS Lambda streaming with SQL processing in each step?

AWS quite recently began supporting response streaming via Lambda functions. While not everyone needs streaming responses, having the flexibility to use them is a game changer. The primary value proposition of a streaming response is to lower the Time To First Byte aka TTFB. TTFB is a response latency metric and one of the key factors in determining a server's performance, especially when dealing with dynamic content that can be large or might require more computing. This solution leverages AWS CloudFront as a CDN in front of a Lambda function that's accessible via a function URL. The CDN allows us to access the Lambda function via a custom domain, allows us to customize the request forwarding behaviour, response caching behaviour, and also helps in lowering the response time and latency by leveraging Cloudfront's edge-optimised networking. Give it a try and let me know your thoughts about this implementation and how you'd potentially use this for your use cases. #aws #cdk #lambda #cdn #cloudfront #iac

https://lnkd.in/dTmZdQyH If anybody is struggling using FastAPI with Amazon AWS S3 Storage or CloudFlare R2 S3 compatible bucket storage, here is a working example. The example shows how to integrate S3 Bucket ( great for streaming video CDN ) into your SqlAlchemy Admin. I hope this helps someone since there wasn't any good examples.