Myriam Fentanes

Observability by design has really helped our serverless teams avoid costly delays and unexpected problems. Daniel Furman and I are sharing tips for how you can keep your serverless functions healthy. We’re on a mission to share our serverless expertise so others don’t have to reinvent the serverless wheel. Check out our observability video and explore more about Capital One Software serverless-first approach: Link: https://lnkd.in/e67dJZ4r

21

1 Comment

A really nice read from Sai Maddali on the benefits of Tiered Storage with Amazon MSK. https://lnkd.in/gGNJ5nyt #ApacheKafka #Kafka #AWS #MSK #tieredstorage #streaming #storage

55

2 Comments

➡️Overview of Microservices Architecture: https://hubs.li/Q02NY0-g0 ➡️Unlike traditional monolithic architectures where an entire application is built as a single unit, #microservices break down #applications into smaller, #autonomous services that can be developed, deployed, and scaled independently. #Cloud #IoT #TechTalk #Innovation #TechBlog

13

Redis has been a popular choice by many, but several organizations have found greater success in efficiency, reducing costs, and performance with alternative solutions. Through these six real-world case studies, learn how companies achieved remarkable improvements by replacing #Redis. #Aerospike https://gag.gl/Q1XDKE

1

The latest workshop on telemetry pipelines by Chronosphere's Eric D. Schabell! In this series, Eric dissects #FluentBit, guiding you through installation, configuration, and the crucial phases of data management. Join as we tackle the challenge of backpressure, understanding its impact on pipeline performance, and more: https://lnkd.in/eRHRJwNc

10

One of the most common questions we hear is “what is the best way to migrate from Apache Kafka to #Confluent?” There is no one answer to this question – context is crucial to guarantee your data management platform is migrated over quickly and efficiently and in the best way to suit an organization's needs.   Fortunately, we’ve created a handbook that can guide you through the best practices for the initial stages and provide you with the resources to maintain your platform post-migration. Take a look: https://lnkd.in/gVsDcZ_3   #ApacheKafka #datastreaming #datamanagement

54

Using the embedded search capabilities of #ApacheLucene with #ApacheGroovy to search for The Apache Software Foundation project name references (and emojis): https://lnkd.in/g4kGrp5v #groovylang

17

Incredibly proud of the latest Kinesis Client Library version (3.0) which, when compared to previous KCL versions, reduces compute costs by up to 33%. KCL 3.0 introduces an enhanced load balancing algorithm that continuously monitors resource utilization of the stream processing workers and automatically redistributes the load from over-utilized workers to other underutilized workers. This ensures even CPU utilization across workers and removes the need to over-provision the stream processing compute workers which reduces cost. Additionally, KCL 3.0 is built with the AWS SDK for Java 2.x for improved performance and security features, fully removing the dependency on the AWS SDK for Java 1.x. #Kinesis #DataStreaming #KCL

94

Check out this session by Marc Klefter from #NDCOslo! He presented: 🔍 An overview of the #Dapr architecture ☁️ Practical considerations for employing Dapr in #cloudnative applications 💡 Implications for application and platform engineering teams working on distributed applications: https://diagrid.ws/4ekxbYW

8

💻 #EdgeComputing is transforming how IT resources are distributed, extending from the data center to virtually anywhere computation is needed. Overcome scaling and integration challenges with our Gorilla Guide. Discover ✅ how to master seamless scaling and ✅ explore the robust capabilities of #Kubernetes, while diving into the modernization of industrial applications for the future. 📩 Download the Gorilla Guide: https://okt.to/vQ3c9O

1

#messagebroker vs. #datastreaming - trade-offs, integration and migration scenarios from #JMS, IBM MQ, TIBCO, or ActiveMQ to #apachekafka A Message broker has very different characteristics and use cases than a data streaming platform like #kafka. Data integration, processing, governance, and security must be reliable and scalable across the business process. This blog post explores the capabilities of message brokers, the relation to the JMS standard, trade-offs compared to data streaming with Apache Kafka, and typical integration and migration scenarios. A case study explores the migration from IBM MQ to Apache Kafka. The last section contains a complete slide deck that covers all these aspects in more detail. https://lnkd.in/ev9neiw7

92

5 Comments

Users consistently tell us that managing the flow of state data in #stepfunctions workflows is hard. We've improved over the years, but this is a game-changer. JSONata is powerful & intuitive. Variables simplify by decoupling states. Can't wait to see what you build! https://lnkd.in/gf6QShDp

23

3 Comments

Tiered storage for Kafka Kafka cluster storage is typically scaled by adding more broker nodes to the cluster. But this also adds needless memory and CPUs to the cluster, making overall storage cost less efficient compared to storing the older data in external storage. A larger cluster with more nodes also adds to the deployment complexity and increases operational costs because of the tight coupling of storage and processing. So, it brings several issues related to scalability, efficiency, and operations. In the tiered storage approach, Kafka cluster is configured with two tiers of storage - local and remote. The local tier is the same as the current Kafka that uses the local disks on the Kafka brokers to store the log segments. The new remote tier uses systems, such as HDFS or S3 to store the completed log segments. Two separate retention periods are defined corresponding to each of the tiers. With remote tier enabled, the retention period for the local tier can be significantly reduced from days to few hours. The retention period for remote tier can be much longer, days, or even months. When a log segment is rolled on the local tier, it is copied to the remote tier along with the corresponding indexes. Latency sensitive applications perform tail reads and are served from local tier leveraging the existing Kafka mechanism of efficiently using page cache to serve the data. Backfill and other applications recovering from a failure that needs data older than what is in the local tier are served from the remote tier. #kafka #storage #cost https://lnkd.in/ehu2Fw2v

2

At KubeCon, Percona CEO Ann Schlemmer shared insights on Kubernetes' evolution from stateless to stateful workloads, tackling data and database challenges. She highlighted Percona’s contributions, including the launch of Everest, a cloud-native, open-source database alternative. Ann also discussed the growth of the Data on Kubernetes (DoK) community, emphasizing the importance of open-source contributions and the ongoing performance and security challenges in data management. https://bit.ly/3BgRaK4 via TFiR #KubeCon #Kubernetes #OpenSource

7

This blog explores the importance of monitoring microservice applications running on Kubernetes and the key metrics to track for optimal performance. Effective #observability empowers you to identify problems early, fostering reliability, efficient scaling, and adherence to SLAs. Crucial metrics include cluster availability, pod health, service availability, and node health. Leverage IBM Instana to monitor these metrics effortlessly. #cloudnative #apm #GenAI https://lnkd.in/gB4scgsa

23

Check out more on Uber's Kafaka infrastructure and the technologies supporting them. Thanks to Matt Saunders for putting this article together. Stanislav Kozlovski meshIQ The Apache Software FoundationApache Spark https://lnkd.in/eeUmcqtD

14

3 Comments