Amit Pandya

📣 [New Blog] One small resource configuration change to use customer-managed KMS key, a giant leap towards secured workflows for customers using AWS Step Functions. As promised, here is the blog and sample application that elaborates the integration of customer-managed AWS KMS key in AWS Step Functions: https://lnkd.in/g3DJqUPj

39

We're back with another edition of the Observability Engineering London meetup 🤩. This time, we'll discuss how to get the most out of AWS OpenSearch for observability. Join us on Tuesday, June 4th, when Eugene Tolbakov will delve into the process undertaken by the Observability team at Chase UK to manage AWS OpenSearch clusters effectively. Utilizing Infrastructure as Code(Terraform), they have streamlined cluster management for efficiency and ease. He'll elaborate on their approach for defining index templates and patterns, configuring roles, and leveraging ingestion pipelines to streamline cluster management. Furthermore, Eugene will outline the enhancements they've implemented to ensure a stable platform and enhance the overall Observability experience and share key insights and learnings from their journey toward operational excellence with AWS OpenSearch management. 👾 Gameplan: 6:15 Welcome 6:30 Eugene Tolbakov | Delicacies of Observability: AWS OpenSearch Cluster from 'rare' to 'well-done 7:00 Q&A 7:20 Networking, Food & Drinks 8:15 Wrap up and head to the pub downstairs to keep the conversation going. #observability #aws #sre #devops #engineering

6

This week's issue highlights the deprecation of Istio's in-cluster operator, updates in Kubernetes 1.31 for PodAffinity and resource allocation, and the announcement of Alibaba's Canal Mesh. Also covered are the upcoming KubeCon schedule and a critical ingress-nginx vulnerability. Featured tools include Kardinal for ephemeral development environments, Karpenter for Kubernetes autoscaling, and LocalStack for AWS integration testing.

4

What is DevOps really about? Is it about delivering IT faster and adding more value? I think Simona Zarskute's blog captures it quite well. DevOps affects your entire business operation. https://lnkd.in/eB3z7qVD #devops #dasadevops #leadership #brunelagilecommunity

2

A relevant fear or an overhyped assertion? What do you think? “A lot of companies liked Hashicorp tooling because it was inherently cloud-agnostic; for multi-cloud organisations, having a common configuration language in Terraform or a secret management solution in Vault was preferable to learning each in AWS, Azure, Google Cloud, or the multitude of other providers.“ Read “The beginning of the end for Terraform?“ by Alistair Grew on Medium: https://lnkd.in/dBkNHWw2

3

𝐄𝐦𝐛𝐫𝐚𝐜𝐞 𝐭𝐡𝐞 𝐂𝐡𝐚𝐨𝐬: 𝐁𝐮𝐢𝐥𝐝𝐢𝐧𝐠 𝐑𝐞𝐬𝐢𝐥𝐢𝐞𝐧𝐭 𝐂𝐥𝐨𝐮𝐝 𝐒𝐲𝐬𝐭𝐞𝐦𝐬 This post explores chaos architecture, a counterintuitive approach to building stronger cloud systems. Presented by Adrian Cockcroft, it challenges the traditional focus on redundancy and failover. 𝐖𝐡𝐲 𝐂𝐡𝐚𝐨𝐬? Today's complex, distributed cloud environments demand a new approach. Cockcroft argues that planned disruptions, or "injected failures," expose weaknesses before they cause real outages. 𝐁𝐮𝐢𝐥𝐝𝐢𝐧𝐠 𝐑𝐞𝐬𝐢𝐥𝐢𝐞𝐧𝐜𝐞 𝐓𝐡𝐫𝐨𝐮𝐠𝐡 𝐂𝐡𝐚𝐨𝐬 Chaos architecture isn't about creating havoc. It's about proactively testing your system's ability to handle disruptions. Here's how it works: - 𝘋𝘪𝘴𝘵𝘳𝘪𝘣𝘶𝘵𝘦𝘥 𝘐𝘯𝘧𝘳𝘢𝘴𝘵𝘳𝘶𝘤𝘵𝘶𝘳𝘦: Spread your system across multiple locations to avoid single points of failure. - 𝘐𝘯𝘵𝘦𝘭𝘭𝘪𝘨𝘦𝘯𝘵 𝘚𝘸𝘪𝘵𝘤𝘩𝘪𝘯𝘨: Automate failover to ensure seamless transitions during disruptions. - 𝘍𝘢𝘶𝘭𝘵-𝘛𝘰𝘭𝘦𝘳𝘢𝘯𝘵 𝘈𝘱𝘱𝘭𝘪𝘤𝘢𝘵𝘪𝘰𝘯𝘴: Design applications to gracefully handle unexpected errors. - A Culture of Experimentation: Foster a team that embraces learning from failures. 𝐊𝐞𝐲 𝐋𝐚𝐲𝐞𝐫𝐬 𝐨𝐟 𝐂𝐡𝐚𝐨𝐬 𝐀𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞: - Fortify Your Foundation: Design infrastructure and services to avoid single points of failure. - Automate Everything: Make switching and interconnections reliable and automated. - Build Fault-Tolerant Apps: Develop applications that handle unexpected errors gracefully. - Empower Your People: Train your team to effectively respond to outages. Chaos Engineering Teams: The Architects of Resilience These dedicated teams build tools and processes to test your system's resilience. This includes: - Game Days: Simulate outages to identify and fix weaknesses. - Failure Injection Tools: Introduce controlled disruptions to test system behavior. - Automated Chaos Platforms: Streamline the chaos engineering process. Finding Your Balance: Risk Tolerance Organizations need to determine their risk tolerance. Is downtime or data inconsistency more critical? This helps define how aggressively you can embrace failures for testing. Learning from Every Outage Chaos engineering is part of a larger incident lifecycle: - Mitigate the Impact: Minimize the damage caused by an outage. - Restore Functionality: Bring the system back online as quickly as possible. - Prevent Future Issues: Adapt your system to prevent similar outages. Failures: A System-Wide Issue Components and human error aren't always the culprits. Sometimes, failures expose a lack of system resilience. Chaos architecture helps you build in enough redundancy to absorb these errors. Test Your Assumptions: Hypothesis Testing Chaos engineering is about formulating and testing hypotheses around your system's resilience. By intentionally injecting failures, you identify and fix potential issues before they become real problems. Embrace the chaos to build stronger, more resilient cloud systems!

1

In this article we create an AWS MSK Kafka resource using AWS IAM Roles and Policies to authenticate user access. We then create an Aerospike Database cluster and insert some sample messages which in turn get shipped to AWS Kafka via the Aerospike Kafka Source connector. The article is focused around using IAM to authenticate clients and will guide the reader step by step on how to achieve this. https://lnkd.in/ebwJQ5bN

2

🌟 **Achieving High Availability in AWS with NAT Gateways!** 🌟 In my latest video, I explore the process of **setting up multiple NAT gateways** in AWS to enhance **high availability** and **fault tolerance** within your cloud architecture. By deploying NAT gateways across multiple Availability Zones, we can ensure that our applications remain resilient and operational, even in the face of potential failures. This strategy is crucial for maintaining seamless connectivity and performance in your VPC. Join me as I walk you through this step-by-step lab guide to fortify your AWS infrastructure! 💡 #AWS #NATGateway #HighAvailability #FaultTolerance #CloudArchitecture #CloudComputing #Networking

16

This guide demonstrates using Terraform's lifecycle meta-argument to ignore changes to parameter values, enabling seamless integration with other processes that modify these values. https://lnkd.in/eK8QYnVv

1

Yahoo’s Kubernetes Journey from on-Prem to Multi-Cloud   🎯 Key Innovations:     - Migrated from on-prem infrastructure to a multi-cloud ecosystem with GCP and AWS.     - Developed ‘Omega’ interface simplifying app deployment to Kubernetes clusters.     - Introduced ‘Cadence’ tool for seamless cluster creation across multiple clouds.    💡 Notable Features:     - **Omega YAML**: A simplified deployment template ensuring quick app setups.     - **Cadence Tool**: A Golang-based tool for managing EKS and GKE workloads without deep cloud provider knowledge.     - **Node Manager Operator**: Automates Kubernetes version upgrades while maintaining workloads.    🛠️ Perfect for:     - Cloud Engineers     - Kubernetes Administrators     - DevOps Teams     - Platform Engineers     - Organizations transitioning to multi-cloud environments    ⚡️ Impact:     - Enhanced deployment efficiency by streamlining application setups.     - Achieved significant cost reduction for operations in cloud environments.     - Facilitated rapid responses to infrastructure changes and reduced upgrade times from months to weeks.    🔍 Preview of the Talk:     Nandhakumar Venkatachalam and Payal Patel from Yahoo discuss the company's transition from a traditional on-prem environment to a cutting-edge multi-cloud strategy using Kubernetes. They delve into the challenges faced, such as networking and security, while showcasing breakthrough tools like Omega and Cadence. The talk emphasizes how these innovations enhance user experiences and streamline operations. Participants will gain valuable insights into effective cloud strategies and operational efficiencies in Kubernetes environments. Watch the full session: https://lnkd.in/gcZS--rY

1

What a fantastic evening, great talks by leading contributors to Spring Projects. - "Scaling APIs with Spring Cloud" by David Syer - "Bootiful Spring Boot 3" by Josh Long Thank you both for 1-2-1 discussions on future of Spring Boot, Java 21/22 How could we partner to learn from each others' experiences and implement blazing fast Java Applications using Spring Boot, Modulith, Event Driven, Gen AI and Virtual Threads. #springboot #java21 #microservices #eventdriven #genai

120

2 Comments

Struggling with legacy and new VM workloads? We’ve got you covered on how to seamlessly run VM workloads on Kubernetes. Watch this on-demand webinar as our senior solution architect, Kevin Ng, shows you how easy it is to run VM workloads on Kubernetes with KubeVirt, featuring real world use cases and a demo. Watch the video now! https://bit.ly/3zrNlk1

11

This article describes how AWS evolved its block storage system (EBS). It's a brilliant, high-level description of the challenges that AWS faced. I was particularly impressed with how they tackled the problem using multiple teams, each specializing in one part of EBS. Often such problems are relegated as "hardware" problems and teams try to speed up their data access patterns instead. It's good to get a taste of the algorithms that power SSDs and disks 😁 https://lnkd.in/dBtfDZGF If you want to know more about tradeoffs in distributed systems, try "System Design Simplified". https://lnkd.in/dqYv-9Tf Cheers!

111

Scaling Strategies on AWS EKS: Understanding HPA, VPA, and Cluster Autoscaler #WorkSmartWithK8s #kubernetes #eks #autoscaling #hpa #vpa #clusterautoscaler https://lnkd.in/eaGQKUGf

18

Usage of eBPF with Real-Time SSL/TLS Encrypted traffic. This is really powerful tool!

12

1 Comment