Hans Li

How to learn #SystemDesign 👇 📌 𝐒𝐲𝐬𝐭𝐞𝐦 𝐃𝐞𝐬𝐢𝐠𝐧 𝐊𝐞𝐲 𝐂𝐨𝐧𝐜𝐞𝐩𝐭𝐬 - Scalability: lnkd.in/gpge_z76 - Latency vs Throughput: lnkd.in/g_amhAtN - CAP Theorem: lnkd.in/g3hmVamx - ACID Transactions: lnkd.in/gMe2JqaF - Consistent Hashing: lnkd.in/gd3eAQKA - Rate Limiting: lnkd.in/gWsTDR3m - API Design: lnkd.in/ghYzrr8q - Strong vs Eventual Consistency: lnkd.in/gJ-uXQXZ - Synchronous vs. asynchronous communications: lnkd.in/g4EqcckR - REST vs RPC: lnkd.in/gN__zcAB - Batch Processing vs Stream Processing: lnkd.in/gaAnP_fT - Fault Tolerance: lnkd.in/dVJ6n3wA - Consensus Algorithms: lnkd.in/ggc3tFbr - Gossip Protocol: lnkd.in/gfPMtrJZ - Service Discovery: lnkd.in/gjnrYkyF - Disaster Recovery: lnkd.in/g8rnr3V3 - Distributed Tracing: lnkd.in/d6r5RdXG - Top 15 Tradeoffs: lnkd.in/gnM8QC-z 🛠️ 𝐒𝐲𝐬𝐭𝐞𝐦 𝐃𝐞𝐬𝐢𝐠𝐧 𝐁𝐮𝐢𝐥𝐝𝐢𝐧𝐠 𝐁𝐥𝐨𝐜𝐤𝐬 - Horizontal vs Vertical Scaling: lnkd.in/gAH2e9du - Databases: lnkd.in/gti8gjpz - Content Delivery Network (CDN): lnkd.in/gjJrEJeH - Domain Name System (DNS): lnkd.in/gkMcZW8V - Caching: lnkd.in/gC9piQbJ - Distributed Caching: lnkd.in/g7WKydNg - Load Balancing: lnkd.in/gQaa8sXK - SQL vs NoSQL: lnkd.in/g3WC_yxn - Database Indexes: lnkd.in/dGnZiNmM - HeartBeats: lnkd.in/gfb9-hpN - Circuit Breaker: lnkd.in/gCxyFzKm - Idempotency: lnkd.in/gPm6EtKJ - Database Scaling: lnkd.in/gAXpSyWQ - Data Replication: lnkd.in/gVAJxTpS - Data Redundancy: lnkd.in/gNN7TF7n - Database Sharding: lnkd.in/gRHb-67m - Failover: lnkd.in/dihZ-cEG - Proxy Server: lnkd.in/gi8KnKS6 - Message Queues: lnkd.in/gTzY6uk8 - WebSockets: lnkd.in/g76Gv2KQ - Bloom Filters: lnkd.in/dt4QbSUz - API Gateway: lnkd.in/gnsJGJaM - Distributed Locking: lnkd.in/gRxNJwWE - Checksum: lnkd.in/gCTa4DrS 🖇️ 𝐒𝐲𝐬𝐭𝐞𝐦 𝐃𝐞𝐬𝐢𝐠𝐧 𝐀𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐚𝐥 𝐏𝐚𝐭𝐭𝐞𝐫𝐧𝐬 - Client-Server Architecture: lnkd.in/dAARQYzq - Microservices Architecture: lnkd.in/gFXUrz_T - Serverless Architecture: lnkd.in/gQNAXKkb - Event-Driven Architecture: lnkd.in/dp8CPvey - Peer-to-Peer (P2P) Architecture: lnkd.in/di32HDu3

8

ZGC: Generational Mode is proposed to be the default ZGC mode for JDK 23 :-) Highly recommending you try ZGC!       https://lnkd.in/gAJC6GYY #zgc #java #openJDK #GC

34

1 Comment

If you are a UX/UI designer, product manager, front-end engineer, executive, or just someone who wants to build an AWS cloud vocabulary and better understand what your technical team is talking about - this book is for you. "AWS for Non-Engineers" by Hiroko Nishimura captured my attention when I sorted the pile of e-books purchased over the past two years on sales. It is easy to read and relatively short - 174 pages total, but if you are not planning to get certified as a cloud practitioner, skip Chapter 7. You can read merely 121 pages in a couple of evenings. I briefly read all the remaining 6 chapters (Introduction to Cloud Computing, Cloud Concepts, AWS Global Infrastructure, Core Services, Security and Compliance, Billing and Pricing) and definitely can recommend it to non-engineers. Suppose you wonder why I decided to read it. In that case, the answer is simple - it helps to view the IT world from a different angle and ease future communication with non-technical decision-makers. The book may get too technical for a complete beginner, though. I bet you may need to revisit Chapter 4 "Core AWS services" a few times because the chance that you forget, for example, what is CloudTrail vs CloudWatch, is high - too many services mentioned to digest in one go. If you feel that way, focus on the first two chapters. You will still get a great boost in understanding.

9

2 Comments

Tiered storage for Kafka Kafka cluster storage is typically scaled by adding more broker nodes to the cluster. But this also adds needless memory and CPUs to the cluster, making overall storage cost less efficient compared to storing the older data in external storage. A larger cluster with more nodes also adds to the deployment complexity and increases operational costs because of the tight coupling of storage and processing. So, it brings several issues related to scalability, efficiency, and operations. In the tiered storage approach, Kafka cluster is configured with two tiers of storage - local and remote. The local tier is the same as the current Kafka that uses the local disks on the Kafka brokers to store the log segments. The new remote tier uses systems, such as HDFS or S3 to store the completed log segments. Two separate retention periods are defined corresponding to each of the tiers. With remote tier enabled, the retention period for the local tier can be significantly reduced from days to few hours. The retention period for remote tier can be much longer, days, or even months. When a log segment is rolled on the local tier, it is copied to the remote tier along with the corresponding indexes. Latency sensitive applications perform tail reads and are served from local tier leveraging the existing Kafka mechanism of efficiently using page cache to serve the data. Backfill and other applications recovering from a failure that needs data older than what is in the local tier are served from the remote tier. #kafka #storage #cost https://lnkd.in/ehu2Fw2v

2

🚀 Exciting News Db2 Users! 🚀 ⌚ Amazon Web Services (AWS) has introduced on-demand Db2 licenses through AWS Marketplace! 🕺 This innovative solution enables Db2 users to procure database licenses for RDS Db2 workloads with ease. #AWS #AmazonRDS #DB2 #DatabaseManagement #Innovation #CloudComputing #TechNews #AWSUpdates

9

Thrilled to enable 🔎 Gemini 'Grounding' support 👉 https://lnkd.in/gUKu3bv4 (+4 more updates 👇) 🛠️ Azure default api version fix 💪 'additional_drop_param' support for Embedding+Img Gen - https://lnkd.in/gZ8ZwccW 🗣️ Speech to text - 'srt' support - https://lnkd.in/gE69asjN 🎉 'IP Address' filtering - https://lnkd.in/gTu9q-iB

6

Cloud Run GPU makes it so easy to host Llama 3.2 models, check this out!

6

### Work-Life Harmony You might have heard the term "work-life balance" and, in fact, used it often when looking for a new job. Asking the recruiter about the work culture and whether there is work-life balance is common. Recently, in one of the interviews, I heard Jeff Bezos talking about work-life harmony instead of work-life balance. At first, it was difficult to grasp what work-life harmony means. I would have also not understood it if I had not felt it. We have this standard operating procedure, or let's say what 90% of us think constitutes a happy life: spend 8 hours at work, come home, enjoy time with family, have good food, go to bed early, get a good night's sleep, and repeat. All this is true when you are happy with the 8 hours you are spending each day at work. From experience..if you are not happy doing the work you spend most of your time on, then you won't be happy. You won't feel like picking up your brush to paint the canvas, won't be happy at the dinner table, or work on your blog that you have been thinking about for a long time. If I had heard about work-life harmony a few years back, I would have never understood or felt what it means. That’s why the saying goes: if you make your passion your profession, you don’t have to work another day in your life. We should enjoy what we are doing, and that is what gives work-life balance. When you enjoy your work, there is no extra effort needed to sometimes put in extra hours or work on weekends. That said, we should definitely have time for our family and hobbies, but those times should be quality time instead of just being present. Create your space; create your work-life harmony. Keep searching until you find it. #worklifeharmony #happywithwork #behappy

8

Istio Service Mesh: great to see Ambient moving up to Beta, delta xDS becoming the default, and api-gateway becoming a first class citizen in Istio. https://lnkd.in/gYPmnTuj

34

Valkey: A Robust, Redis-Alternative for Modern Applications! Redis is no longer fully open source, prompting major tech players such as Amazon, Google, Huawei, and Ericsson to explore alternatives. This shift has led to growing support for ValKey, a flexible and secure open-source solution, increasingly recognized as a strong contender in the space. ValKey 8.0's Groundbreaking features: 1. Introduced dual-channel replication and migration state tracking, minimizing data loss during node migrations or rearrangements. 2. Enhanced throughput to 1.2 million requests per second using multi-core and asynchronous I/O threading, surpassing Redis's single-threaded approach. 3. Added per-slot metrics infrastructure, enabling granular performance insights to optimize resource allocation and load balancing. AWS has fully integrated ValKey into its services, including ElastiCache and MemoryDB, while significantly lowering costs: - 33% reduction for ElastiCache Serverless, - 20% for node-based ElastiCache, and - 30% for MemoryDB. The database landscape is evolving, and transitioning to ValKey is a compelling option to consider, whether you're operating on AWS or another platform. Share your suggestions or experiences with in-memory datastores designed to deliver lightning-fast performance and handle massive throughput effectively. https://lnkd.in/gPZjHX3W #Valkey #Redis #InMemoryDB #TechInnovation #FlexibleLicensing #DataSecurity #ModularDesign #DataPersistence #CloudTech

11

For anyone involved in large-scale data processing, particularly those considering Apache Flink or Google Cloud Dataflow. The article gives guideline on how to setup a fair benchmark consideration as well as potential optimization that can be done on the pipeline: Real-world Benchmark: It offers a practical, real-world benchmark study conducted by Yahoo, providing valuable insights into the cost and performance of these platforms under specific use cases. This can help organizations make informed decisions about their technology stack. Fair Comparison Methodology: The blog post outlines the methodology used to ensure a fair comparison between Flink and Dataflow, emphasizing the importance of controlling variables and establishing a consistent testing environment. This level of rigor adds credibility to the findings. Cost Optimization Focus: It focuses on cost-effectiveness, a critical factor for businesses. The post helps readers understand how to minimize costs on both platforms.

1

VPC with zone subnets, alb, nat and igw https://lnkd.in/gZxkgdBx

5

Day 51/100 of 100-day DSA Q.1 https://lnkd.in/gwR9ext7 Q.2 https://lnkd.in/gerrh9kw #DataStructures #Programming #Coding #Tech #ComputerScience #SoftwareEngineering #LeetCode #CodeOptimization #TechCommunity #ProblemSolving #Cpp #ProgrammingChallenges #TechSkills #100DaysOfCode #100DaysOfDSA #DSAChallenge #DailyCoding #CodeEveryday #CodingChallenge #Backtracking #Stacks #Queues #SlidingWindow #TwoPointers

5

Scaling up Kafka and making sure it's performing at its utmost efficiency is a holy grail for enterprise software architecture. Kafka is designed for infinite scale but only if you design and adapt the right Cluster configuration based on your growing needs. These are a few metrics to watch out for.

21

Managing and delivering a robust CI/CD pipeline at scale is a complex and evolving challenge. The term “CI/CD” holds varied meanings across different work cultures, reflecting diverse priorities and practices. As the technology landscape continues to grow, CI/CD has become just one component of the broader and increasingly significant developer enablement ecosystem. Read more at https://lnkd.in/gPwGFrTk

12

These instances also offer up to 50 Gbps network bandwidth and up to 40 Gbps Amazon Elastic Block Storage (Amazon EBS) bandwidth compared to up to 30 Gbps network bandwidth and up to 20 Gbps Amazon EBS bandwidth on Graviton3-based instances

24

https://lnkd.in/g_5WieaK Optimizing MongoDB Checkpointing for Performance #MongoDB #MongoDBTricks #MongoDBPerformance #DBA #NoSQL

6