Archit Garg

Top learnings from my discussion with Kishore Gopalakrishna - Language choice for system development should balance team expertise, hiring potential, and ecosystem maturity. A well-implemented system in a familiar language often outperforms a poorly implemented one in a "faster" language. - Distributed systems are complex - they can fail in unexpected ways, making your own computer useless. This challenge is often underestimated. Understand the failure modes. - When building new systems, design for the long-term (10 years) but plan for the near-term (2-3 years). Implement for current needs while keeping the architecture flexible for future growth. Design for long term, plan for short-medium term. - Testing distributed systems requires going beyond known test cases. Develop test frameworks that generate new scenarios automatically to uncover unforeseen issues. - Cost optimization in data systems often involves trade-offs. Kafka, for example, sacrificed transactionality and exact-once delivery for higher throughput and lower cost per log. Its all about the Trade-offs. You get some, you lose some. - When approaching problems, start with first principles. Understand why something is expensive or inefficient before trying to optimize it. Don't jump on a solution. Understand the real problem first. - Open-source success relies heavily on community building. Foster respectful communication, clear processes for resolving conflicts, and a culture of helpfulness to grow a strong contributor base. There are many communities, but only the ones with good culture thrive. - Real-time analytics can significantly boost user engagement. Even small reductions in data latency can lead to measurable improvements in user experience and product value. - When considering real-time analytics, focus on how it can improve your users' lives rather than just technical capabilities. Many support issues can be eliminated by giving users direct access to real-time data. Product isn't useful if it making users' lives better. - Innovation in data products often comes from changing perspective - instead of asking "Do we need this?", ask "How can this improve our users' experience?" This shift can uncover valuable use cases for real-time analytics. Asking the right questions will help you reach the right answers. - When building data products, consider the long-term evolution of technology. Design choices should anticipate future developments, such as improvements in network speed or storage capabilities. Keep your design extensible. - As users become more data-savvy, there's increasing demand for self-serve analytics tools. Empowering users with direct access to data can reduce support overhead and enable more informed decision-making. Empower users to do more with data quickly. ———————————————————————— In case you haven't watched the episode yet, you can watch it here: youtu.be/SmLlJmU9Daw Also subscribe if you like such discussions.

41

3 Comments

Swiggy eyes up to $12.7 billion valuation for mega $1.4 billion-plus IPO; launch likely in early November (caution: 70% is going to bank accounts of existing investing only 30% is going into business development)

1

One of the most common mistakes engineers make while writing multi-threaded programs is assuming common operations are thread-safe ⚡ I went through the assembly code to understand why exactly "count++" is not thread-safe and compiled my learnings into a super interesting and highly practical video that covers - how threads compete for the shared resource - how to disassemble code to instructions - how code actually gets executed on the CPU give it a watch: youtu.be/kBHd7kn_1EU ⚡ Admissions for my System Design August 2024 cohort are open, if you are SDE-2, SDE-3, and above and looking to build a rock-solid intuition to design any and every system, check out arpitbhayani.me/course #AsliEngineering #ConcurrencyInDepth

238

5 Comments

Read this anecdote from someone interviewing for Spotify SWE in Infra team. He was asked to do a "Live root-cause analysis of a failure which previously occurred in Production at Spotify" How the interview worked: You explain your thought process (check connectivity between System A and B via ping, DNS name resolution, look at open ports, etc...) and they tell u the result like "ping returned 'Name or service not known'". It's like traversing through nodes of a decision tree and getting new insights at every checkpoint, and then deciding the next step! Doesn't it sound interesting? What are some interviews that you really enjoyed? #observability #monitoring #engineering #devops #infra

38

2 Comments

UBS estimated Blinkit’s market share at 40-45%, with Swiggy's Instamart at 20-25%, followed by Nexus Venture Partners-backed Zepto at 15-20%, and bigbasket.com’s BBNow at 10-15%. With a valuation of $15.4 billion, #quickcommerce platform Blinkit is contributing a higher share to Zomato’s valuation than its mainstay food delivery business, according to UBS. https://lnkd.in/g8MFhN8v

2

Difference between FAANG (and equivalent) engineering director(or principal engineer)role and non FAANG engineering director roles as per my experience: 1) Really understand what an director role needs - it's about complexity of the problem space and the organization and not how much you deliver. In smaller companies getting shit done is important and helps you grow. In larger companies it's easy to get shit done but more important to get the right things done. As your levels increase, the expectation is that you are responsible for identifying what's important. 2) Once you have a good sense of what you should be doing, be ruthless in focusing only on important things. There is a lot of noise with scale. Even an amazing product with a billion+ users will have hundreds of thousands of issues and edge cases that cause problems. If you try to address things coming to you vs consciously decide what should be worked on, you will have much lower impact than what you could have. Be extremely picky and make progress towards important things. 3) With large companies there is a lot of organizational friction. Your engineers need a lot of cover to do their job. There might be turf wars, other teams vying for their attention, suboptimal delegation etc. Figure out how to be fair but firm when it comes you shielding your team. Most people at Google are sufficiently smart - figure out win wins, if there are conflicts bring clarity (even if it means giving up a little turf) and unblock people so that they can get to work. This will become more and More important with time.

7

Intuit's recent decision to lay off developers, citing the need for more advanced AI skills, comes off as a disingenuous manoeuvre that sidesteps the truth. It's hard to buy into the narrative that these layoffs are truly about acquiring employees with better AI proficiency. In reality, it seems like a convenient cover for a cost-cutting strategy that involves offshoring jobs to cheaper markets. Think about it: the idea that current developers lack sufficient experience with AI tools like ChatGPT, Copilot, or other generative technologies is misleading. These tools have been part of the industry landscape for the past few years, and developers everywhere have been getting up to speed with them. The notion that there's a significant skill gap among developers when it comes to these tools is simply not accurate. The tools themselves, while helpful in certain contexts, are far from the magic solutions they are often hyped up to be. At best, they serve as fancy autocompletes—useful for streamlining some tasks but incapable of replacing the nuanced, expert work required for sophisticated software development. The real issue here is that Intuit's move undermines the trust and morale of its employees. Firing seasoned developers under the pretext of needing more AI-savvy replacements is not just misleading, it's a slap in the face to those who have dedicated their skills and efforts to the company. It creates an atmosphere of insecurity and distrust, where employees are left wondering if their hard work and loyalty mean anything in the face of corporate cost-cutting. Moreover, this strategy reflects a short-sighted understanding of what AI can achieve. While AI tools can certainly assist with certain aspects of development, they are not yet sophisticated enough to replace human expertise in creating complex, production-ready software. Believing otherwise is a risky gamble that could compromise the quality and innovation of Intuit's products in the long run. If the true intention behind these layoffs is to offshore jobs to cheaper markets, then Intuit should be honest about it. Hiding behind the AI narrative is dishonest and disrespectful to the employees who are being let go. Offshoring comes with its challenges, such as communication barriers and potential declines in work quality, and it is not the silver bullet it might appear to be. In an era where ethical business practices and corporate social responsibility are increasingly valued, such a strategy can erode customer trust and loyalty. #intuit #layoff #jobs https://lnkd.in/dv3qD9kS

4

1 Comment

Why closures are able to hold the values even after getting returned? A good article I found from Amandeep Singh explains this in detail. It's pretty crisp and clear! Especially check example 2 and it's explanation. https://lnkd.in/ggcsPH8i #frontend #JS #closures #languageinternals

5

System Design interview question asked at Atlassian : "Real-Time Emoji Broadcast for 10 Million concurrent users?" Have you wondered how Emoji's backend works at Disney+ Hotstar for millions of concurrent users? I have dissected this problem in my latest article in my newsletter - "Y School Of Tech" designed for engineers curious to learn the why behind every Tech concept. I have written a systematic approach to solving this real-world System Design problem. The link is in the comment. I am writing tech designs for real-world System Design problems. Consider subscribing to keep yourself updated here - https://lnkd.in/gt2Xuv-n #InterviewQuestion #YSchoolOfTech #Staycurious #SystemDesign #Architecture

7

1 Comment

The topic of "Why Kubernetes" has been a discussion point constantly. To get the ground reality, Kalpesh Bhalekar & I have had multiple interviews across Startups and Mid-Market companies who gave multiple reasons for moving to K8s: 1. High up-time requirement in markets like E-commerce, Travel & Healthcare. 2. Single powerful API to cover AWS, Azure & GCP. 3. Cost efficiency with effective resource utilization. Here are a few reasons why Kubernetes is becoming a household name across the market. If you plan to evaluate Kubernetes, just ping me or sign up for our waitlist. (Link in comments) #kubernetes #devops #developerexperience #cloudtechnology

14

1 Comment

Continuing my #100PapersChallenge, In the previous post, I discussed in-memory caching solutions like Memcache at the Facebook scale. This time, I'll continue the concept of caching, particularly cache coherence. Today's paper discusses a fundamental system for building a distributed file system, focusing on distributed lock services for cache coherence. You might be surprised to learn that this paper was written in 1996, while distributed systems have recently gained popularity. Another interesting fact is that it was part of the MIT distributed systems course from 2002 to 2024, initially covered in the Distributed Hash Table (DHT) section in 2002. Let's dive deep into the paper. 🌐 Read the article on my website: https://lnkd.in/gv_2KiJV 📖 Read the article on Medium: https://lnkd.in/g63rDPa6 Please subscribe to my newsletter to stay updated with my latest posts: 📩 Subscribe here: https://lnkd.in/g_gGV3rm MIT Course Link: https://lnkd.in/g7w3Zv8k Previous Post Link: https://lnkd.in/gwPZqWw5 👍 Like, share, and comment to spread the knowledge! Happy Learning! #100PapersChallenge #DistributedSystems #CacheCoherence #Frangipani #MIT #DistributedFileSystems

9

Intuit leveraged K8sGPT and GenAI in their on-call process, resulting in improved issue detection, debugging, and resolution. 🚀 On-call engineers often face frequent alerts, tedious log scanning, and multiple team handoffs to resolve issues. Often, few teams have high operations load and engineers face fatigue resulting in low productivity. 😌 Intuit was facing similar issues with their complex Kubernetes deployment comprising 325+ clusters and 7000 services. They came up with a methodical approach to solve the problem through GenAI. What were the goals of the project ? 👉 Reduce the Mean Time To Detect (MTTD) an issue 👉 Reduce the Mean Time To Remediate (MTTR) an issue How did they solve the problem ? 1️⃣ Golden cluster signals - Single view of cluster's health and availability. It helped engineers to detect the bad clusters quickly reducing MTTD. 2️⃣ Debugging with K8s GPT - Adopted open-source K8s GPT to analyze root cause of failures. K8s GPT has integration with Open AI and other models. 3️⃣ Remediation with GenAI - Since public LLMs don't have Intuit specific data, they hosted local models and augmented them with Intuit specific data (RAG) that suggested remediation Intuit is planning to extend the tool and use it for other use cases, such as traffic and Java virtual machine debugging. They have published a detailed blog on how they adopted K8s GPT and the challenges they faced. You can find the link in the first comment. 👇 On-call weeks are challenging, but investing in GenAI can streamline the process and improve work-life balance. Share in the comments if you've used GenAI to improve your on-call process. Also, explore how K8sGPT could boost your team's operational efficiency. Also, folks who are interested to know more about K8s GPT can go to the github link, understand the code and start contributing. 🔥 #chatGPT #GenAI #softwareengineering

56

9 Comments

Intuit is laying off 1,800 employees while planning to hire 1,800 new ones. It’s always sad to know when such a large MNC plans for mass lay-offs like this. It impacts so many lives we can never imagine. If you're affected or looking for new opportunities, I hope this helps. Here’s how to stand out and secure referrals in the most strategic way: 1️⃣ The resume should always be related to the job description. For eg. If JD mentions experience in Java or Spring, highlight your Java project in your resume first. 2️⃣ Never apply to a job directly when you need a referral. Just share the job ID link and once you get a referral, apply from the link received in the mail. 3️⃣ When reaching out for a referral, never just text - ‘Hi.’ Better to mention the reason in the first line of the message itself. Ensure you follow up within 2-3 days. 4️⃣ Share your resume in pdf format and the name format preferably should be Name_Role_Resume.pdf. 5️⃣ Getting referrals does not guarantee interviews. The best case is to ask for a referral from someone who is posting about an opening in their team. It has the max chance of starting the interview loop sooner. Most importantly, do process your emotions - you will come out stronger 🙌 Stay positive, and keep pushing forward! Reach out if I can be of any help. Make sure you follow the tips given above :) #layoff #intuit #referral #hiring #jobposting

161

7 Comments

Understanding how the CPU works and how to write thread-efficient programs is very important. Thread Context Switching - When a program is executed in a single-core CPU, the CPU can work with and execute only one thread at any given time. Each thread gets its share of CPU time. For example, when a thread is writing something to the disk, and if it's consuming time, the CPU switches to another thread while the earlier thread is waiting. Threads are context-switched, and it looks like it is multitasking, and things are going on in parallel, which is not. Processing : Sequential processing is where programs are executed in a sequence, one by one, getting allocated with the CPU Concurrency: Parts of the program can be run independently and combined for the result and to speed up the execution on a single CPU Parallel: Each program can be run on each CPU independently and combined for the final results to speed up the execution Single Vs. Multi-thread - By running multiple threads vs. single threads to execute a program, the overall execution time can be reduced, and efficiency can be improved. Process Vs. Threads: A process uses CPU, memory, disk, and other resources to execute a program. A process can spin up one or multiple threads to complete the job. Sync Vs Asyn: Sync is executed in a sequence and cannot be skipped, but Asyn can run its block of the program separately from the main program and doesn't have to wait for completion to move on Thread Deadlock: When the resources required by a thread are being blocked by another one, and each one is waiting on another one, it leads to a deadlock state #cpu #development #technology #softwaredesign

456

35 Comments

Under the small span of my career in Product based companies, I have seen engineers valuing only their technical skills. But despite all the late nights and flawless code, people still do not move to top roles. Here's the truth: your salary, title, and next promotion? They're not about your skills. They're about your impact. Want to increase your impact? You need a Product Engineering mindset power-up: • 𝗜𝗺𝗽𝗮𝗰𝘁 𝗯𝗲𝗮𝘁𝘀 𝗼𝘂𝘁𝗽𝘂𝘁: Features are useless if they don't create real value. • 𝗨𝗻𝗱𝗲𝗿𝘀𝘁𝗮𝗻𝗱 𝘁𝗵𝗲 '𝘄𝗵𝘆': Code without context is meaningless. • 𝗢𝘄𝗻 𝘁𝗵𝗲 𝗽𝗿𝗼𝗱𝘂𝗰𝘁: Don't just ship tickets - shape the vision. • 𝗧𝗵𝗶𝗻𝗸 𝗹𝗶𝗸𝗲 𝗮 𝗹𝗲𝗮𝗱𝗲𝗿: Code is a tool, value is the goal. This isn't about leaving engineering behind - it's about evolving into a high-impact Product Engineer. Be the engineer who bridges code and user impact. That's the future of engineering. I have seen people hiring for specific "skills" in engineering rather than mindset. Though basic engineering/coding skills should be there, and knowledge of some basic tools, but those shouldn't be a blocker. Hire for talent and attitude/mindset and keep upskilling employees. Skills in demand are ever-changing in the software engineering world. I hope this change also comes in the hiring processes and we focus a lot more on "𝙝𝙤𝙬 𝙥𝙚𝙤𝙥𝙡𝙚 𝙖𝙥𝙥𝙧𝙤𝙖𝙘𝙝 𝙖 𝙥𝙧𝙤𝙗𝙡𝙚𝙢", rather than "𝙬𝙝𝙖𝙩 𝙞𝙨 𝙩𝙝𝙚 𝙨𝙤𝙡𝙪𝙩𝙞𝙤𝙣 𝙩𝙤 𝙩𝙝𝙚 𝙥𝙧𝙤𝙗𝙡𝙚𝙢". The problem will change, but how you approach a problem mindset will help the teams in the long run #ProductEngineering #SoftwareDevelopment #Mindset #Hiring

26

1 Comment

How do you become a better programmer while being an SRE? Many SigNoz users and customers are SREs and DevOps engineers. In my conversations with them sometimes they lament the fact they don’t “code” any longer and want to sharpen their edges by writing application code. But they rarely get a chance to do this at their job. Coding is also like building a muscle. If you go months and months without writing any application code, when you do finally need to write code, suddenly you will find that you have lost the muscle. It's like not benching for a year and then trying to bench press. (Infrastructure-as-code is a different muscle entirely, and more config files than code). While strictly speaking SRE job does involve writing code - scripts, yaml, etc. IS CODE. But to those who miss writing application code, here’s some tips on getting started: 1. Pick a language you want to code in first, decide a hobby project and build it from end to end. With AI assistants like ChatGPT and CoPilot easily available now, an easy way is to chat with the LLMs to develop a plan and build the project. They can act as a sparring partner for you and also help in rubber ducking or resolve errors which may come up 2. Contribute to an open source project : One problem with doing something which is not part of your main job is to keep yourself motivated an accountable. I find open source communities to be a great way to help in this. Rather than choosing a hobby project of your own, you can just choose to work on a open source project. Start asking questions, joining community calls and interacting on Github/Slack. You will find problems in the project that need to be solved - and you can raise your hand to solve it. The advantage is you have people in the community to help you if you get stuck and if your PR gets accepted finally then you have a “greener” github profile to show for 🙂 3. Practice Leetcode everyday : For people who want something more piecemeal, leetcode could be great way to start. Agree it is not strictly application code, but it still helps you solve problems with code. And if you agree that writing code is like building a muscle, Leetcode is doing 10 pushups everyday. It is not tough to do - but if you do it everyday, you are on your way. 4. Build sample apps and demo projects for your team : This may not be applicable to all teams, but if you work in a developer focused product (like SigNoz) your team may need sample apps for their docs and demos. Raise your hand when you get such an opportunity. 5. Study books on design patterns and check code structures of open source projects: While this is not as good as “writing code”, if you really can’t get time to put keystrokes to your code editor, just start reading how other open source projects are solving problems. Studying more about design patterns and their application in real projects will help you get the juices flowing and who knows may give you enough motivation to get started 🙂

53

6 Comments

"Blazing Fast: Uber Serves 40 Million Requests per Second, Slashing CPU Cores by 95% from 60K to Just 3K, with Integrated Cache" 🚀 Don't Just Band-Aid Your Performance: Cache Like a Pro 🛠️ Cache is every developer's first line of defense against sluggish apps. But simply throwing a cache at a problem is like slapping a bandage on a gushing wound. It might offer temporary relief, but it won't fix the underlying issue. Uber’s epic struggle: While trying to make their apps faster and able to handle more users while stuck with old-fashioned storage. When there's suddenly a massive increase in people wanting to read data, it pushes them right to the edge of disaster. ❌❌ A BIG TECH HURDLE❌❌ Solution: Docstore comes to the rescue. Beyond the Quick Fix: 1.DocStore Query patterns - It turned out that more than 50% of the queries coming to Docstore are ReadRows requests. Build what’s required and what makes sense. 2.Bypassing Cache - Bypassing the cache in Uber's microservices ensures strong consistency for critical flows like retrieving items in an eater's cart, while leveraging cache for less critical tasks such as fetching a restaurant's menu. 3.Brilliant Cache Invalidation Strategy using CDC and Flux - Cache Invalidation is one of the critical problems in Software Development. This strategy ensures cache consistency within seconds, preventing the risk of uncommitted transactions affecting cache integrity. 4. Special Redis Codec for Row Reads: Unique row values in MySQL are cached in Redis using a special codec to ensure compatibility. This facilitates sharing cache resources across databases while maintaining data isolation. 5. Cache Warming and Cross-Region Replication: Instead of using inbuilt Redis replication, Leveraging the Query Engine for Cross-Region Replication ensures consistency. 6. Sharding Differently: Sharding Redis clusters differently from the database scheme optimizes caching for diverse data access patterns, enhancing overall system performance and scalability while resolving potential inefficiencies in cache management and data retrieval. 7. Adaptive Timeouts: Optimizing Redis Performance with Dynamic Timeouts. This prevents issues like premature failures or excessive latencies. Percentile latency ranges: P75 latency decreases by 75%, and P99.9 latency drops by over 67%, all while curbing latency spikes. ⏱️ As of the current date, CacheFront supports over 40M requests per second across all production Docstore instances, with continuous expansion. 📈 Blog Link : https://lnkd.in/ghvAY3_Z This is undoubtedly one of the finest reads I've encountered on distributed cache systems. I highly recommend giving it a read. 📚 Big Thanks to Preetham Reddy Narayanareddy Afshin Salek Piyush Patel Zurab Kutsia Eli (Oleg) Pozniansky #Systemdesign #distributedcache #cache #p99latency #systemdesign #design

21

1 Comment

Until a year ago I had a bias, 'Kubernetes is only relevant for late-stage companies'. During our market research, we were surprised to see startups as small as 20 employees using Kubernetes from day 1. Vedant Vyawahare & I have seen multiple reasons why companies moved to Kubernetes: 1. Serving unpredictable markets like travel and e-commerce. 2. Optimizing resources and avoiding provisioning. 3. Need to leverage Cloud credits across AWS, GCP & Azure Here are a few reasons why Kubernetes is becoming a household name. If you are planning to evaluate Kubernetes, feel free to ping me or sign up for our waitlist. (Link in comments) #Kubernetes #devops #devtools

30

3 Comments