Sakibul Mowla

Caching | The Art of System Design Just published a new video that dives deep into caching—a fundamental concept for scalable system design! 🚀 In "Caching | The Art of System Design," I explore how caching can dramatically improve system performance and availability. I cover different caching strategies, use cases, and common pitfalls to watch out for. This video is perfect for anyone wanting to enhance their understanding of system reliability and speed, whether you're prepping for interviews or designing robust systems. Check it out here: https://lnkd.in/gwfquBjD If you find it helpful, I’d love for you to like, comment, or share it with others who might benefit. Your feedback always helps me improve! 😊

2

*UPDATE: 1st Nov - System Design Roadmap - 200+ downloads in last 2 days with 5 star ratings and testimonials. Get your copy today and leave me a comment for what more you would like in there* Never before have I felt such a need to write a roadmap for system design interview preparation. A friend of mine in London got redundant. She called me last week to discuss the topics for system design interviews. This was not the first time someone asked me about the ‘topics’ and how to prepare. After receiving multiple queries in the last three months, I decided to put this together for everyone who needs to know. This starter guide includes:  - a detailed syllabus to learn system design for beginners - a section on how to approach system design questions - a list of exercises to apply the concepts learned You can grab the doc from my Topmate link. It’s free :) Here’s the link: https://lnkd.in/eYdvRW5b (Note: Please select pay 0 during the checkout option on Topmate, I found the platform easy to host the resource)

99

16 Comments

It took a while, but I finally published Part 2.1 on the series I've been writing about Career Crafting in Modern Software Engineering. I struggled for the past few months getting this out because I kept trying to fit all the ideas in a single article. Now I'm doing this in 4 sub-parts, so this is sub-part 1/4! I hope you enjoy the article! https://lnkd.in/gs7qfNh6

20

Hello everyone, Inspired by the wealth of knowledge shared across various posts, I dedicated some time over the weekend to delve into the realm of caching. Understanding the intricacies of caching problems and solutions fascinated me, leading me to believe that the most effective way to grasp a concept is by implementing it myself. In this endeavour, I embarked on the journey of building an LRU Cache from scratch. Through meticulous thought processes and reasoning, I navigated through the stages of development, gaining profound insights along the way. I firmly believe that by documenting this journey, not only have I solidified my own understanding, but I also hope to contribute to the collective learning of others. In the post, I'll guide you through the stages of implementing a constraint eviction strategy and optimizing the eviction strategy for better performance. Check out the post on medium Building LRU Cache Iteratively https://lnkd.in/g-mxRSpR Check out the code here on the GitHub https://lnkd.in/gp-dTVdU

25

1 Comment

🚀 **System Design Spotlight: The Power of Event-Driven Architecture** 🚀 As a senior software engineer with the privilege of having worked at Google and Microsoft, and having taught thousands of engineers at Scaler, I’m constantly fascinated by the innovations in system design that drive our industry forward. Today, I want to share insights on one of my favourite topics: **Event-Driven Architecture (EDA)**. ### Why Event-Driven Architecture? In today’s fast-paced tech world, the ability to process and react to events in real-time is crucial. EDA decouples event production from event consumption, leading to highly scalable and maintainable systems. ### Key Benefits: 1. **Scalability**: Components scale independently, essential in microservices architectures. 2. **Resilience**: Asynchronous event communication enhances system resilience. If one component fails, others continue operating. 3. **Real-Time Processing**: Ideal for applications requiring real-time data processing, such as financial systems, IoT applications, and social media feeds. 4. **Flexibility and Extensibility**: New features or services can be added without disrupting existing ones. ### Practical Example: Consider a modern e-commerce platform. When a user places an order, several events are triggered: - **Order Placed**: Inventory updates. - **Payment Processed**: Accounting records the transaction. - **Order Shipped**: Customer notification. Each event is processed by different microservices asynchronously, ensuring a seamless user experience. ### Tools of the Trade: - **Apache Kafka**: Handles high throughput with distributed streaming. - **AWS Lambda**: Executes code in response to events in a serverless environment. - **RabbitMQ**: A robust messaging broker for reliable message delivery. ### Conclusion: Event-Driven Architecture is a game-changer for building scalable, resilient, and flexible systems. It’s a powerful paradigm that aligns perfectly with modern software development demands. Whether working on microservices, IoT, or real-time data processing, EDA offers a robust solution for complex workflows. If you’re passionate about system design and want to dive deeper into EDA or other advanced topics, let's connect! I love sharing knowledge and learning from this incredible community. 🌟 #SystemDesign #SoftwareEngineering #EventDrivenArchitecture #Microservices #TechInnovation #Scalability

100

2 Comments

One of the most common mistakes engineers make while writing multi-threaded programs is assuming common operations are thread-safe ⚡ I went through the assembly code to understand why exactly "count++" is not thread-safe and compiled my learnings into a super interesting and highly practical video that covers - how threads compete for the shared resource - how to disassemble code to instructions - how code actually gets executed on the CPU give it a watch: youtu.be/kBHd7kn_1EU ⚡ Admissions for my System Design August 2024 cohort are open, if you are SDE-2, SDE-3, and above and looking to build a rock-solid intuition to design any and every system, check out arpitbhayani.me/course #AsliEngineering #ConcurrencyInDepth

238

5 Comments

Embracing Code Reviews and Alternative Solutions As software engineers, we often spend hours immersed in our own code, confident that our design patterns, logic, and methods are the best approach. The clarity in our heads translates into a sense of ownership and pride in our work, which is essential to driving innovation and solving complex problems. However, when our code undergoes a code review or we receive feedback from a peer, it’s natural to become defensive, certain that our path is the right one. This confidence can be valuable—it enables us to advocate for our ideas and push through challenges. But once in a while, it might be worth modeling the alternative. What if the reviewer’s suggestion is valid? What if their critique points to an edge case or a more elegant solution we hadn’t considered? How would implementing their idea change the system’s performance, readability, or scalability? Instead of dismissing alternative perspectives outright, we can ask ourselves: • If they’re right, how does that change the codebase or the user experience? • How can I use this insight to strengthen the project overall? In software development, embracing alternatives doesn’t mean admitting defeat—it’s about collaboration and continuous improvement. If the feedback leads to a better solution, great! If it doesn’t, we still gain clarity on why our original approach works, and we can confidently return to it. The beauty of software engineering lies in its iterative nature. We can always refactor, rebuild, and refine tomorrow. The willingness to entertain “if our peer is right” not only strengthens our code but also fosters a culture of openness and shared learning within the team. After all, no single function, framework, or developer has all the answers. But together, through constructive feedback and an open mind, we build something greater than the sum of its parts. #softwareengineering #teamwork

10

Hello Everyone, Welcome to Part 2 of our series on Building a Cache Iteratively. In this instalment, we'll delve into the concept of introducing threads into the equation, understanding the challenges of concurrent access, and devising a solution within a data structure. We'll tackle these issues in the most basic yet effective manner, gradually building a reliable system. You can find the detailed discussion in the following Medium article: https://lnkd.in/gsKYTyzN For those interested in exploring the code base, it's available on GitHub: https://lnkd.in/gp-dTVdU Feel free to dive in, explore, and share your thoughts! #LRUCache #Cache #LLD

14

Today I'm going to teach you the secret of how to repeatedly hire world-class software engineers. I've been working on this strategy for a decade and I'm sharing it today because interview skills and processes in the industry are still shockingly poor. Are you ready to hire the best? The greatest engineers I've worked with always embodied three particular competencies. As I focused my hiring process towards these skills, a good stream of hires with regular greats became a great stream of hires with no mistakes. Let's dive in: 1. Ability to Communicate 🗣️ Can you and the candidate share ideas and concepts back and forth with each other? And I don't just mean rote "tell me about a time when..." questions - can you actually communicate, mind to mind, human to human? Some candidates may be gruff, others chatty, but the ones who are able to meet you in dialogue have ticked the first box ✅ You need to bring your A-game as an interviewer here - don't let the candidates sit in silence or waffle, and remember that they are likely far more nervous than you are. 2. Ability to Reason about Code 👩‍💻 If you're hiring a software engineer, you need to look at some code together. It doesn't need to be complex, what matters is whether the candidate can think through the implications of the structure and execution on the fly. An engineer who can't reason about simple code that they've never seen before isn't going to do much better with code that they're living in every day, in my experience. Don't use this as an excuse to drown your candidates in complexity, though! 3. Ability to Learn 🧑‍🎓 This is the most important one, and it asks a lot of you as an interviewer. You need to find the edge of the candidate's knowledge on a subject and push them that little bit further. Some will extrapolate into the unknown, some will take a guess, others will ask you to explain the answer. All of the great hires I've made have been unsatisfied with being unable to answer a question and immediately wanted to learn more. All of them. Every. Single. One. It's not just about ability to learn, it's about curiosity and drive, and you should value those qualities. I've spent as many years developing this framework as I have using it, and I'm very confident now in being able to identify a candidate who will succeed in one of my teams within about 30 minutes, with the rest of the process working to confirm and triple-check my assessment of their skills. Using this framework I've hired amazing engineers from all walks of life and backgrounds, and there's nothing to stop you doing it too.

56

13 Comments

And here I had hoped that we had passed the era of: "the best software developers were the ones who could best write the code" a very long time ago. I think most of us realized in the naughties, that the best software developers were the ones who wrote the most readable and readily changeable code. Picking the right abstractions, making them as clear as possible. I don't know if AI is capable of that - it probably is. It just wont get there unless: 1. our prompts identify the value propositions of the abstractions we want to see. 2. the training rewards readability, and a balance between cohesion and decoupling.

1

Two steps for a software engineer to write code: 1. Make it work 2. Make it work again with simple interfaces and minimal code and dependencies While most engineers would stop at step 1, I do believe what makes a good engineer become a great engineer is in step 2. Step 1 delivers OKRs, achieving immediate short-term ROI, but step 2 raises the quality of the code, impacting the long-term ROI (easy to use and integrate, lower maintenance cost, less incidents in production, more flexible for extensions, often better for performance, and easy to ramp up for new people). It's not just about getting the work done, but how well you do the work. We call it craftsmanship. It may be a surprise that it does not take much time to get into step 2, only a few extra miles of deeper thinking. And if you make it into your habits, it would come naturally.

258

12 Comments

"The Staff Engineer's Path" offers some great insights into the IC career journey. As ICs progress, they gain broader knowledge but often lose the opportunity to dive deep into the details, which can eventually impact their ability to be great decision-makers when critical opportunities arise. Many of the tech leads I admire manage to balance both breadth and depth, though it's a tough challenge. Here’s my contrarian take: it’s okay to "level down" at times to recharge and refocus. This could mean taking on a hands-on project within your current company to code a solution end-to-end, stepping away to learn new technologies during a career break, or joining a startup to explore new ideas and business dynamics. Sometimes, these moments of stepping back allow for growth in unexpected and rewarding ways. What’s your perspective on finding depth in a senior IC role? https://a.co/94p8SVa

51

1 Comment

Some of the best software engineers start off as QA engineers. They just have to think about software in finer detail and they won't release something half-arsed out to staging or production. This detailed thinking extends to their software engineer skills if they decide to move into software engineering roles. It's funny how so many places relegate QA/Testing/Performance Testing into the 'something we do at the end if we need to because our software is always perfect' bucker. It reminds me of localisation, something else that very often is an afterthought. Hard code all your text, audio, and video links and don't worry about abstracting them out. The point here is that ahead of time you can have the architecture in place to get localisation done and more importantly deliver better quality software out quicker if both are first class citizens of the dev process. Let's stop running fast to create broken software. #Software #QA #SoftwareEngineering #BrokenSoftware

17

1 Comment

Refactoring code should be done along with your work on your tasks. See something => fix something. Incrementally improving code is way more effective than waiting for a big bang time to fix all the tech-debt cause let's be honest that rarely works out. Prevention is better than cure. Worried about refactor PR approvals? Refactor existing code first, make sure tests and behavior are in shape, and either commit it independently or put it lower in a stack of diffs so it’s easier to review, and easier to understand what is doing what. #refactoring #techdebt

21

2 Comments

𝐒𝐜𝐚𝐥𝐚𝐛𝐢𝐥𝐢𝐭𝐲 𝐚𝐭 𝐔𝐛𝐞𝐫: 𝐒𝐢𝐦𝐩𝐥𝐢𝐟𝐲𝐢𝐧𝐠 𝐭𝐡𝐞 𝐂𝐨𝐦𝐩𝐥𝐞𝐱 Did you know that even a basic feature like counting a driver's completed jobs at Uber involves complex scalability solutions? Here’s how Uber engineers tackle such challenges at scale. For Uber, counting jobs isn't just a number—it's an operation that must be executed amidst handling millions of transactions daily. The scale demands robust, scalable, and responsive solutions. Uber uses Apache Pinot, an advanced OLAP datastore, Uber manages both real-time and historical data efficiently. However, scaling this system meant overcoming several obstacles: 𝐂𝐚𝐩𝐚𝐜𝐢𝐭𝐲 𝐏𝐥𝐚𝐧𝐧𝐢𝐧𝐠: Estimating storage and compute power required innovative approaches due to unpredictable load patterns. 𝐎𝐩𝐭𝐢𝐦𝐢𝐳𝐢𝐧𝐠 𝐐𝐮𝐞𝐫𝐲 𝐏𝐞𝐫𝐟𝐨𝐫𝐦𝐚𝐧𝐜𝐞: Techniques like Inverted Indices and Bloom Filters were implemented to enhance search efficiency and reduce load. 𝐇𝐚𝐧𝐝𝐥𝐢𝐧𝐠 𝐃𝐚𝐭𝐚 𝐒𝐩𝐢𝐤𝐞𝐬: Jitter was introduced in request timings to manage and smooth out peak loads, ensuring system stability. This goes beyond just job counting. It’s about building resilient systems that enhance user and partner experiences at scale. This is a perfect example of the behind-the-scenes engineering feats that make simple features work seamlessly for millions of users Source: https://lnkd.in/gBWJSM4m 𝐅𝐨𝐫 𝐦𝐨𝐫𝐞 𝐬𝐮𝐜𝐡 𝐜𝐨𝐧𝐭𝐞𝐧𝐭, 𝐬𝐮𝐛𝐬𝐜𝐫𝐢𝐛𝐞 𝐚𝐧𝐝 𝐟𝐨𝐥𝐥𝐨𝐰! Go Careers #Engineering #DataAnalytics #Uber #ApachePinot #TechnologyInnovation #BigData #Technology #softwareengineer #software #softwaredeveloper

17

I put together some notes on the practice of error propagation and API contracts in Go: https://lnkd.in/dp_hGBTA

18

2 Comments

In one of my recent posts complaining about software engineering practices, I mentioned that a good strategy for encouraging quality is to make sure that developers are steered in the right direction by their development environment - the phrase I used was (approximately) to "make the easy thing also be the right thing". Some folks chimed in and said that the right thing often costs too much upfront. That was precisely my point - if it costs too much upfront, it's too hard. The right thing should be easy, not hard! It turns out that - O cruel fate! - it takes a lot of hard work to make the right thing also be the easy thing. One of my favorite papers on this topic came from folks in my orbit at Google a few years ago. Members of my team had done a lot of work on static analysis for Java, and worked closely with our colleagues in engineering productivity to write up the lessons learned about what makes a good static analysis tool. The entire article is worth reading, but spoilers: after a lot of case studies, they identified that static analysis tools must be: Integrated. Developers will not go out of their way to run these tools, so the tool must be part of the developer's workflow, and not take too long to run. Have actionable errors. Developers ignore incomprehensible errors, as well as errors for which the followup action isn't clear. Ideally, the tool will tell you exactly what to do. Trustworthy. A tool cannot have too many false positives. For a static analysis tool to be included in our default code review workflow at Google, developers should feel the check is pointing out an actual issue at least 90% of the time. The error must manifest in practice. If a developer knows that a bug will never actually happen, they are unlikely to fix it. The fix must be cost-effective. Bugs that require you to rewrite your code from scratch must be very important. All of this requires a strong partnership between the static analysis tool developer and the user. One of the things I have enjoyed most about working at Google is that that kind of partnership is possible - Google has one big multi-billion line codebase, with one set of compilers and static analysis tools that we all share. The fact that Google has embraced this philosophy for static analysis tools has meant that they are widely embraced, adopted, and seen as a win. Side-note: One of the things that I've most liked about moving to Rust is that several of these lessons are baked into the compiler. It is very, very common to find that the Rust compiler knows exactly what change you need to make to your code, and tells you. For example, I find it very easy to forget whether I'm dealing with a reference or a value in Rust, and the compiler will literally spit out a diff for me. Sometimes the error messages could be improved, but, by and large, I've been pleased to see their embrace of this approach. The entire article is worth reading. https://lnkd.in/dJhCXBSE

21

Expressing http routes in Golang is now enhanced with 1.22 Features like methods matching and wildcard matching get introduced. They have changed their precedence logic for matching patterns too. https://lnkd.in/ge8t79uE

2