Chaitanyasuma Jain

Training LLMs using reinforcement learning using human feedback. I was slacking on this topic for 6 months and finally was able to go through it, and wow I am amazed at what technologies are being worked on in production-ready systems these days. Generally in RL of LLMs, we have a reward model (let's say a good/bad text classifier) that gives a reward (prediction 1 for good, -1 for bad) and an RL algorithm (based on multiple loss functions) updates the weights of the LLM model to achieve better results. This is an iterative process between the reward model, the LLM, and the RL algorithm. Overall, my take is the most important aspect of training LLM using RLHM is obtaining that human feedback and training a reward model.

28

💡 Bridging the Gap Between Tech and Product 💡 One of the biggest adjustments I’ve made in my role at Amazon as an SDE is learning to adapt my communication. While SDEs and Product work closely together, it’s clear that we sometimes speak different languages. Many of the technical details that I naturally focus on aren’t essential for Product to know—and simplifying those details has been a valuable lesson. One conversation with my PM that stands out: Him: “I don’t understand what you’re trying to say here. Can you explain Harshit, please?” Me: “I’m trying to say ‘this,’ ‘this,’ and ‘this.’” (each representing a technical concept) Him: “Then write ‘this,’ ‘this,’ and ‘this!’” Experiences like these have taught me that communicating with impact isn’t just about delivering the facts. It’s about focusing on what matters most to your audience. Learning to step outside the SDE mindset and share clear, straightforward insights has made my role more rewarding and has strengthened collaboration with Product. #TeamCollaboration #SDELife #CrossFunctional #AmazonExperience

7

1 Comment

Amazon-to-Google-to-Amazon #a2g2a thought of the day: The Intensity, oh, the Intensity!!! When I went from Amazon to Google, it felt like I had just gotten off the freeway and was driving on a side street. At the time, I couldn’t quite disentangle whether the slowdown was because this was June of 2020 and covid had turned the entire world upside down or because it was a deeper part of the DNA of the company. Now that I’m back at Amazon, I feel like I was indeed driving on a side street for a while and now I’m accelerating onto the freeway onramp once again. There’s most definitely a difference in intensity between the cultures. Intensity by itself is neither good nor bad, but it depends on the context. As covid shut down the world, I appreciated Google’s emphasis on their employees well-being. In 2020, googlers received a generous 14 weeks of additional time off, fully paid, to be used any way we needed given the extraordinary world circumstances. I had two boys that we were now suddenly homeschooling, so I very much appreciated the benefit. The other side of that is that at times I wanted to step on the gas and it was quite hard. You can’t single-handedly do that, you need people around you to be as excited as you are and to follow you on the journey. Working backwards from firm dates is a bit of a foreign concept to Google and it’s table stakes at Amazon. "Intensity" is NOT necessarily the same thing as "working long hours" (although it is often confused). I have a family and my time with them is very important to me. It's about what you do with the hours that you do spend at work. The last few weeks, the intensity has been healthy. I’ve felt more alive. It’s energizing. I see the freeway up ahead and the adrenaline is beginning to kick in again!

306

28 Comments

Comet - LLM model monitoring, trace every step along with the evaluation & ranking to see which models provide more value for our problem is really an advantage to stay ahead in the solution offering process.

1

⏫ Leveled up my understanding of retrieval systems (R+AG) 🔍 and put those learnings to work by improving GitHub Issues management! Here’s a breakdown of what I explored: ✨ Embedding-based Retrievals – How embeddings drive more meaningful search by mapping concepts into vector spaces. ⚠️ Pitfalls of Simple Vector Search – Identified common issues like irrelevant results and learned how to tackle them effectively. 🔍 Query Expansion with LLMs – Used Large Language Models to broaden search coverage and uncover deeper insights. ⚖️ Cross-Encoder Re-Ranking – Tuned result rankings with cross-encoders to bring the best matches to the top. 🛠️ Embedding Adaptors – Learned how to tweak embeddings for different tasks to keep retrieval systems flexible and efficient. Super excited to apply these concepts further! If you're working on retrieval systems, let’s connect and exchange ideas! 🚀 #AI #ChromaDB #Retrieval #LLMs #GitHub

15

5 Comments

**This is the best way to ask for a referral** I have referred more than 60-70 people since joining Walmart, and any way try to refer everyone who asks for it. But Girish was different. And I feel like the way he approached the entire thing is how I would approach people if needed in the future, for referrals. 🟩He pinged me with a short description, job id, email id, phone number as part of the same message. 🟩He was at a mid tier startup at that point and offered to help my friends without me pinging him a job id. 🟩He went out of his way to enquire with his HR through subsequent weeks for each and every role I asked him for, and gave me regular updates about it. Convinced his HR up until she scheduled an interview. 🟩He followed back with me, with all his interview updates at Walmart, how the round went, what all things were asked after he got the interview call. The result? I ended up following back with my HR personally multiple times for him. Just because he offered to help back. I remember him saying on a call with me, “Aryan bhai aapko jitne startups mei chahiye meko batao aap. Mai bass Abhi badi company target krra hu but startups mei meri reach kaafi hai”, “Mai Bangalore aaunga to aapse pakka milunga” and he was so genuine, I ended up becoming great friends with him. What I mean to say is, offer help to ask for help. That is what the engineering community is about. Also fun fact, Girish is 1 of the only 2 people who have cracked the interview through my referral. Do you guys have your favourite referral stories?

1,322

58 Comments

🚀 Demystifying the Math Behind Fine-Tuning Large Language Models (LLMs)! 🚀 Fine-tuning LLMs might sound like rocket science, but it doesn't have to be! I've distilled all the key concepts into an easy-to-understand document. Dive into: 🔍 Quantization: Making models lighter without losing their edge. 🎯 Precision: Full vs. half – balancing detail and efficiency. 🔧 Calibration: Fine-tuning models for top performance. 🛠️ Quantization Modes: Post-Training Quantization (PTQ) and Quantization-Aware Training (QAT). 📄 Check out my document and discover the secrets!

18

“Why did I leave Amazon?” I've received a lot of questions about my recent move from Amazon to Uber. Amazon was a great place to work, and I learned a lot. But after 2 years, I felt like I wasn’t growing anymore, and the team environment wasn’t the best fit for me. Even after trying a new team, things didn’t improve, so I resigned. Though I had a MAANG name on my resume, the job search wasn’t easy. I didn’t receive many interview calls at first. Here’s how I tackled it: → I worked on improving my skills. → Reached out to senior SWEs on LinkedIn for advice and networking These efforts paid off, and I received 7 job offers, including one from Uber. The biggest lesson I learned is that you need to continuously upskill and adapt. ⚡ If you’re also trying to switch companies and struggling, you can easily connect with senior engineers at Preplaced. Just book a free strategy call and get started: https://lnkd.in/ghmwF3jf They have been through the same journey and will provide you with personalized roadmap, resources, mock interviews, and support to help you succeed in your career transition. They’ll stick with you till you reach your goals, so, do give it a try! ✨ All the best! #careerswitch #amazon #collab #preplacedcollab #uber #jobswitch

502

23 Comments

Netflix just had a killer quarter, which saw their market cap exceed $325B 😲. Not only is the company firing on all cylinders, their alumni are also making waves! The Netflix culture is notorious for their relentless focus on excellence and fostering an environment where talented people can thrive. Led by one of the OGs of Silicon Valley - Reed Hastings 👑 - we are not surprised to see some amazing people coming out of the institution. Already former Netflix employees are responsible for: 📈 +65 startups raised +$1M 💰 +$1B in total funding Why do they make so good founders? 🔹 Subscription masterminds 🔹 Streaming experts 🔹 User Experience 🔹 Incredible work ethic 🔹 Incredible bar for talents Netflix has attracted top people from: Harvard, Lockheed Martin, Google, Stanford, Goldman Sachs, Amazon, ATM, Microsoft, Apple, Northwestern University, Wharton etc. Over 25 companies have been founded by former Netflix employees, including amazing companies like Tabular (now part of Databricks), ClickHouse, Roku, inspir.ai, Justos, Colovore, Curai Health, GreyscaleAI, SignalRank, Orkes, Tobiko, Gradient and more! They are being led by some of the world's best tech operators and technologists and excited to see their companies grow: Daniel Weeks, Yury Izrailevsky, Anthony Wood, Xiangjun (XJ) Wang, Antonio Molins, Xavier (Xavi) Amatriain, Paul Kirincich, Shawn Purcell, Viren Baraiya, Tobias (Toby) Mao, Chris Chang and many more! Source: Palle Broe & Harmonic

72

4 Comments

optimizing Large Language Models (LLMs) from a software engineering perspective can significantly impact latency and costs. with my recent time invested in building production-grade RAG solutions, here are a few key pointers I felt can be taken into consideration: - Long context prefill in the prompt passed to the LLM can increase latency and costs. opt for prompt compression or decomposition to mitigate this issue. - avoid using an agentic framework with multi-turn behavior, like self-iterating loops, or techniques like self-consistency prompting, to prevent higher latency and LLM costs. agentic frameworks like crewAI or autogen by ms can be overkill with higher inference time - if using vector embeddings, opt for a vector db with lower retrieval latency, such as sqlite-vss or redis. - implement semantic caching to reduce LLM API calls and lower response time. reducing output length can also aid in processing or decoding phases. you can also explore using YAML instead of JSON for structured output as YAML is less verbose and hence, fewer tokens, however, I still believe this is task-dependent. - tasks performed by LLMs that are not strictly sequential can be split into parallel calls. sequentially required steps can be merged into a single LLM call to avoid round-trip latency. - ensure proximity between LLMs and tools infrastructure-wise. Allowing third-party plugins to deploy on the infrastructure where large models run can further reduce latency. - using async programming paradigms ( for eg. concurrent API calls ) or parallel programming ( for eg. using multi-threading for parallel API requests ) can also help reduce the inference time and increase the throughput. do check LiteLLM (YC W23) though. - using query routing strategies like semantic router by Aurelio AI or Martian or Unify or RouteLLM by Anyscale and LMsys can also help boost cost efficiency and speed. I am yet to test these out though, but hopefully soon. some notes: - there is a tradeoff between throughput and time per output token: if we process let’s say multiple user queries concurrently, we'll have higher throughput compared to running the queries sequentially, but we'll take longer to generate output tokens for each user. - large language Models (LLMs) generate text in a two-step process: "prefill", where the tokens in the input prompt are processed in parallel, and "decoding", where text is generated one 'token' at a time in an autoregressive manner. Each generated token is appended to the input and fed back into the model to generate the next token. Generation stops when the LLM outputs a special stop token or when a user-defined condition is met (e.g., some maximum number of tokens has been generated). what are the other potential strategies for building production-grade rag solutions? #SoftwareEngineering #LLMOptimization #DataProcessing #GenAI #llms

42

10 Comments

Exploring Hybrid Retrieval-Augmented Generation (RAG) for Enhanced Information Retrieval In my recent research in the  realm of information retrieval, I came across Hybrid RAG which balances between semantic (vector similarity) and syntactic (keyword) search approaches for achieving accurate and comprehensive results. A cutting-edge solution that combines the strengths of both methods to revolutionize how we interact with data. Key Features: - Semantic Search (Vector Similarity): Ideal for handling queries with typos or nuanced intents, leveraging powerful embeddings like HuggingFace to capture semantic relationships. - Syntactic Search (Keyword Matching): Perfect for precise matches on keywords, abbreviations, and names, employing techniques such as BM25Encoder for effective keyword search. - Hybrid Approach: By blending vector and keyword search scores using advanced fusion techniques like Reciprocal Rank Fusion (RRF), Hybrid RAG ensures robust and context-aware retrieval. Why Hybrid RAG Matters: Semantic search (using dense embeddings) excels in capturing semantic similarity and context, making it effective for understanding the meaning behind queries and documents. On the other hand, syntactic search (using techniques like BM25) is better at capturing exact matches and structural similarities. Combining both ensures a more comprehensive coverage of relevant information. Implementing Hybrid RAG: -- Set up with Pinecone for indexing and retrieval. -- Utilize HuggingFaceEmbeddings for semantic search. -- Employ BM25Encoder for efficient keyword matching. -- Integrate PineconeHybridSearchRetriever for seamless fusion and retrieval. Unlock the power of Hybrid RAG to enhance your information retrieval capabilities. Explore more about how this innovative approach can elevate your data analytics and decision support systems. #HybridRAG #InformationRetrieval #SemanticSearch #SyntacticSearch #AI #MachineLearning #DataScience #TechnologyInnovation #PineconeAI #GenAI #MLOps 

7

Which has more audience ? 1.) How to get into Big Tech as a SWE OR 2.) How to grow to senior/staff roles from new grad. I see a lot of folks on Youtube teaching DSA, System Design and selling courses online which caters to 1 But don’t find enough channels focused on 2 Does that mean there is more audience for 1 ? #softwareengineering

53

22 Comments

Did you know there are no "roles" (SDE, SDET, TPM, …) when you get to L8 IC (Senior Principal) at Amazon? It’s one of Amazon’s peculiarities. [Side note: in general, Amazon SDEs tend to be "jack of all trades" where we pragmatically do whatever is necessary to make our organization successful at that given moment (within reason). In fact, I believe being well-rounded and able and willing to wear a number of hats is one of the things that makes me valuable to an employer. We don’t over-index on "that’s not part of my role doc." Other companies where I worked (eg Google) have very clear swimming lanes and people stick to them pretty religiously, eg. "oh no I don’t talk to customers, I’m an SDE and talking to customers is for TPMs." Whereas at Amazon I never once had a manager tell me, "what you did (although impactful, important and necessary) was not really your job." ] My little side note about this critical piece of amazon culture explains why as you get to L8 the concept of a "role" no longer really makes sense. The boundaries between roles get fuzzier and fuzzier as you go from Senior Engineer (L6) to Principal Engineer (L7), and they become unimportant when you get to L8. We’re all just “Senior Principals” - whether you came via the SDE path, the TPM path, or in my snowflake case, the SDET path. In fact I think most L8s are snowflakes in that their path to L8 was unique and the value they bring to the company is also unique. In phonetool (our amazon directory) you can have whatever unofficial title you want (also within reason...), so I chose to keep the "SDET" title to reflect my background and passion for testing, engineering and operational excellence, and developer tools. I also hope that by keeping the title, others in my (old?) role can see that SDETs at Amazon do not have a "ceiling" in their career. And lastly, it's a fun conversational piece ("you're an L8 SDET???") (The image here is "Peccy" - a beloved little icon that has represented Amazon's Peculiar Ways for years) [Edit: Strictly speaking there are a few L8 IC roles that do have a different job code, like PMT or Scientist - but I think we generally view all L8 ICs in the same lens]

928

41 Comments

632 𝐌𝐢𝐧𝐮𝐭𝐞𝐬: 𝐌𝐲 𝐏𝐫𝐨𝐣𝐞𝐜𝐭 𝐉𝐨𝐮𝐫𝐧𝐞𝐲 I spent 632 minutes deciding my project structure—something you won’t find anywhere on the internet. (And no, I didn’t watch a single YouTube video for guidance.) 𝐇𝐞𝐫𝐞’𝐬 𝐰𝐡𝐚𝐭 𝐡𝐚𝐩𝐩𝐞𝐧𝐬 𝐰𝐢𝐭𝐡 97% 𝐨𝐟 𝐥𝐞𝐚𝐫𝐧𝐞𝐫𝐬 𝐛𝐞𝐟𝐨𝐫𝐞 𝐬𝐭𝐚𝐫𝐭𝐢𝐧𝐠 𝐚 𝐩𝐫𝐨𝐣𝐞𝐜𝐭: 1. “I need to learn GitHub, MLOps frameworks.” 2. “I need to master this, that, and a hundred other things.” And they often get stuck in a cycle of constant learning without ever implementing their knowledge. I believe that once you understand Python and machine learning basics, you should jump into building projects. The process of actually doing a project will teach you more than any course ever could. Plus, it builds real confidence! 𝐇𝐞𝐫𝐞’𝐬 𝐰𝐡𝐚𝐭 𝐈 𝐝𝐢𝐝: 1. Even though I know ML algorithms well, I didn’t build a model in this project. Instead, I wanted to sharpen my Python and Bash scripting skills (I earned the topper badge from IIT Madras in these courses, too!). 2. For this project, I applied my 𝐌𝐋𝐎𝐩𝐬 (𝐙𝐞𝐧𝐌𝐋), 𝐁𝐚𝐬𝐡 𝐬𝐜𝐫𝐢𝐩𝐭𝐢𝐧𝐠, and 𝐏𝐲𝐭𝐡𝐨𝐧 knowledge. I also learned something new: writing unit tests using 𝐏𝐲𝐭𝐡𝐨𝐧’𝐬 𝐮𝐧𝐢𝐭𝐭𝐞𝐬𝐭 𝐦𝐨𝐝𝐮𝐥𝐞. This helps test your code to make sure it works as expected. 𝐇𝐞𝐫𝐞'𝐬 𝐦𝐲 𝐆𝐢𝐭𝐇𝐮𝐛 𝐥𝐢𝐧𝐤 : https://lnkd.in/gK3at2j3 P.S. I’m still updating the tests folder and will soon upload a video explaining the whole project. Start your ML project now! Don’t wait. Connect with Md Ayan Arshad, and let’s build some amazing things together!

45

13 Comments

I never expected Google’s ML round to be this challenging 🤯 But thankfully, I managed to breeze through the interview and am left with crucial lessons. 1. *System Design Is Key:* You'll have to design ML systems end-to-end. Practice mapping out solutions while considering factors like scalability and model deployment. 2. *Coding Skills Matter:* ML engineers are expected to code efficiently. So, practice coding challenges focused on data structures, algorithms, and ML-specific tasks. 3. *Communication Is Critical:* Google emphasizes clear communication. Practice explaining your thought process, assumptions, and solutions. In a nutshell, ML interviews at Google test you on all core and communication skills, so having an all-around preparation is a must. But now, you don’t have to go at it alone! You can get help and guidance from top ML engineers. They can assist you in learning the required skills, test you through mock interviews, and share success strategies 🚀 Start your upskilling by booking a FREE trial - https://lnkd.in/dVDNETE5 Now, interviews won’t be challenging since you have these mentors. All the best ✨ #google #interviewprep

248

8 Comments

Here are the 3️⃣ things If I had to reappear for the SDE interview at a Product-Based Company today! Many working professionals aspire to work at companies like Amazon, MoEngage, Microsoft, and others. If I had to reappear for interviews at companies, here's what I would focus on doing better: 👉 I would focus on arrays, trees, graphs, sorting, and dynamic programming and practice problems daily. It would help me improve my coding and problem-solving abilities. 👉 Would revisit design patterns, scalability, and distributed systems. I'd practice designing scalable, high-performance systems to ensure I can handle design questions confidently. 👉 Focus on building projects to demonstrate my practical skills. A portfolio of projects can serve as proof of my abilities and help me strengthen my resume. Preparing for SDE interviews at PBCs requires Dedication & Consistent Practice. If you're looking for structured guidance and mentorship to ace your interviews, I’d suggest checking Bosscoder Academy Check here: https://bit.ly/4duh8a7 They can help you with ✅ Structured programs for learning DSA, System Design, and Full Stack Development. ✅ Personal mentorship from industry veterans. ✅ Live classes and 24/7 doubt support. ✅ Real-life, relevant projects with Placement Assistance.

586

42 Comments

🚀 Amazon S3 (Simple Storage Service) achieves its high durability, quantified as an astonishing 11 nines(99.999999999%) of durability. Here’s how they do it: 🔸 Data Redundancy: Utilizing mechanical hard disks for their cost-effective storage, Amazon S3 replicates data across multiple disks, employing erasure coding to split data into shards, significantly reducing the probability of data loss. 🔸 Data Integrity: To combat bit flips and data corruption, Amazon S3 implements checksums using client SDKs, ensuring data integrity through rigorous checks and the use of efficient checksum algorithms. 🔸 Data Audit: Amazon S3 periodically scans storage devices to verify data integrity, quickly re-replicating data if any issues are detected, maintaining a balance between repair and failure rates. 🔸 Data Isolation: By storing metadata and file content separately and operating isolated data centers within an availability zone, Amazon S3 minimizes the risk of simultaneous failures. The commitment to data durability is evident in every layer of Amazon S3’s design, from the hardware to the engineering culture. It’s a testament to the power of thoughtful system design and its impact on reliability. #AmazonS3 #DataDurability #SystemDesign #CloudStorage #DataIntegrity #EngineeringCulture

5

After that yesterday I posted the schedule of upcoming posts for Machine learning at scale, I received a few questions related to what type of content I choose to post. Let me clarify a few things! ➡ "You don't have a clear focus! You talk about: ML system design, LLMs, Feature stores and more. Pick something and stick to it! 🎲 " ✅ Correct! What I post is what I learn in my free time that I like sharing. I don't have an agenda, I don't follow the advice of "be the guy that does X thing and niche down to get more followers". I just like all things ML and that's what you will see. I work in tech, I am not a content creator and I don't particularly focus on being one. (No offense to anyone that chooses to go down that route). I just prefer spending my energy more on learning what I like and distilling it down for others rather than crafting the perfect creator profile. If more people follow me because of this, of course that's a win-win and I am happy, but I value my authenticity a lot. ➡ "Does that mean you are not going to do a deep dive series on topic X, Y or Z? 🙇‍♂️ " ❌ On the contrary! I have big plans on covering a selection of topics in more depth. One benefit of having scheduled ahead some many posts is that I can focus on "bigger projects" so to speak. So stay tuned for that! I think the AI space is full of noise and I plan on being a small part in crafting something high quality. ➡ "You are boring me to death talking only about deep tech topics 💤💤. Give us more stories in the ML space related to career growth, managing up, influence etc." ✅ ❌ Yes and no, eheh. I understand that getting spammed every week with technical articles that require a certain amount of focus can be daunting. But that's what I like, sorry not sorry :D. On the other hand, I plan on integrating more "real life stories" about my life in a big tech and how to manage a career in the ML space as I recognize the value of it. So stay tuned for that! :)

30

4 Comments