Aman Pahariya

Congratulations Chirag Thakur on your Google Cloud Platform certification! 🌟 This is a fantastic achievement and a testament to your hard work and dedication.   It's great to see how the supportive environment at Blu Parrot and the guidance from Nitin Kalra have contributed to your success. Excited to see how you will continue to leverage these skills to push the boundaries in tech. Keep soaring high! 🚀   #GoogleCloud #Certification #BluParrot #CloudArchitecture #Data #AI #datalake #generativeAI #promptengineering

6

How do you train a model on 10K H100s? Recently, Soumith Chintala shared a high-level overview of how to train a model on a large cluster of 10K H100s. While insightful, it missed some of the tactical steps that companies need to get started. Given my experience at Amazon Stores working on our massive GPU largest training clusters, I wanted to share a step-by-step approach for building a training cluster from scratch—specifically one that leverages the cloud, as business can’t afford massive on-prem setups. 1. Set Up Your Training Cluster For flexibility and efficiency, I highly recommend using batch computing services like AWS Batch. With cloud batch computing, you can dynamically scale resources up and down based on your needs, and it often comes with additional benefits like built-in job scheduling, heterogeneous clusters and bad node eviction . Static clusters in the cloud, in my experience, are cumbersome to manage and lack the flexibility that the cloud is known for. 2. Choose Your Training Stack A training stack typically has four components: a job launcher, an artifact store, a training framework, and a metadata tracker. I recommend the combination of TorchX, S3, NeMo, and MLFlow. This setup is battle-tested and well integrated: MLFlow has native S3 capabilities, and NeMo 2.0’s reuse of PyTorch Lightning components makes it easy to leverage community-built callbacks. Traditional network storage solutions often lead to bottlenecks due to grey failures. Switching to S3 gives you virtually limitless storage without the typical scaling overhead. 3. Monitor and Auto-Recover Your Jobs As your training cluster scales, both in size and complexity, custom tooling becomes essential. If a training run involving over 1,000 GPUs gets stuck over the weekend, it could lead to a costly on-call. Implement tools to monitor job efficiency, auto-recover stuck jobs, terminate non-responsive jobs, and evict bad nodes. These automations make training clusters more reliable and reduce the need for manual intervention. By following these tactical steps and combining them with the principles Soumith outlined, Amazon Stores has managed to build a massive training cluster with multiple nines of availability and a high effective utilization. If you're curious about getting started with training a large language model from scratch or how to leverage AWS for your training cluster, drop your questions in the comments. #MachineLearning #MachineLearningEngineer #AITraining #AWS #NeMo

5

A Day at the Apache Kafka 🚀 Today, I had the pleasure of attending an in-person Apache Kafka hosted by CRED in Bangalore. Here’s my feedback on this incredible event: 📍 Venue: Cred Aero, Bengaluru 🔹 Agenda Highlights: - @Vikas Sharma , CRED : Talk: Running Kafka Optimally in Production: Practical Usage Feedback: Vikas shared practical insights on optimizing Kafka in production environments, with valuable strategies for resource management. - Udayaram Kammara , Bazaarvoice : Talk: How Bazaarvoice Utilizes Kafka for Both Stream and Batch Processing Scenarios Feedback: Udayaram's session on Kafka's use at Bazaarvoice was enlightening, covering real-world use cases and challenges. - Diptiman Raichaudhuri , Confluent : Talk:Stream Processing for the Non-JVM World: pyflink and kafka-python Feedback: Diptiman's explanation of Apache Flink's architecture and the demonstration of kafka-python and pyflink were fascinating and beneficial for non-JVM users. What I Learned: - Effective strategies for running Kafka in production. - Real-world use cases and challenges in stream and batch processing. - Leveraging pyflink and kafka-python for stream processing. Overall, this meetup was a treasure trove of knowledge and inspiration. The sessions were well-organized, and the speakers were exceptionally knowledgeable. I’m excited to implement these learnings in my projects and continue exploring the potential of Apache Kafka and related technologies. 🙌 #ApacheKafka #KafkaMeetup #CRED #Bangalore #EventStreaming #StreamProcessing #TechCommunity #Networking #Python #Flink #DataPlatforms

24

2 Comments

Intuit leveraged K8sGPT and GenAI in their on-call process, resulting in improved issue detection, debugging, and resolution. 🚀 On-call engineers often face frequent alerts, tedious log scanning, and multiple team handoffs to resolve issues. Often, few teams have high operations load and engineers face fatigue resulting in low productivity. 😌 Intuit was facing similar issues with their complex Kubernetes deployment comprising 325+ clusters and 7000 services. They came up with a methodical approach to solve the problem through GenAI. What were the goals of the project ? 👉 Reduce the Mean Time To Detect (MTTD) an issue 👉 Reduce the Mean Time To Remediate (MTTR) an issue How did they solve the problem ? 1️⃣ Golden cluster signals - Single view of cluster's health and availability. It helped engineers to detect the bad clusters quickly reducing MTTD. 2️⃣ Debugging with K8s GPT - Adopted open-source K8s GPT to analyze root cause of failures. K8s GPT has integration with Open AI and other models. 3️⃣ Remediation with GenAI - Since public LLMs don't have Intuit specific data, they hosted local models and augmented them with Intuit specific data (RAG) that suggested remediation Intuit is planning to extend the tool and use it for other use cases, such as traffic and Java virtual machine debugging. They have published a detailed blog on how they adopted K8s GPT and the challenges they faced. You can find the link in the first comment. 👇 On-call weeks are challenging, but investing in GenAI can streamline the process and improve work-life balance. Share in the comments if you've used GenAI to improve your on-call process. Also, explore how K8sGPT could boost your team's operational efficiency. Also, folks who are interested to know more about K8s GPT can go to the github link, understand the code and start contributing. 🔥 #chatGPT #GenAI #softwareengineering

56

9 Comments

🚀 Bengaluru-based Conversational AI Startup Gnani AI Raises $3M 💰 Gnani AI, India's first voice-first small language model (SLM) for vernacular languages, has secured $3M in funding from tech holding company Info Edge. The investment will fuel the startup's expansion plans. Key Highlights: • Gnani AI is building a series of voice-based SLMs, trained on millions of hours of proprietary audio datasets and billions of conversations in 12 Indian languages • The startup has over 100 customers in banking, financial services, insurance, telecom, automotive, and healthcare industries • Gnani AI's models run on the edge, reducing costs and enabling quick fine-tuning based on enterprise needs • The startup specializes in creating customized models for individual companies, adding real value to their data CEO Ganesh Gopalan emphasizes the importance of multimodal models, focusing on voice and text fusion. By taking their models to enterprises and helping them build on top of it, Gnani AI is revolutionizing the conversational AI landscape in India. #ConversationalAI #VoiceFirst #IndianLanguages #Funding #Expansion

12

1 Comment

𝐁𝐞𝐡𝐢𝐧𝐝 𝐭𝐡𝐞 𝐒𝐜𝐞𝐧𝐞𝐬 𝐨𝐟 𝐈𝐏𝐋 𝐒𝐭𝐫𝐞𝐚𝐦𝐢𝐧𝐠 𝐚𝐭 𝐉𝐢𝐨: 𝐀 𝐋𝐨𝐨𝐤 𝐚𝐭 𝐭𝐡𝐞 𝐓𝐞𝐜𝐡 𝐌𝐚𝐠𝐢𝐜! 🤯 Ever wondered how JioCinema manages to stream IPL matches flawlessly, even with millions of viewers tuning in? 🏏 Well, I recently listened to a fascinating podcast of Arpit Bhayani with Prachi Sharma, a Sr. Engineering Director at JioCinema and she shed light on the intricate tech behind those smooth streams. 𝐺𝑖𝑣𝑒 𝑖𝑡 𝑎 𝑤𝑎𝑡𝑐ℎ - youtu.be/36N1Bz7qW0A  🎼 𝑺𝒉𝒆 𝒓𝒆𝒗𝒆𝒂𝒍𝒆𝒅 𝒔𝒐𝒎𝒆 𝒐𝒇 𝒕𝒉𝒆 𝒔𝒆𝒄𝒓𝒆𝒕𝒔 𝒕𝒐 𝒕𝒉𝒆𝒊𝒓 𝒔𝒖𝒄𝒄𝒆𝒔𝒔: 𝗣𝗹𝗮𝗻𝗻𝗶𝗻𝗴 𝘀𝘁𝗮𝗿𝘁𝘀 𝗺𝗼𝗻𝘁𝗵𝘀 𝗶𝗻 𝗮𝗱𝘃𝗮𝗻𝗰𝗲: They don't just switch on the servers when the first match starts. Planning and meticulous audits of the entire system, from front-end to back-end, begin months before the IPL season. 𝗙𝗲𝗮𝘁𝘂𝗿𝗲 𝗙𝗹𝗮𝗴𝘀 𝗳𝗼𝗿 𝘀𝗺𝗼𝗼𝘁𝗵 𝘀𝗮𝗶𝗹𝗶𝗻𝗴: To avoid any glitches, they use feature flags to turn on and off specific functionalities during the match. Imagine having a switch to turn off certain features on the fly if they're causing issues - that's the power of feature flags! This allows them to isolate any problems and ensure the core viewing experience remains unaffected. 𝗚𝗿𝗮𝗰𝗲𝗳𝘂𝗹 𝗗𝗲𝗴𝗿𝗮𝗱𝗮𝘁𝗶𝗼𝗻 𝗳𝗼𝗿 𝗮 𝘀𝗲𝗮𝗺𝗹𝗲𝘀𝘀 𝗲𝘅𝗽𝗲𝗿𝗶𝗲𝗻𝗰𝗲: Even if something goes wrong, they have strategies in place to minimize the impact on viewers. For example, they have a backup system where data can be processed later, ensuring no one misses the action. 𝗣𝗿𝗲-𝗦𝗰𝗮𝗹𝗶𝗻𝗴 𝘁𝗵𝗲 𝗗𝗮𝘁𝗮𝗯𝗮𝘀𝗲: They don't just rely on auto-scaling. They proactively scale up their database infrastructure before the match based on anticipated traffic load. This ensures that the database can handle the peak demand without any lag or outages. 𝗧𝗵𝗲 𝗽𝗼𝘄𝗲𝗿 𝗼𝗳 𝗰𝗮𝗰𝗵𝗶𝗻𝗴: The way they use caching is crucial to optimize performance, ensuring that viewers don't experience lag. They even pre-cache content before the match starts to avoid overloading the database. Think of it like a pre-cooked meal, they have the most popular dishes ready to go, so there's no waiting for the database to cook up the information! 𝗧𝗵𝗲 𝗶𝗺𝗽𝗼𝗿𝘁𝗮𝗻𝗰𝗲 𝗼𝗳 𝗗𝗡𝗦: They discovered a unique challenge where a small ISP wasn't updating its DNS records, causing issues for Jio Cinema users. They added a check in the front-end to detect if the app can't resolve DNS properly, and then automatically switches to a more reliable DNS server, such as Google DNS. 𝗕𝗮𝗰𝗸𝗼𝗳𝗳 𝗥𝗲𝘁𝗿𝗶𝗲𝘀 𝗮𝗻𝗱 𝗦𝘁𝗮𝘁𝗶𝗰 𝗗𝗮𝘁𝗮: They implement strategies like exponential backoff retries for API calls. This means that if an API call fails, the system tries again, but with increasing delays between each attempt. They also have the ability to switch API calls to deliver static data if needed, ensuring that the user doesn't experience a complete outage. #IPL #JioCinema #TechInsights #StreamingMagic #BehindTheScenes 

11

Lots of posts on Zomato about new features on "re-ordering food from a cancelled order." Folks are discussing -- what are loopholes, how to enhance it, how to make it fullproof, and so on.   Have you given this much time and thoughts on the feature or product that you are working on ??   Many don't know why they're building a particular feature; they don't know what's in the business usecase or what value it adds. Many will just say -- the product team asked, so we are adding this feature. #thoughlessness #softwareengineer #product

13

LLVM Social Bangalore is back with an online meet. And this time, we’ll be hosting Prof. Manas Thakur from Indian Institute of Technology, Bombay on a very interesting topic about #JIT #Compilers! Title: Fixing the see-saw: Enabling precise optimizations in JIT compilers, efficiently Date: 18th May 2024 (Saturday) Time: 3:00 PM onwards Venue: Online (Google Meet) RSVP: https://lnkd.in/gJeMR_2V Note : You must RSVP to get the meeting link! Please do this ahead of time. Abstract (In the Professors own words): Just-in-time (JIT) compilers offer novel ways to optimize programs based on their execution profile and have found significant usage in languages with managed runtimes. However, as the JIT-compilation time directly affects the execution time of a program, precision-enhancement techniques from the static-analysis world seldom find way into these otherwise powerful systems. In this talk, I would first illustrate the above challenge using examples of program analysis aimed at enabling stack allocation of heap objects in Java. We would see how we have developed techniques to not only allocate a large number of objects on stack (leading to reduced garbage collection and improved performance), but to also handle challenges posed by possible dynamism in Java Virtual Machines. We would also understand how we facilitate code specialization in even newer dimensions by reusing compiled code for the lazy dynamic language R. All parts of the story would be bound by our encompassing goal: efficiency during program execution. Website: https://lnkd.in/g664qcVy Research Group: https://lnkd.in/gSHMyV8R Google Scholar: https://lnkd.in/gxZNaDni #JustInTime #CompilerDevelopment #IITB #stackallocation #heap #staticanalysis #pl #research #performance #java

122

5 Comments

Generative AI has been the talk of the town in recent years. Its paramount ability to efficiently solve diverse business problems using minimal resources and time has attracted the attention of leading market players. We've witnessed a flood of AI products being launched recently. ChatGPT has become my favorite tool of late. It can save you hours if utilized correctly. It can be employed for product recommendations, content creation, and myriad other tasks. By customizing the fundamental model, we can develop various generative AI assistants to address these use cases. I invite you to read my new blog on developing a custom generative AI assistant using AWS services 📰 #ai #generativeai #aws https://lnkd.in/eXAdYdRC

15

🚀 Just launched a unique experiment merging Gen AI and Bollywood! In my latest video, I test ChatGPT and RAG with Sentence Window Retrieval on a tricky Andaz Apna Apna question, and the results are eye-opening. 🎥 Watch as RAG’s accuracy shines where ChatGPT stumbles—perfect for data science enthusiasts and Bollywood fans alike! Check it out and see how Gen AI can really connect with our favorite movies script ! #GenAI #DataScience #Bollywood

9

IPL: Razorpay's Second Innings 🏏 The title "IPL: Razorpay’s second innings" really intrigued me. What does a fintech platform have to do with IPL? 🤔 It turns out, user behavior during an IPL match, like ordering food or gaming, had an indirect impact on Razorpay. In 2019, they struggled with traffic bursts during IPL matches, which lasted over 10 minutes—something their engineering hadn't anticipated. After this challenging IPL season, they strategized to handle future traffic bursts better. Here are the key problems they identified and solved: Problems: 🧵 Thread Exhaustion: PHP Laravel, their backend, lacked proper DB connection pooling, blocking threads and connections when the gateway took longer. 🚦 Throttling: They weren’t throttling requests, making them vulnerable to DDoS attacks. 🏦 Banking Gateway Backpressure: Banking gateways experienced 20x their normal P99s during traffic bursts. 🚥 Smart Routing Requests: There was no way to identify the load on different banking gateway nodes to route requests to less loaded ones. 📊 Alerts & Monitoring: Their alerting mechanism was too slow, giving them no time for remediation. Solutions: 🧵 Thread Exhaustion: They used ProxySQL as a layer between the backend and MySQL, deploying it as a sidecar with each backend pod for fault tolerance. 🚦 Throttling: Implemented simpler fixed window rate limiting instead of the earlier one which was a leaky bucket algorithm implementation backed by a Redis store, also moved this service to an NGINX-based proxy server as a side car with dedicated cache and new rate-limiting configurations (e.g., requests per route per merchant). 🏦 Easing Banking Gateway Backpressure through Smart Routing: Built a machine learning-based system to route transactions smartly, considering earlier payment success/failure events. This improved success rates by 60% during bursts and 4% during normal periods. 📊 Alerts & Monitoring: Using Victoria Metrics and Grafana, they developed a real-time alerting system for business and engineering KPIs. This is how Razorpay overcame their engineering challenges! For more interesting engineering stories, follow me. https://lnkd.in/gbzx8qen #Engineering #Fintech #Scalability #MachineLearning #DevOps 🚀

9

💡Ever wondered how delivery giants like Swiggy use technology to optimize delivery during peak traffic conditions? Well, they apply something called grouping of orders where they batch orders based on a number of parameters like - 1. Geographic closeness: Orders that are geographically closer are grouped together 2. Delivery timings: Orders with similar delivery times are grouped 3. Weight and volume of order: Their algorithm also considers parameters like the weight and volume of the orders to ensure driver safety. These are some parameters among many others that their algorithm processes. While these optimisations do lead to faster delivery, but a more pronounced impact is that it translates to fewer rides by the delivery agents. And fewer rides mean lesser carbon emissions, therefore lesser environmental impact.✨️ Watch the full video on - https://lnkd.in/g_5aaRPJ And subscribe to my YouTube channel 'Learn With Spreeha' for more such tech stories! #technology #algorithms #swiggy #zomato #LearnWithSpreeha

15

Spent some time today going through Meesho's blog on how they reduced 90% of their Redis infra cost and it is filled with interesting details ⚡ The blog covers their design, key decisions, and nuances around picking the right data structure and tuning Redis the right way. Give it a read, it is pretty informative. The high-level overview of it 1. key-value based initial design leading to bloated memory 2. moving to Redis Hash 3. it still bloats memory, but beyond a threshold 4. internals details on how Redis uses ziplist to implement hash 5. tuning `hash-max-ziplist-entries` configuration 6. minimizing calls to Redis using Lua script I have seen many people adopting KV based approach and then throwing money on infra to handle scale. But Meesho's blog highlighted how knowing database tunables can help you significantly reduce your infra costs. give it a read - https://lnkd.in/gp9N3JMh To be honest, this is a pretty interesting blog for anyone using Redis in production. I made a similar blunder back in 2020 and then spent 2 weeks optimizing it, so a hard relate :) #AsliEngineering #SystemDesign

663

16 Comments

Discussed the ABCs of a PM career with Mahendra in this conversation - do check it out if you are an aspiring PM, and let us know your thoughts! Thanks for having me, Mahendra! 😀

21

3 Comments

Agent API: Build AI Agents in 5 Minutes!

4

🚨 Beware of Online Stock Trading Scams🚨 In a shocking turn of events, three individuals in Bengaluru have lost a total of ₹9.54 crore to online stock trading scams within just one week. The victims, including two businessmen and an engineer, were deceived by promises of high returns and early access to IPOs. The scammers added the victims to WhatsApp groups where they shared investment tips. The victims were then persuaded to download trading apps and transfer large sums of money, believing they were making substantial profits. Initially, they received a small profit to keep them engaged, but the others saw no returns. Stay vigilant and always verify the authenticity of investment opportunities. Let’s work together to prevent such scams and protect our community. #CyberSecurity #OnlineScams #InvestmentFraud #StaySafe Source: https://lnkd.in/gEhbZtP5

3

🚀 𝗕𝗿𝗲𝗮𝗸𝗶𝗻𝗴 𝗜𝗻𝘁𝗼 𝗠𝗟 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴 𝗶𝗻 𝟮𝟬𝟮𝟱! 🚀 So excited to share insights from a fantastic session led by Kartik Singhal, Senior Machine Learning Engineer at Meta, who provided invaluable guidance on breaking into the field of ML Engineering in 2025. Thanks to Meri Nova and Break Into Data for bringing this session to life! Here’s a quick recap of Kartik’s key points for aspiring ML Engineers: 𝗛𝗮𝗻𝗱𝘀-𝗢𝗻 𝗣𝗿𝗼𝗷𝗲𝗰𝘁𝘀: It’s not just about theory—build, build, build! Showcase your work on GitHub and work on diverse ML projects. This experience is invaluable when applying for roles. 𝗦𝗽𝗲𝗰𝗶𝗮𝗹𝗶𝘇𝗲: With the vastness of the field, choose an area that excites you. Specializing can set you apart whether it’s NLP, Computer Vision, or Reinforcement Learning. 𝗦𝘁𝗮𝘆 𝗨𝗽𝗱𝗮𝘁𝗲𝗱: Machine learning evolves quickly, and tools change frequently. Stay current with research papers, join communities, and consider advanced courses or certifications in new technologies and trends.  𝗡𝗲𝘁𝘄𝗼𝗿𝗸𝗶𝗻𝗴: Connect with professionals in the field, join ML communities, attend conferences, and reach out to mentors. This can lead to opportunities and keep you motivated on your journey. Big shout-out to everyone who attended and asked thought-provoking questions! Thank you to 𝗠𝗲𝗿𝗶 𝗡𝗼𝘃𝗮 and 𝗕𝗿𝗲𝗮𝗸 𝗜𝗻𝘁𝗼 𝗗𝗮𝘁𝗮 for making this session possible—here’s to helping more aspiring ML engineers break into the field! 🚀 #MachineLearning #DataScience #CareerAdvice #BreakIntoData #MLengineering #Networking #CareerDevelopment

43

1 Comment