Karthik TJ

Using generative AI, NVIDIA operations built an AI Planner agent, developed on NVIDIA Inference Microservices (NIM). The agent leverages LLM, NeMo Retriever and CuOpt NIM to reduce re-planning time from hours to just seconds. https://nvda.ws/4bIfS2T

2

Did you know there are no "roles" (SDE, SDET, TPM, …) when you get to L8 IC (Senior Principal) at Amazon? It’s one of Amazon’s peculiarities. [Side note: in general, Amazon SDEs tend to be "jack of all trades" where we pragmatically do whatever is necessary to make our organization successful at that given moment (within reason). In fact, I believe being well-rounded and able and willing to wear a number of hats is one of the things that makes me valuable to an employer. We don’t over-index on "that’s not part of my role doc." Other companies where I worked (eg Google) have very clear swimming lanes and people stick to them pretty religiously, eg. "oh no I don’t talk to customers, I’m an SDE and talking to customers is for TPMs." Whereas at Amazon I never once had a manager tell me, "what you did (although impactful, important and necessary) was not really your job." ] My little side note about this critical piece of amazon culture explains why as you get to L8 the concept of a "role" no longer really makes sense. The boundaries between roles get fuzzier and fuzzier as you go from Senior Engineer (L6) to Principal Engineer (L7), and they become unimportant when you get to L8. We’re all just “Senior Principals” - whether you came via the SDE path, the TPM path, or in my snowflake case, the SDET path. In fact I think most L8s are snowflakes in that their path to L8 was unique and the value they bring to the company is also unique. In phonetool (our amazon directory) you can have whatever unofficial title you want (also within reason...), so I chose to keep the "SDET" title to reflect my background and passion for testing, engineering and operational excellence, and developer tools. I also hope that by keeping the title, others in my (old?) role can see that SDETs at Amazon do not have a "ceiling" in their career. And lastly, it's a fun conversational piece ("you're an L8 SDET???") (The image here is "Peccy" - a beloved little icon that has represented Amazon's Peculiar Ways for years) [Edit: Strictly speaking there are a few L8 IC roles that do have a different job code, like PMT or Scientist - but I think we generally view all L8 ICs in the same lens]

929

41 Comments

Using generative AI, NVIDIA operations built an AI Planner agent, developed on NVIDIA Inference Microservices (NIM). The agent leverages LLM, NeMo Retriever and CuOpt NIM to reduce re-planning time from hours to just seconds. https://nvda.ws/4bIfS2T

20

1 Comment

🚀 Exciting Opportunity Alert 🚀 Are you a Principal Engineer passionate about Distributed Machine Learning? Check out this role in Santa Clara, CA: https://bit.ly/3Zl5M4O. Key skills: #gpu #apache #spark #machinelearing #ml #dl #deeplearning #acceleratedcomputing #cluster #rapids #mLlib #XGBoost #PyTorch #cuML #pytorch #tensorflow #apis #api #Kubernetes #YARN #Ray #C++ #Scala, #Python #nvtabiular #cuda #horovod #petastorm #dll. Interested? Reach out to [email protected] for more details. Opportunity awaits! 🌟 #engineering #machinelearning #distributedcomputing

3

You can now create a million S3 buckets per account, but the buckets beyond two thousand will now be charged at a rate of $0.02 per bucket. That’s… I have no words. If you were relying on quotas to prevent bucket sprawl, well you’ll have to change that. https://lnkd.in/ge7_FacW #aws #s3

15

Hello, colleagues, this Yahoo article is interesting and relatable for professionals. Here is a teaser: "Amazon has an incredibly granular list of criteria that need to be met before you can be considered for promotion, so I worked with my manager to identify the gaps in my performance, ... and I created a project for each gap that, if completed, could serve as evidence of my ability. I worked backward to create a strict timeline to finish my projects ... that helped me get systematic about my approach moving forward." https://lnkd.in/dsfvPeAk Take a lesson from this person. I seems Amazon gets very analytical when it comes to promotions. And in a company that size, it is impossible to not promote without rigid check-off type policies or whatever XYZ job promotion software. But, hey, there is a lot I will do for a $500,000 salary. Please take note of this last sentence, if anyone works for Amazon and wants a Quality Engineering Manager/Director anywhere in the world.

1

1 Comment

Here is a quick look at how to achieve high-quality semantic search. Lot of exciting work happening in this space. Solving real world problems with AI. #googlecloud #vertex #AI #google

31

Own your promo doc, part 2 out of 2 Part 2 is the story of writing my Senior Engineer to Principal Engineer promo (2013-2014). (Part 1, the story of writing my SDE-II to Senior Engineer promo, is at https://lnkd.in/gMHMzFR4) One thing I wanted to do differently was that the doc should be Forward-Looking (“here’s what I’m going to do” - proactive) rather than the standard “here’s what I did” (reactive). Amazon tends to follow a process called "Working Backwards" in a lot of docs - and I figured it would apply well to promos. I wrote the Forward Looking doc well before I thought I was ready. It didn’t just list the things I had accomplished, but the things I felt I could/should accomplish in the next 12 months. I then shared the doc with a selected group of Directors and Principals around me to seek feedback. The basic question was, “if this document was being discussed a year from now and everything in it was then true, would you be Inclined or Not Inclined? Does it ballpark describe the body of work of a Principal?” It was not a comfortable experiment: I certainly put myself out there and felt very vulnerable. The people I sought feedback from were also very direct and candid and challenged me in uncomfortable ways, but I knew they cared about my growth. Their advice helped me to be focused and deliberate on where I spent my time in the next year. When I felt the doc was good enough, I shared it with my manager. I didn’t get promoted the first time around, but I did get promoted six months after my first failed attempt. That was roughly a year after I initially wrote the doc. This is pretty standard for PE promos - if you get promoted the first time around chances are your manager waited too long! Years later, I was having coffee and catching up with the manager that promoted me to Principal. He said something curious: “You know, I was planning on promoting you, but your doc helped me do this six months earlier!” I had spent about 30 hours of my life writing, revising, incorporating feedback, and thinking about it. I don’t remember what kind of financial benefit getting promoted half a year earlier had for me specifically at the time, but let's use the following thought experiment: just using today’s public numbers from levelsfyi, Amazon L6 averages $405k, Amazon L7 averages 680k, so the difference is $275k/yr. Having L7 compensation six months earlier would mean an additional 138k. Given 30 hours of additional work, that’s an ROI of $5000/hr. I'd say that's a pretty decent ROI for that investment of time! (there were many other benefits to being promoted earlier, but that one is easy to quantify and articulate)

362

22 Comments

Seeking a full time onsite hybrid AI Architect Engineer Manager for leading RAG and LLM Generative AI role to relocate and lead RLHF, prompt engineering for zero shot learning, one shot learning, few short learning. Engineer Architect or Manage and lead teams on Fine Tuning LLM models on PEFT, RLFL, Prompt Tuning context window, and to work on Reward model using LoRA, and to work on agent to instruct LLM on Context window and lead AI Generative LLM RAG performance for scalability. Engineer Architect or Manage and lead Generative AI LLM Lang Chain Use cases, and work on Hugging faces for encoded/weights. Engineer Architect or Manage and contribute to building RAG Retrieval Augmented Generation to retrieve external library data for tailored to specific models domain in prompt analysis. Additional talents that I have which include People first partnerships, deliver on AI architect accountability and ensure focus on diversity inclusion for delivery business value. Manage stakeholders data projects to improve business revenue. People talents that I offer are people loving manager who is humble and offers critical creative innovative AI thinking with the team. Engineer Architect or Manager the architecture for AI Lang Chain Generative AI Use Cases that need LLM text words. Contribute as Engineer Lead on improved production pipeline supply chain signal processing and be the Engineer Architect for large language text analysis (LLM). Research for teams can be more productive in using Lang Smith for LLM large language models. Assist on curating sandbox to develop bidirectional decoder/encoder transformer (BERT) masking language next sentence (MLM). Lead again on OpenAI GPT for optimized performance to build chat what-if analysis for the business. Manage Engineer to lead scaling the team on development with performing LLM Chain and predictions models and build predict messages that pull from the Hugging Face Hub objects and prompt engineering with prompt templates and human message responses.

3

Amazon S3 now supports conditional writes!! .... but why should you care? Because of these 5 reasons.. 1. Prevents Overwriting and Ensures Data Integrity: Conditional checks prevent accidental overwrites, maintaining data integrity while allowing successful creation of new objects. 2. Simplified API Usage and Server-Side Validation: Clients can perform writes and existence checks in a single operation, with Amazon S3 handling all conditional logic, reducing API calls and offloading responsibility from clients. 3. Efficient Parallel Processing for Large-Scale Workloads: Multiple clients can write concurrently without risking data loss, making it suitable for distributed operations like analytics or machine learning tasks. 4. Eliminates Client-Side Coordination and Improves Performance: By removing the need for complex client-side consensus mechanisms and reducing API calls, overall system performance is improved. 5. Consistent Behaviour Across Operations: The same conditional logic applies to both PutObject and CompleteMultipartUpload operations, providing a uniform approach to data management. Biggest one for me is the simplification of client logic. Follow Kaivalya Apte if you like #SoftwareEngineering #DistributedSystems #Databases #StreamProcessing

46

4 Comments

🚀 **System Design Spotlight: The Power of Event-Driven Architecture** 🚀 As a senior software engineer with the privilege of having worked at Google and Microsoft, and having taught thousands of engineers at Scaler, I’m constantly fascinated by the innovations in system design that drive our industry forward. Today, I want to share insights on one of my favourite topics: **Event-Driven Architecture (EDA)**. ### Why Event-Driven Architecture? In today’s fast-paced tech world, the ability to process and react to events in real-time is crucial. EDA decouples event production from event consumption, leading to highly scalable and maintainable systems. ### Key Benefits: 1. **Scalability**: Components scale independently, essential in microservices architectures. 2. **Resilience**: Asynchronous event communication enhances system resilience. If one component fails, others continue operating. 3. **Real-Time Processing**: Ideal for applications requiring real-time data processing, such as financial systems, IoT applications, and social media feeds. 4. **Flexibility and Extensibility**: New features or services can be added without disrupting existing ones. ### Practical Example: Consider a modern e-commerce platform. When a user places an order, several events are triggered: - **Order Placed**: Inventory updates. - **Payment Processed**: Accounting records the transaction. - **Order Shipped**: Customer notification. Each event is processed by different microservices asynchronously, ensuring a seamless user experience. ### Tools of the Trade: - **Apache Kafka**: Handles high throughput with distributed streaming. - **AWS Lambda**: Executes code in response to events in a serverless environment. - **RabbitMQ**: A robust messaging broker for reliable message delivery. ### Conclusion: Event-Driven Architecture is a game-changer for building scalable, resilient, and flexible systems. It’s a powerful paradigm that aligns perfectly with modern software development demands. Whether working on microservices, IoT, or real-time data processing, EDA offers a robust solution for complex workflows. If you’re passionate about system design and want to dive deeper into EDA or other advanced topics, let's connect! I love sharing knowledge and learning from this incredible community. 🌟 #SystemDesign #SoftwareEngineering #EventDrivenArchitecture #Microservices #TechInnovation #Scalability

101

2 Comments

📣Happy to share that Amazon Titan Image Generator is now generally available in Amazon Bedrock, giving you an easy way to build and scale generative AI applications with new image generation and image editing capabilities, including instant customization of images. 🙌📷 #Amazon Bedrock #Amazon #Artificial Intelligence #Generative AI

5

📣Happy to share that Amazon Titan Image Generator is now generally available in Amazon Bedrock, giving you an easy way to build and scale generative AI applications with new image generation and image editing capabilities, including instant customization of images. 🙌📷 #Amazon Bedrock #Amazon #Artificial Intelligence #Generative AI

1

Amazon-to-Google-to-Amazon #a2g2a thought of the day: the way Amazon thinks about its Principal Engineers is unique - and I love it. “I always thought of you (and my other Principal Engineers) as Amazon employees that were on loan to me.” Yesterday I was catching up with my good friend and ex-manager Dan Sommerfield. Dan promoted me to Principal Engineer back in 2014 and was my manager for a number of years. As we were laughing about some of our shared past, Dan said that and it stuck with me. A few weeks ago I posted in my a2g2a series about Swimming Lanes. I’ve also posted in the past about the Amazon Principal Engineering community (see links to both of those in my comments). Today’s reflections combine those. Amazon Principal Engineers tend to have a healthy disregard for organizational and role boundaries. And Amazon’s DNA gives them the freedom to do so. At these big faang companies, for better or for worse, most orgs tend to optimize for themselves. It makes sense in terms of business needs: they need to move fast and expand their business. But this is often how we end up with duplicative work siloed usually at Director level. I’ve seen, time and time again: Principal Engineers look across organizations and connect dots that others don’t. The Amazon Principal Engineering community is a giant organic mesh. It worked incredibly well as a big mesh when I became a PE in 2014 and we were about 250 PEs in the entire company, but I’m continuously amazed that it still works in 2024 at today’s scale. This is a testament to the culture that some of our founding figures set decades ago- and our Distinguished Engineers continue to carry the torch today. It’s not that this doesn’t happen in other companies; it did happen at Google and Microsoft too. But it’s so intrinsically embedded in the DNA of an Amazon Principal Engineer that it’s a lot more natural and expected here than anywhere else where I’ve worked or considered working.

594

24 Comments

Understanding how the CPU works and how to write thread-efficient programs is very important. Thread Context Switching - When a program is executed in a single-core CPU, the CPU can work with and execute only one thread at any given time. Each thread gets its share of CPU time. For example, when a thread is writing something to the disk, and if it's consuming time, the CPU switches to another thread while the earlier thread is waiting. Threads are context-switched, and it looks like it is multitasking, and things are going on in parallel, which is not. Processing : Sequential processing is where programs are executed in a sequence, one by one, getting allocated with the CPU Concurrency: Parts of the program can be run independently and combined for the result and to speed up the execution on a single CPU Parallel: Each program can be run on each CPU independently and combined for the final results to speed up the execution Single Vs. Multi-thread - By running multiple threads vs. single threads to execute a program, the overall execution time can be reduced, and efficiency can be improved. Process Vs. Threads: A process uses CPU, memory, disk, and other resources to execute a program. A process can spin up one or multiple threads to complete the job. Sync Vs Asyn: Sync is executed in a sequence and cannot be skipped, but Asyn can run its block of the program separately from the main program and doesn't have to wait for completion to move on Thread Deadlock: When the resources required by a thread are being blocked by another one, and each one is waiting on another one, it leads to a deadlock state #cpu #development #technology #softwaredesign

456

35 Comments