Vivek Purushotham

Thanks People+ai! Check out the video below to know what I am up to these days! I am building hardware to make AI accessible to all!

59

5 Comments

Excited to be presenting at The Future Memory and Storage 2024 (FMS'24) next week, with Navneet Rao. As data sets get larger, optimizing data movement between the different compute elements CPU, DPU, GPU etc is critical to achieving better efficiency and TCO. This talk focuses on how CXL can be used to minimize data movement when a processing pipeline is composed of heterogenous compute elements. Thanks Ajay Gupta for the support. Paper Title: Improving SW-HW processing pipeline for storage stack/service workflows w CXL Paper Abstract: Storage services & workflows are adopting disaggregated storage architecture enabled by newer ethernet standards: 400/800GE. This also necessitates need for HW accelerators for operations like compression, encryption, erasure coding, deduplication etc with some usages requiring operations at storage initiator. These operations require data context to be translated from memory into storage context right at the Storage initiator (typically, CPU) before transmission over 400/800GE to the destination media block/object/file storage. The storage stack thus is a blend of protocol processing & multiple translations enforcing data segmentation using SW-based execution or through accelerator usage (e.g., TLS encrypt / decrypt, NVMe data digest, etc.). Almost always, requires multiple data movement between cpu memory domain and lookaside accelerator memory domains with PCIe link becoming the bottleneck. This paper proposes a CXL based implementation to preserve memory context of data as long as possible, chaining storage and network functions to deliver higher performance for both local & disaggregated storage accesses: block, object, file across varied storage media: volatile, ephemeral, non-volatile. The solution will enumerate CXL concepts to enhance current IO paths and show how adopting CXL dataflow (Type1 / Type2 / Type3) allows for data plane in only one memory domain with control plane straddling between cpu and accelerator memory domains. This delivers higher throughput, more IOPs per node & also reduces deployment cost by enabling solution upgrades requiring slower hardware refresh cadence.

65

2 Comments

Its been great to be part of Nvidia AI Summit in Mumbai, India to witness and experience the latest advancements in Nvidia AI on recent acceleration libraries, to know that Blackwell GPUs to be shipped from Q4 and a very thoughtful and inspirational fire side chat between 2 icons Jensen Huang and Mukesh Ambani. As per Jensen, Nvidia is AI in India (except for letter V resembles closely with India) which is all set to take a center stage in the world with AI Infrastructure coming up (20 times more compute in India compared to last year) and Application developer ecosystem with heavy training across the stack As per Mukesh Ambani, Nvidia has Vidia in it, which translates to Knowledge (Vidya) and those who sincerely strive with knowledge, prosperity follows. He is all set to disrupt with Jio huge Data centers coming up. Its interesting to have own green power from Reliance to power up their own DC which can make the operational costs very competitive and a great benefit to all players in India While the initial focus is India, this has the great potential to transform the world to solve many use cases across many domains Exciting times at Nvidia AI Summit, Mumbai, India NVIDIA #AI

106

4 Comments

Google's Trillium, the successor to its Tensor Processing Unit (TPU), marks a significant leap in AI workload performance and efficiency. With enhanced compute performance, increased memory bandwidth, and scalability, Trillium empowers developers to train next-generation AI models faster and deploy them more effectively. Key features include expanded matrix multiply units and increased clock speed for enhanced compute performance, doubled High Bandwidth Memory capacity for larger models, and support for scaling up to 256 TPUs in a single pod. Trillium also introduces specialized dataflow processors for accelerating models that rely on embeddings, and multislice technology for scaling AI models across multiple TPU pods, supporting up to tens of thousands of chips. Additionally, Trillium's open source support for libraries like JAX, PyTorch/XLA, and Keras 3 enables seamless integration with existing development workflows. #google #ai #TPU https://lnkd.in/gqqPfTUp

4

Hey there, LinkedIn! I’m Venkata Ramakrishna Yamijala, but feel free to call me Ram! I’m a Technical Writer at Broadcom Inc., and over my 17-year career, I’ve been dedicated to transforming complex technical information into content that’s clear and user-friendly—because nobody should need a Ph.D. to understand a user manual, right? After all these years, I’ve come to realize that learning should never stop, especially in our fast-evolving tech world. That's why I decided to go back to school and enroll in the M.Tech in AI & ML program at IIT Jammu through Intellipaat. It’s been an incredible journey so far, diving deep into the world of Artificial Intelligence and Machine Learning, and discovering how these technologies can revolutionize technical writing and instructional design. Education isn't just about getting degrees; it's about staying curious, constantly upskilling, and being prepared for the future. Even after nearly two decades in the field, I believe there’s always more to learn, more to explore, and more ways to innovate. And what better place to do that than at IIT Jammu with Intellipaat? Their support and cutting-edge curriculum have been crucial in this exciting new chapter of my career. Imagine a world where tech manuals practically write themselves—well, maybe not quite yet, but we're on our way! Thanks to Intellipaat for the top-notch education and support. I’m excited to continue this journey and explore how these advanced technologies can transform the way we create and consume educational content. To all my fellow technical writers and industry colleagues: if you think education is just for the young or when you’re starting out, think again! Learning is a lifelong journey, and it’s never too late to upskill and innovate. Whether you're looking to stay ahead of the curve or just curious about what all this AI buzz is about, consider this your nudge. And who knows? Maybe one day, our AI tools will appreciate a well-crafted user guide as much as we do! 

76

5 Comments

Intel has faced several challenges recently, including increased competition, production delays, and shifts in demand for chips. Their transition to newer manufacturing processes has been slower than expected, impacting their market position. Additionally, the rise of AMD and Nvidia has intensified competition in both CPU and GPU markets. As a result, there have been fluctuations in their stock price and market performance. However, Intel is actively working on strategies to regain its footing, including investments in new technology and partnerships. Intel is focusing on several key areas for its future: Advanced Manufacturing: They're investing heavily in new fabrication plants and technologies to enhance their production capabilities and catch up with competitors like TSMC. AI and Data Centers: Intel is prioritizing AI and data center solutions, aiming to expand its presence in these rapidly growing markets. Product Diversification: They’re developing new chip architectures and enhancing their offerings in both consumer and enterprise markets, including gaming and edge computing. Strategic Partnerships: Intel is forming partnerships with other tech companies to leverage synergies and expand its ecosystem. Long-term Roadmap: The company has laid out a long-term roadmap, with plans for new CPU architectures and innovations in the coming years. Overall, while Intel faces significant challenges, they are taking steps to adapt and reposition themselves in the competitive semiconductor landscape. Intel has experienced a notable decline in revenue, particularly due to factors like reduced demand for PCs, supply chain issues, and increased competition. Their financial struggles have raised concerns among investors and analysts. Despite these challenges, Intel is implementing a turnaround strategy focused on new product development, advanced manufacturing, and expanding into growth areas like AI and data centers. The effectiveness of these efforts will be crucial for their recovery in the coming years. Gelsinger has been at the helm during a challenging period, but he has also initiated significant changes and a turnaround strategy focused on innovation and manufacturing. Supporters argue that he brings necessary leadership and vision, while critics may point to ongoing struggles. Ultimately, whether he should stay or go depends on the effectiveness of his strategies in revitalizing the company and responding to market pressures. Stability and long-term vision are crucial in such a critical phase for Intel.

2

If you're at #FMS2024, come join me and an amazing group of co-presenters—Assaf Sella from Kioxia, John Lorenz from Yole Group, Vishwas Saxena Western, and Aadharsh Kannan from Western Digital—at the AIML-203 Generative AI Application Track. Don't miss this interesting session on Generative AI, where we will cover four important topics: ·      The of the importance of Flash for Rag-Based LLMs  ·      Gen AI Foundation to use cases,  ·      Generative AI: Memory Market Impacts ·      LLMs on the Edge Devices This session offers invaluable insights and good set of takeaways for both seasoned professional or just beginning your journey in Gen AI. Mark your calendars Aug 7th: 3 to 4.05 PM PT. #GenerativeAI #AIML #FutureOfMemory #StorageSolutions #TechInnovation #LLMs #EdgeComputing #fms2024 #pliops

42

2 Comments

This is a bold move : or at least it would be if it existed in real life. Is this the day when we we are all done with the algorithms, where there is no optimisation left to do, and transformers are as good as they are ever going to get, so we burn them to ASIC ? And what is more : important not sure it's an ASIC : it looks like a hybrid circuit to me. It's an order of magnitude faster than a B100 : a 10X speedup seems low considering the tradeoff of freezing an unoptimised design. Will Meta Layer Skip etc overwhelm the productivity gins from ASIC even before this leaves the factory ? https://lnkd.in/e5ikcCmh I mean : this was inevitable, but surely now is not the time ?

1

1 Comment

It is very important to reduce KV cache memory consumption during long context inference. We introduce ThinK, a method that exploits substantial redundancy across the channel dimension of the KV cache. Our findings reveal that even KV caches optimized by eviction methods, such as SnapKV and H2O, it can still benefit from pruning approximately 40 to 50% of their key cache channels. Our approach results in achieving over a 20% reduction in KV cache memory compared with SnapKV and H2O. Paper: https://lnkd.in/gxpeVdC9

154

Nvidia CEO Jensen Huang claimed that Nvidia had spent around $10 billion in research and development for Blackwell's NV-HBI die interconnect. Veteran semiconductor engineer Jim Keller, who had worked on AMD's K7, K12 and Zen architectures, criticized this figure and claimed that the same outcome could be achieved for $1 billion through using Ultra Ethernet rather than the proprietary NV Link system. [ https://lnkd.in/gw-WwFJ6] The two connected GB100 dies are able to act like a large monolithic piece of silicon with full cache coherency between both dies. The dual die package totals 208 billion transistors. Those two GB100 dies are placed on top on a silicon interposer produced using TSMC's CoWoS-L 2.5D packaging technique. Read more: https://lnkd.in/gw-WwFJ6

1

2 Comments

Our goal is to simplify AI and accelerate your journey from prototype to deployment. It was great working with Siddhika Nevrekar on this blog post, which highlights our vision and the efforts to democratize AI. Read more here: https://lnkd.in/gmBijf7B

12

India is making significant progress in developing its semiconductor ecosystem. Government initiatives and programs aimed at attracting investments are already showing results. In an interview with EETimes, Jaswinder Ahuja discusses the remarkable progress and promising future of this sector. He also highlights how these efforts are accelerating growth and positioning India as a key player in the global semiconductor market. Read the full interview here: https://ow.ly/ATTq50T0EOk #Cadence #SemiconductorIndustry #IndiaRising #TechInnovation #JaswinderAhuja #EETimes #Investment #GovernmentInitiatives #FutureOfTech

54

Don’t miss out! Vaibhav Katkade, Sr. Product Manager at Google Cloud, is set to share cutting-edge best practices for architecting Google Cloud for observability. Discover how real-time, end-to-end monitoring can give you the insights you need to quickly identify and resolve network issues. Secure your spot now and stay ahead of the game!

3

Really enjoyed reading this blog from Partha Ranganathan that clearly explained Google Cloud's motivation to build their own custom cloud CPU #onArm: Performance, flexibility, efficiency and sustainability. "The flexibility of Axion is essential if we want to more quickly and broadly meet the infrastructure needs of our customers and partners — especially in the fast-moving age of AI." https://lnkd.in/gJu-BwC8

27

5 Comments

CAS latency is a critical factor in RAM performance—it determines how quickly your memory responds to CPU requests. Understanding it can help optimize your system’s speed and efficiency. #TechInsights 👇 https://lnkd.in/gHxExww4

11

Our trajectory with Google Cloud epitomizes a transformative strategic pivot. What began as a challenging market positioning has evolved into a compelling narrative of operational resilience and technological leadership. Over seven pivotal years, we've fundamentally reimagined our enterprise value proposition—transitioning from a peripheral cloud solution to a mission-critical AI innovation partner. The financial metrics tell a powerful story: we've systematically converted recurring quarterly deficits into a robust framework for sustainable profitability. Buckle up the next 7 years are going to be epic @ Google Cloud! 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 🚀 #google #googlecloud #GCP #gcpcloud

26

1 Comment

What happens if your CPU gets something wrong? If it wakes up one day and decides 2+2=5? Well, most of us will never have to worry about that. But if you work at a company the size of Google, you do, which is why this paper on "mercurial cores" is so fascinating. What the authors report--and supposedly this is common knowledge at the hyperscalers--is that a couple cores per several thousand machines are "mercurial." Due to subtle manufacturing defects or old age, they give wrong answers for certain instructions. These can cause all sorts of impossible-to-diagnose issues. Some rare problems at Google that were traced back to bad CPUs include: - Mutexes not working, causing application crashes - Silent data corruption - Garbage collectors targeting live memory, causing application crashes - Kernel state corruption causing kernel panics What makes CPUs go bad? It's very hard to tell. The authors posit that issues are becoming more frequent as CPUs get more complex, but there aren't solid numbers behind that. There are certainly strong relationships between frequency, temperature, voltage, and bad CPU behavior--most mercurial CPUs only cause problems under very specific conditions, but those conditions vary from CPU to CPU. Age is another source of problems, as older CPUs are more likely to exhibit problems. Bad CPUs are an especially serious problem because they're very hard to detect. If cosmic rays flip bits in storage or on the network, that can be detected through error coding. But there's no analogy for a CPU that allows cheap online verification of its correctness. Instead, the best detection techniques involve monitoring for symptoms. If a core exhibits exceptionally high rates of process crashes or kernel panics relative to its fellows, that's a strong indication something is wrong with it. For the most critical applications, the authors propose triple modular redundancy--redoing each of its computations on three cores and majority-voting a reliable result. More than anything, this paper is a call to action--letting everyone know that CPUs can fail. So now, if you ever find a bug you can't diagnose, you can blame the CPU!

10

1 Comment

I got a situation where i need to schedule some data transfer from Compute Engine (Linux VM) to GCP Cloud Storage. For this task one of the best method is using google cloud SDK (Installed this in your VM). But the size of this SDK is almost (300 MB - 1 GB). When you have a little storage for your job then it's really hard to afford 1 GB from production. So, I solved it by using Service Account Key. You need to create a Key from the service account (Need to have access to do this job). Then just inside VM you need to write a simple Script (In my case It was PHP), and then schedule your job that can fetch and send your data to the cloud storage. And you can also schedule your remove job (Script) that can remove cloud storage data from the VM. #GCP #devops #operations

6