Sriram K.’s Post

Let's revisit one of my top posts regarding 2024 GPU deployments, updating for the 2025 forecast. The Nvidia GB200 GPU, the current holy grail of AI GPU chips... Independent forecasts put the 2025 shipment goal at 4.2 - 4.5million chips. When packaged as an NVL72 rack solution energy demand reaches 130kW for 72 GB200 chips. Coupled with its supporting network deployment, this chip forecast represents up to 9GW of new installed compute base. This is up to 4x the energy demand of H100 chips that were forecasted for 2024, and nearing total energy demand of New York City (depending on actual utilization).

Micron Technology announces 12-high HBM3E Memory, bringing 36 GB capacity and 1.2 TB/s bandwidth. As AI workloads continue to evolve and expand, memory bandwidth and capacity are increasingly critical for system performance. The latest GPUs in the industry need the highest performance high bandwidth memory (HBM), significant memory capacity, as well as improved power efficiency. Micron HBM3E 12-high boasts an impressive 36 GB capacity, a 50% increase over current HBM3E 8-high offerings, allowing larger AI models like Llama 2 with 70 billion parameters to run on a single processor. Looking forward to significant AI innovations as Micron HBM3E gains industry adoption in coming months. https://lnkd.in/g_mnkM3b

As intelligent computing evolves, several trends are shaping the landscape: CPU performance is outpacing Moore’s Law, GPU performance is widening the gap, and deep learning accelerators are surpassing GPUs. To meet these demands, efficient data movement and innovative memory architectures are crucial. The transition to chiplets facilitates best-in-class multi-vendor solutions, scalability, reuse, and long-term improvements in yield and cost. Learn more about the industry's first fabric solution for multi-cluster and multi-chiplet designs: https://lnkd.in/gs3JVVES

The Liqid news just keep coming out of GTC24! New performance white paper on the Liqid UltraStack. We achieve both game changing performance and efficiency leveraging 20 L40s GPU's attached to a single Dell PowerEdge R760 or R7625. #delltechnologies #nvidia #ai #ml #hpc #gtc24

ETHZ CSCS Alps supercomputer makes it way into #top500 at place #6 worldwide. Equiped with the latest NVIDIA H200 GPUs inside Hewlett Packard Enterprise Cray it delivers one of Europes most advanced and powerful #HPC infrastructures for cutting-edge AI research. Where are the other supercomputers found? North America: 181 supercomputers Europe: 157 supercomputers Asia: 147 supercomputers

Are you passionate about helping customers achieve peak performance for their #AI #workloads? Then checkout the #AMD #Instinct #GPUs. They are not your typical consumer graphics cards. They are data center focused GPUs designed specifically for high-performance computing (#HPC) and #artificial intelligence (#AI) workloads. https://lnkd.in/gJQkuRxX. Here are some key characteristics of AMD Instinct GPUs that differentiate them from consumer options: Focus on HPC and AI: Unlike gaming-oriented Radeon GPUs, Instinct cards are optimized for tasks like scientific simulations, deep learning, and other computationally intensive applications. CDNA Architecture: Instinct GPUs are built on AMD's CDNA architecture, which is specifically designed for HPC and AI workloads. This architecture features Matrix Core Technologies for efficient processing of AI tasks. Large Memory Capacity: Instinct cards come with high-bandwidth memory (HBM) in large capacities, allowing them to handle complex datasets used in AI and HPC. Support for Specialized Data Formats: These GPUs are designed to work with data formats commonly used in HPC and AI applications. Overall, AMD Instinct GPUs are powerful tools for #data #centers tackling demanding workloads in AI and #high-#performance #computing.

"Is a #GPU, #ASIC or #chiplet-based SoC better for #AI as we switch from training to inference?" by Alphawave Semi https://lnkd.in/eMGV9Uyu The demand for AI applications continues to rise and this connectivity issue comes at the same time as the costs of general-purpose GPUs for specific workloads and their high power consumption are motivating a move away from GPU-centric compute architectures, towards custom silicon and chiplet-based designs. #semiconductor

💡 Silicon Power’s heatsinks for DDR5/DDR4 modules are engineered to meet the growing demands of high-reliability applications, including #AI edge servers, to ensure optimal performance and stability with precision cooling.   These aluminum alloy heatsinks provide even and efficient heat distribution across the entire surface of the heatsink, to achieve peak air cooling and keep DRAM modules running at their best, even under heavy workloads 💻 .   Whether you’re optimizing for AI, data centers, or high-performance computing, our heatsink solutions ensure your DRAM modules stay cool, reliable, and ready to deliver. They are available with or without our industrial-grade, high-performance DDR5/DDR4 modules, depending on your needs. #SPIndustrial #heatsink

From #H100, #GH200 to #GB200, How NVIDIA Builds AI Supercomputers with SuperPod? As Al models grow larger, training on a single GPU is no longer feasible. The challenge now is interconnecting hundreds oreven thousands of GPUs to form a supercomputer that acts as a sinale, unified system. This has become the industry's mainfocus. NViDlA's DGX SuperPOD, the next-generation Al architecture for data centers, is designed to meet this challenge. lt delivers the computational power needed for Al model training, inference, high-performance computing (HPC), and hybrid workloads - boosting predictive accuracy and reducing deployment time. In this article, we'll explore NViDlA's three generations of GPU interconnect architectures: H100, GH200 and GB200. https://lnkd.in/gYczZBXt

Here's a discussion of the cost of NVIDIA Blackwell platform GPUs and the energy savings, as described by CEO Jensen Huang and others.