Andrew Feldman

Got a chance to discuss TCG Key Per I/O capabilities and its possible use cases with confidential compute at the Confidential Compute Consoritium's TAC call yesterday. Please see the link below. The topic is covered from around 20 mins into the video and runs till about the 1-hour mark. https://lnkd.in/gvupy8iP

20

In a test on #OpenAI’s Whisper model, Ampere Altra #CloudNative Processors enabled higher performance at a lower cost than #GPUs. Find more reasons to go #GPUFree. https://ow.ly/eTsi50RlEkK

85

Interesting move and appears a win/win for both companies. That said, may take awhile to implement. I've toured many HPC & AI and enterprise computing datacenters, as well as a number of cryptomining facilities, and the infrastructure is starkly different. HPC & AI datacenters are typically air conditioned, raised floor rooms with direct liquid cooling to chip and high speed (e.g. InfiniBand) interconnect. This is very, very different from the shoeboxes on shelves model, often in former warehouse buildings, with a simple Ethernet connection to the outside world. https://lnkd.in/g-GFNCZ4

12

2 Comments

Archival storage demand is increasing, driven by data intensive application such as AI, but archival storage must be very cost effective versus other higher performance storage to be competitive.  This puts constraints on $/TB costs of this storage.  An archival storage media that can combine low costs per TB and higher performance would have definite advantages for long term accessible storage. Current HDD and magnetic tape technologies continue to develop and by 2029 a competing optical storage technology, even with faster storage access, needs to provide less than $2.5/TB media storage costs (magnetic tape projected costs by 2029).  Such an optical storage technology, such as Optera Data, could be a viable alternative, assuming similar drive and library costs to magnetic tape storage.   Read more below: www.opteradata.com

29

2 Comments

YOCO is a novel decoder-decoder architecture for LLMs, enhancing memory efficiency by caching key-value pairs only once. YOCO markedly reduces KV cache memory and prefilling time by orders of magnitude. YOCO makes 1M-length LLMs practical. https://msft.it/6040YnEVM

391

3 Comments

New SiFive Performance P870-D Complements Intelligence Chipset for AI TL;DR - SiFive announced the SiFive Performance P870-D, a high-performance RISC-V datacenter processor designed for demanding AI workloads, offering up to 256 cores, enhanced scalability, and advanced Reliability, Availability, and Serviceability (RAS) features. This processor positions SiFive as a competitive player in the data center market, providing data center architects with a flexible, power-efficient solution for AI-driven applications. SiFive aims to leverage RISC-V's open-source flexibility to challenge established industry giants like Intel and AMD. Read the Research Note in the comments below.

1

1 Comment

Keeping your data close to the GPUs is essential, so we offer VAST Data as part of our #GPUasaService AI Platform. Let us help you #StepontheGaaS and move your #AI strategy forward. #Fast #Secure #Private Dominic Ken Michael Bill Traci

10

1 Comment

Exciting developments! Discover how Exostellar Inc. is partnering with Amazon Web Services (AWS) to revolutionize semiconductor design, helping organizations significantly cut costs and optimize their processes. With semiconductor design workflows relying on compute-intensive tasks like functional verification, place and route (P&R), physical verification, and electrical signoff, managing costs can be a challenge. Exostellar’s solutions on AWS are designed to provide optimal performance and scalability for cloud-based EDA tools while keeping costs in check. Ready to learn more? Check out this insightful blog to see how we’re making a difference in the industry: https://lnkd.in/g9jTdQWq Nayan Lad Frederic Julien Rohan Prakash Zhiming Shen Nick Cox Tony Shakib Neel Desai #Semiconductor #HPC #CloudComputing #AWS #Exostellar #TechInnovation

17

I am excited to speak at IEEE-EDPS later this week, and share my latest thoughts on technologies for next compute paradigm shift in the era of pervasive intelligence. You can still register last minutes at https://lnkd.in/gACWUuib #ComputerChips #PervasiveIntelligence #EmergingTechnologies

115

3 Comments

New Transformer architecture could enable powerful LLMs without GPUs https://buff.ly/4b3XAYT #ai #transformers #llm #cpu

2

Interesting see the progress of RISC-V going “up the stack” to HPC….in a server world of virtualized workloads orchestrated by cloud native techniques like kubernetes, it opens up opportunities for innovative architectures.

5

Hybrid SW and FPGA development is ideal for accelerating fintech applications due to superior performance, flexibility, and energy efficiency. FPGAs, customized for specific tasks, provide faster processing speeds and reduced latency, crucial for high-frequency trading and real-time analytics. Developing for a hybrid environment, you can lower development/operating costs and support sustainability. This combination enhances computational capabilities, making it a compelling choice for fintech firms aiming to excel in a rapidly evolving industry.

9

The GPU hosting industry is experiencing unprecedented growth, as discussed in the recent "From CPU to GPU Infrastructure" panel at infra/STRUCTURE Summit 2024. Moderated by Philbert Shih of Structure Research, the panel featured insights from Jeff Hinkle, CEO of GPU Advanced Resources Inc, and Zhen Lu, CEO of RunPod. They shared experiences navigating this rapidly growing market, balancing growth with sustainability, and overcoming entry barriers. With demand skyrocketing, this sector is on a promising trajectory. Check out the full article on Data Center POST to learn more about the trends shaping the future of GPU hosting: https://lnkd.in/etddDSAj. #GPUInfrastructure #CloudHosting #TechGrowth #infraSTRUCTURE2024 #RunPod #GPUResources

9

We've teamed up with Vultr to bring next-generation #AI and #vectorsearch to the 𝐦𝐚𝐧𝐮𝐟𝐚𝐜𝐭𝐮𝐫𝐢𝐧𝐠 𝐢𝐧𝐝𝐮𝐬𝐭𝐫𝐲! By leveraging Qdrant's advanced vector search on Vultr's high-performance Cloud compute, powered by NVIDIA GPUs, we’re expanding AI into groundbreaking applications like smart factories, automotive, robot design, robotics, supply chain, and quality assurance. Here's how we're making an impact: ▶️ Optimizing production lines and enabling predictive maintenance to minimize downtime and enhance efficiency. ▶️ Speeding up the design process while meeting safety and performance standards more efficiently, leading to faster innovation. ▶️ Enhancing the functionality and responsiveness of robotic systems with improved cognitive capabilities, driving smarter automation. ▶️ Utilizing advanced analytics for better demand forecasting and inventory management, ensuring smoother operations. ▶️ Strengthening processes to meet the highest quality standards, delivering superior products consistently. Start building the future of manufacturing with us. Discover how you can start using AI can revolutionize your operations:

28

1 Comment

Supermicro's New Multi-Node Liquid Cooled Architecture with Maximum Performance Density Purpose-Built for HPC at Scale The New Rack-Scale Ready FlexTwin™ Systems Deliver Unprecedented Compute-Density in a Multi-Node Form Factor, with DLC dual CPUs up to 500W Each per Node, Front Node Access and Optimized Networking

17

1 Comment

Get a deeper look into the results submitted by NVIDIA in #MLPerf Inference v4.1, highlighted by the first ever submission using the NVIDIA Blackwell architecture, increased performance using the NVIDIA H200 Tensor Core GPU, and more.

1

OpenAI and Broadcom in talks about developing new AI chip OpenAI is in talks with Broadcom and other industry leaders to develop a new chip tailored for AI applications, potentially reducing reliance on NVIDIA #GPUs. Sam Altman has been instrumental in spearheading efforts to secure partnerships with #chipmakers, tech giants like Microsoft, and government bodies to enhance #compute capacity essential for #AI #innovation. This move underscores OpenAI's commitment to advancing artificial general intelligence (#AGI), aiming to push the boundaries of AI capabilities beyond current standards. Despite the formidable challenges and substantial financial requirements, including Microsoft's substantial investment, OpenAI remains focused on expanding access to AI benefits through strategic partnerships. For more information: https://lnkd.in/dYHej2Kc #AI #ArtificialIntelligence #OpenAI #TechInnovation #Semiconductors #Broadcom #Microsoft #AGI #AIHardware #TechPartnerships

4

1 Comment

For anyone interested in Confidential Computing on Arm platforms, you must watch the update that Dan Handley from Arm provided at Linaro Connect. He covered the current state, what's coming next in the Confidential Compute Architecture (CCA) specification, and what to expect. Watch the recording at https://lnkd.in/dRiCz9TH.

24

🌱 WEKA is committed to raising awareness about the energy and carbon impact of performance-intensive solutions like #AI, #ML, and #HPC. That's why we're excited to be included in The_TechArena's Data Center Compute Efficiency Report 2024 - a comprehensive look at disruptive innovations across the market and how they will shape the industry and our 🌎. The report outlines WEKA's unique approach and includes commentary from WEKA President Jonathan Martin on the inefficiencies of traditional data management vs. WEKA's software-based zero-copy architecture. Check out the full report: https://weka.ly/3UbZdhc #sustainability #innovation #aistorage

44