🧐 Shafik Yaghmour

Check out this technical conversation between Dr. Waku and Groq Compiler Engineer Gautam Rayaprolu. They cover how inference works on GPUs v LPUs, issues of reliability in large clusters, the Groq approach to building data centers, & more. https://hubs.la/Q02RqZR10

39

Still spending months of development time porting your code to OpenMP, NVIDIA CUDA, AMD HIP or FPGA HDL? If so, then it’s time to accelerate your development flow with CacheQ Systems, Inc. C compiler QCC: + Average 80-90% savings on development porting time + 100% debugging time savings since bugs are introduced during manual code porting + Eliminate the need for GPU, multicore CPU, or FPGA domain expertise + Yield performance on par with an expert's hand-coding + Expedite your time to testing and time to market #cacheq #cuda #gpu #cpu #fpga #hdl #multithreading #multicore #heterogeneous #hardwarecompilers #algorithmdeveloper #ai #ml #openMP

1

Most major tech companies are at Computex in Taiwan this week, and we're seeing a flood of announcements. The industry is buzzing following Jensen Huang's announcement of the next-generation Rubin GPU platform and associated Vera Arm CPUs. These will replace Grace, Hopper, and Grace Hopper as NVIDIA's scalable AI platform in 2026 and beyond. It's excellent to see NVIDIA honor another woman in science with their platform names, in this case American astronomer Vera Rubin who passed away in 2016. #Rundown #Computex #AI

1

Join us on August 29 at 9am PT | 12pm ET for an exclusive webinar with NVIDIA. We’ll dive into fine-tuning LLMs and deploying them at scale—securely and cost-effectively—across any infrastructure. You’ll Gain insight on: 💡 Efficiency boosts with NVIDIA NIM 🔧 Optimized resource use and performance 🧠 Advanced model training with NVIDIA NeMo 📈 Scalable deployment strategies 👉 Don’t miss out—reserve your spot now! https://lnkd.in/e_S62arN #AI #MachineLearning #Webinar #NVIDIA

5

New 📝 Step-by-step instructions on how to ✅ Optimize LLMs with NVIDIA TensorRT-LLM, ✅ Deploy the optimized models with Triton Inference Server, ✅ Autoscale LLMs deployment in a Kubernetes environment. Technical Deep Dive: https://nvda.ws/3NyjtWX

216

1 Comment

More bragging... The 28.8Tbps standalone deep buffer Asic using #Express5 silicon boasts around 3W power per 100Gbps. This is an extremely power-efficient chip in the 7nm process node, beating our competitors' chips in the 5nm process, hands down! How do we do that? 🤔 ✔ The standalone Asic has 4 PFEs across the two dies. However, our elegant architecture does direct memory-to-memory transfers and does not require expensive Xbars for PFE-PFE communication. ✔ The architecture is optimized for less data movement with VOQ, hybrid buffering, and shared packet processing data structures. ✔ When the 800G WAN datapaths are stamped 18 times, any inefficiencies in the design are multiplied by x18. By super-optimizing the data buses to carry multiple packets simultaneously, we eliminate over-design and reduce power consumption ✔ The most power-hungry module, integrated 800G MACsec, uses patented techniques to optimize the AES engines and the number of operations while handling line rate encryption/decryption. ✔ Dynamic clock gating with >95% efficiencies in most blocks aggressively reduces active power by keeping unused flops clock gated. ✔ Using the latest EDA tools in both front-end and PD, we achieved an additional 7-8% savings. ✔ And many more... Read my article (https://lnkd.in/gVckWwEW) for a deeper understanding of the power-saving techniques used in Express5 and PTX routers. #Express5 #junipernetworks #ethernetswitch Juniper Networks

201

3 Comments

Working on deploying ONNX models for performance-critical applications? Try our NVIDIA Nsight Deep Learning Designer ⚡ A user-friendly GUI and tight integration with NVIDIA TensorRT that offers: ✅ Intuitive visualization of ONNX model graphs ✅ Quick tweaking of model architecture and parameters ✅ Detailed performance profiling with either ORT or TensorRT ✅ Easy building of TensorRT engines Learn more and download ➡️

2

Dynamic load balancing. Built-in. The latest #IntelXeon processors include the Intel Dynamic Load Balancer (Intel DLB) accelerator. Learn how this can help you optimize network performance. https://intel.ly/3Uktcnw

1

We've just completed testing Intel's newest 5th-generation Xeon processor with Qdrant and observed major improvements in how quickly and efficiently it handles searches. 🔍 Here's what we found: ✔ Query speeds on machines with 32 cores are now 1.38 times faster than the previous generation. ✔ Latency for vector search operations has been reduced by approximately 2.79 times compared to the earlier Sapphire Rapids processors. ✔ AI inference performance has improved by up to 42% with the new Emerald Rapids processor. ✔ Vector search operations are now 38% faster than before. For an in-depth analysis of these benchmarks, visit the full research report: https://lnkd.in/dd5dfiMP

27

The NPU is a low power, energy efficient processor engine that elevates the game of AI model deployment on your local machine. Benjamin Consolvo, AI Engineering Manager at #Intel, discusses the new Intel® NPU Acceleration Library and how its enabling large multimodal models (LMMs) in this Hugging Face blog. https://intel.ly/3S42OwR #GenAI

523

10 Comments

Researchers from UCI and Cisco Propose CrystalBall: A Novel AI Method for Automated Attack Graph Generation Using Retriever-Augmented Large Language Models Cybersecurity is a fast-paced area wherein knowledge and mitigation of threats are most necessary. In this respect, the attack graph is one tool that security analysts mainly resort to for charting all possible attacker paths to the exploitation of ... https://lnkd.in/eBzcXKgf #AI #ML #Automation

1

Why has the FPGA developer experience lagged behind software? Some of the key points I took away from the FPGA panel at the NYC STAC Summit with experts from Hudson River Trading, Morgan Stanley, UBS, MA CAPITAL U.S. LLC, Cadence Design Systems, and AMD were: ➡️ Is tooling the biggest challenge? While FPGA tools have certainly advanced, software tooling has outpaced them in driving productivity gains. At the same time, the increasing complexity of FPGA designs demands more development time, which can make hardware engineer productivity seem stagnant. To bridge this gap, FPGA teams have adopted software toolchains and practices, leading to some gains. However, the most significant productivity boost will come from embracing higher levels of abstraction. This shift, though necessary, challenges the industry’s focus on optimizing every watt and nanosecond. As designs get bigger, FPGA engineers may need to relinquish some control to unlock greater productivity and time to market. ➡️ Open source is the best practice. The group shared several practices from their teams that have significantly enhanced the engineer experience, with open source emerging as the top recommendation. The panelists stressed that it’s not about reducing costs. As Michael Gorbovitski put it perfectly, “You can’t be a consumer of technology and expect to stay on the cutting edge. You need to be a participant.” ➡️ Is AI the future? The panelists agreed that machine learning models will play a key role in testing designs, while GenAI will assist in creating regression suites and test cases. AI is making certain tasks inexpensive and will undoubtedly transform development processes. While the panel acknowledged it could take years to fully realize these changes, Vijay from Cadence highlighted that significant advancements might arrive as soon as this fall! If you missed this passionate discussion and share our enthusiasm for the topic, join us at the STAC Summit in Chicago tomorrow as the conversation continues. 🗓️ October 29th 📍 Chicago 👉 Register at https://lnkd.in/evhUX3du Thank you to Benjamin Maron (HRT), Michael Gorbovitski (Morgan Stanley), Rajiv Perera (MA Capital), Lakshmi N Aiyer (UBS), Vijay Akkaraju (Cadence) and Matthew C. (AMD) for this fun discussion!    #FPGA #ML #HFT #lowlatency #trading #STACsummit #AI #hardwareengineering #Verilog #SystemVerilog #VHDL 

64

Today we will write two functions that are similar to ones we did earlier. But this time we will detect overflow for signed addition instead of unsigned addition. Doing the add is the same. But detecting overflow is more involved. We feel the pain of having limited resources with RISC-V. Overflow can only happen if the signs of the two source operands are the same. Then, and only then, an overflow happens if the sign of Src1 changes as a result of the add. Here is the RISC-V assembly source: sb_add_:  lb   t0,0(a1) #Src1           lb   t1,0(a2) #Src2           addi t2,x0,t0 #Src1 Copy           xor  t3,t1,t0 #Diff Signs           addi t4,x0,-1 #All 1's           xor  t3,t3,t4 #Same Signs           add  t0,t0,t1 #Do the Add           xor  t4,t2,t0           and  t2,t0,t3 #Chg in Sign           sb   t0,0(a0) #Store Sum           srl  a0,t2,31 #Get Bit           jalr ra       #Done RISC-V is a load/store ISA so we have to load both sources before we do anything else. We then have to make a copy of Src1 in t2 so that we can see if the sign changed. On the 3rd line we test to see if the signs are different using an XOR. But what we really need is to know whether they are the same instead. That means we neet to NOT the result. But RISC-V does not have a NOT. We have to XOR it with all 1's. Register x0 has all 0's but we don't have a register with all 1's so we have to make one in t3. To do this we just load −1 which is all 1's. Then we XOR that with t2 which is the same as a NOT. Next we actually do the add of the two sources. We can then store the byte to the Dst. Next we have to see if the sign changed using another XOR. Finally, we detect overflow with an AND. The high-order bit now contains our Signed Overflow bit which we isolate by shifting it all the way to the right using the srl instruction. This last step was easy and the 3-operand model lets us put it into the return register a0. Now let's do it with /ϕEng32. Our Flags will make it a trivial problem so let's use them! sb_add_: ld     ⓈQ0,sb:(Ⓧ1)         add?Ⓒ0 ⓈQ0,sb:(Ⓧ2)         st     ⓈB0,(Ⓧ0)         clr    ⓊQ0         bst#V  ⓊB0,#0         rts The 1st line loads Src1 and sign-extends it. This is implied since the Subject is a Quad while the source is a Byte. Since it is a Signed Byte, it will have to be sign-extended to fit the Dst. The same thing happens in the 2nd line except that it is used as Src2 in the add. Also, adds don't involve flags by default so we must specify a Flag Set which is Ⓒ0. On the 3rd line we store the result. Now comes the easy part. We first clear the return value which will contain the Overflow Flag. Next we just store the V flag to bit 0 of byte 0. Finally, we return. Wasn't that easy? The RISC-V code took 12 instructions for a total of 48 bytes. Our ϕEng32 function used 6 instructions for a total of 18 bytes. ϕEng32's code was only 35.75% that of RISC-V; just over 1/3 the size. #computers #ISA #electronics #assembly #programming

1

Learn how to reduce CPU launch overhead and accelerate #AI workloads using new #CUDA Graphs enhancements. Read about all the new improvements and benchmark numbers now. #HPC

66

3 Comments

Check out the below article led by my colleague Beenish Zia which is a great example of optimizations which can be made to open source CODEC implementations as important building block to support a software IT approach to Pro-AV. #iamintel

13

Curious about how your system handles memory under real-world conditions? Our solutions architect, introduces a characteristic-based memory consumer tool – a powerful way to model memory workloads based on time-based patterns. Key features of the tool: ✅ Flexible memory consumption modeling across varied time scales, ✅ Ideal for testing workload scheduling, system stability, and high-memory scenarios, ✅ Valuable for diagnosing memory-related performance issues and stress-testing environments. Ready to explore its potential? Watch the full video: https://hubs.ly/Q02Wq-Qj0 #MemoryManagement #SystemTesting #CodiLime

10