Adi Fuchs

Curious about the performance of Deep Neural Networks (DNN) on different hardware? Join Mirabilis Design at #DAC2024 on June 26th at 10 AM [Booth 2342] . Our session on "Comparing the Performance of DNN on CPU, GPU, and TPU" leverages VisualSim to provide detailed performance metrics. #VisualSim enables accurate simulation of neural network workloads, allowing designers to choose the optimal hardware configuration for their AI applications. Visit: https://lnkd.in/gxEp2ekD for more details or catch us at Booth 2342, DAC 2024! #DNN #AI #PerformanceAnalysis #VisualSim

3

Are Google Tensor Processing Units proprietary? Technically, yes. But with Instachip, you can pip install parts of an open source TPU into your own hardware project! But first...what is Instachip? It's a tool that lets you download Verilog modules from your command line. Just run "𝚟𝚙𝚖 𝚒𝚗𝚌𝚕𝚞𝚍𝚎 <𝙲𝙾𝙼𝙿𝙾𝙽𝙴𝙽𝚃>", and the component will be installed with documentation, pin diagrams, waveforms, etc. Our robust algorithms parse your component, recursively resolving subdependencies and conflicts. Engineers have used Instachip to set up designs as complex as Tensor Processing Units in a single command! Hobbyists, students, and NVIDIA engineers all use Instachip, so it's quite reliable. Most engineers choose to use it offline to securely protect circuit data. We've open sourced the tool! Check it out: https://lnkd.in/e_RTRDDD Search for any module: https://verilogmodules.org But before downloading, check out how you can use Instachip to pip install some TPU designs onto your computer! https://lnkd.in/ehKmj_YR

40

9 תגובות

Introducing Open-Binius! 🚀 "Hardware IPs for accelerating ZK proofs over binary fields" Open-source (MIT) FPGA code. A community effort with academia. Check out our initial results in the table below. LFG! 🔗Github Repo: in first comment 👇 #opensource #zkp

19

1 תגובה

Dive into the Future of Computing with the EDGE-ME Project! 😁 Are you passionate about pushing the boundaries of technology? The EDGE-ME project at DIISM, University of Siena, offers a thrilling opportunity to explore cutting-edge programming methodologies and tools designed for complex applications. Here's your chance to work with innovative hardware platforms that use both shared and distributed memory architectures! What We Do: • Innovative Programming Models: We combine the FastFlow parallel programming model with the DataFlow-Threads (DF-Threads) execution paradigm to achieve high-performance, energy-efficient computing. This is particularly exciting for applications on heterogeneous platforms, including FPGAs (Field-Programmable Gate Arrays). • Open-Source Tools: Our team is dedicated to developing open-source tools that bridge parallel programming paradigms with DF-Threads execution. This makes application development on diverse architectures more efficient and accessible. • Real-World Applications: Collaborating with our industrial partner, SmartMe.io, home of the famous Arancino boards, we evaluate our applications across various domains. This real-world testing ensures our solutions are practical and impactful. • High-Level and Low-Level Integration: We study how to seamlessly integrate high-level programming models with low-level execution mechanisms. This holistic approach enhances the efficiency and scalability of software systems. • Edge Computing Enhancements: We extend the FastFlow framework by incorporating Edge infrastructure management tools like Stack4Things. This integration promises to revolutionize the High-Performance Computing (HPC) ecosystem. EDGE-ME paves the way for building scalable, adaptable, and dependable parallel applications on the edge. Think of it as a revolution in parallel computing, making it easier for everyone to join the game and reap the benefits. This is more than just a project; it's a step towards a future fueled by efficient, powerful computing at the very edge of the network. Stay tuned for exciting updates! Ready to shape the future? Become a part of the EDGE-ME project and embark on a journey of discovery, learning, and impact. Contact prof. Roberto Giorgi [email protected] - see the available position below.

24

1 תגובה

Intel submits Gaudi 2 results on MLCommons’ newest benchmark by fine-tuning Llama 2 70B using low-rank adapters and training MLPerf GPT-3 model with 1,000+ Gaudi 2s on Intel Tiber Developer Cloud. #IAmIntel https://bit.ly/3KRop7O

4

ϕPPL has the same execution model as C. So programs that use only scalar processing written in C should run about the same speed as equivalent programs written in ϕPPL. But ϕPPL has parallel features that C lacks. When the processor has supporting parallel hardware then programs written in ϕPPL can run appreciably faster. For example, we can add two vectors together in ϕPPL this way: A ⇐ B + C; Whereas in C you must write: for (i=0; i< Count;++i) A[i] = B[i] + C[i]; Today I read a post comparing the speed of execution of programs written in Rust as compared to their equivalents written in C. C programs ran about 11× as fast as their Rust equivalent. C++ typically runs about 1/3 slower than its C equivalent. It is partly due to its more complicated frame stack. Python is much slower and typically runs at around 1% the speed of C. #computers #programming #rust #c #c++ #python

2

Here is a short blog post on why SambaNova's RDU is the Best Inference Solution! https://lnkd.in/dd3hiSN8 (The numbers are taken from https://lnkd.in/duqgAuEj)

8

🌟 2024: The Year of Generative AI Deployment! 🌟   If 2022 was the year the world discovered the potential of generative AI (GenAI) and 2023 marked breakthroughs in large-scale training, then 2024 is set to revolutionize how organizations deploy these powerful systems. Companies are now actively testing GenAI in production, unlocking unprecedented benefits. However, this shift isn't without its challenges—managing complexity and ensuring rapid response times are crucial for a seamless, human-like experience.   🚀 Introducing NVIDIA NIM™   NVIDIA has launched NIM™ inference microservices, designed to streamline GenAI deployment and significantly reduce time to market. Learn more about NIMs here https://lnkd.in/dyFkBfbi.   But what if the specific model you need isn't available as a NIM yet? You can still be extremely performant, and we’ll show you how.   📚 Unlocking Potential with Optimized Inferencing   In our latest blog , we explore how to leverage NVIDIA's cutting-edge inferencing technologies—Triton Inference Server and TensorRT-LLM—to enhance large language model (LLM) inferencing in production. We showcase the DictaLM-2.0 model with a custom tokenizer (which has recently led the Hugging Face Open Leaderboard for Hebrew LLMs) and demonstrate how to create optimized engines for both FP16 and quantized INT4 versions, all served through Triton Server. This setup allows for efficient interaction with the LLM, delivering streaming outputs with minimal latency.   🔗 Dive Deeper!   Check out our blog and explore the accompanying Jupyter notebook that details each step of the process. We’d love for you to try it out and share your feedback! Blog - https://lnkd.in/d68QYepb Unofficial Jupyter notebook - https://lnkd.in/digcgqQh   Big shoutout to the incredible team behind this initiative and the guys over at Dicta for the amazing collaboration: Shaltiel Shmidman Amit Bleiweiss Zvi Lapp   #GenerativeAI #MachineLearning #NVIDIA #TritonInferenceServer #TensorRT #LLM

18

1 תגובה

🚨 Big news! NVIDIA is set to roll out about 1 billion RISC-V cores in their chips next year! That’s right—each NVIDIA chip will pack between 10 to 40 RISC-V cores. 🎉 This marks a major shift from proprietary architectures to the open RISC-V architecture, especially for specialized control and data processing in their GPUs. This trend is HUGE for the industry! As one of the top chip makers, NVIDIA's widespread adoption of RISC-V could seriously boost the development of this open architecture across the board. 🌟 #NVIDIA #RISC_V #TechTrends

1

How do LLMs handle the complexity of dynamic sequence lengths? 🌀 In our latest post, we dive into how vLLM and TensorRT-LLM perform with real-world datasets where sequence lengths constantly change. 💡 Learn how fluctuating sequence lengths impact key metrics like throughput and TPOT, revealing each scheduler’s adaptability. 📊 #AI #LLM #Optimization #TensorRT #TRTLLM #vLLM #Deployment

14

12 techniques to reduce your LLM API bill and launch blazingly fast products Cost-efficiency and minimal latency are critical for deploying production-ready LLM applications.   Sahar Mor's article sheds light on 12 innovative techniques to slash your LLM API bills, with gems like Semantic Caching and Summarizing Conversations leading the charge. This isn't just about saving tokens; it's a strategic playbook for scaling smartly in an AI-dominated landscape.  #MachineLearning #AI #NLP #CostOptimization #Innovation #TechTrends https://lnkd.in/ec3qst-M

19

1 תגובה

Take a look at newest improvements of our #OpenVINO: 1. Better LLM service using OpenAI-compatible APIs 2. Enhanced GenAI performance on both CPU and GPU devices 3. Support for INT4 GPTQ 4. Our new C++ Generate API for more efficient text generation 5. Ongoing batching for LLMs

13

Check out the latest issue of HiPEAC Info #HiPEACinfo73 for an interview with Onur Mutlu “An idea whose time has come: Onur Mutlu on processing in memory, holistic architecture design and fundamentally better computing systems": https://lnkd.in/gpUpEUV He discusses what's changed in the role of memory in computing systems since his 2018 HiPEAC interview, how disruptive hardware concepts get taken up, promising directions in computer architecture, and more! You can read the full interview here: https://lnkd.in/e-_aew5V HiPEAC 2018 interview with Onur Mutlu, "It's the memory, stupid": A conversation with Onur Mutlu": https://lnkd.in/ejVxpUjy HiPEAC ETH Zurich, Department of Information Technology and Electrical Engineering #memory #systems #computerarchitecture #computing #machinelearning #MemoryCentric #ProcessingInMemory #ProcessingUsingMemory #hardware

70

2 תגובות

I'm happy to annouce that our work "Toleo: Scaling Freshness to Tera-scale Memory using CXL and PIM" accepted to ASPLOS'24 (delayed to ASPLOS'25 for presentation) is now available on arxiv. 🌟Scale trusted memory size from hundreds of MB to tens of TB by expanding the span of trusted from a single trusted processor to an entire platform including intelligent memories. 🌟Design a new scheme of freshness protection that reduces the space requirement by 50x. 🌟Reduce deployment cost by spacing sharing one intelligent memory device among multiple CPUs. Confidential computing is key to assuring data privacy in the cloud. It protects the privacy and integrity of code and data while they're processed on untrusted third-party systems. However their security guanrantees are weekened to opt for scalability, ease of deployment and lower cost etc. In particular, Intel abondoned protection against memory replay attack in their attempt to bring SGX to Xeon Scalable processors. Our work Toleo brings back the freshness protection to trusted memory with the scalibility to protect TBs of data. We leverage trusted smart memory connected through a secure CXL IDE network as the root of trust, and eliminates the need for an unscalable Merkle tree to protect memory freshness. Futhermore, our space optimizations make it feasible for one 168 GB Toleo smart memory device to provide freshness to a 28 TB CXL-expanded main memory pool in a rack server for a negligible performance overhead.

64

🚀🌟founded by🔥 Eyal Cohen , a hardware and algorithms engineer (formerly of Mellanox, Saifun, and Oren Semiconductor), who also holds a Ph.D.🚀🌟 🌟COGNIFIBER raises $5 million to develop processors that perform AI tasks 1,000 times faster than Nvidia🌟🌟 🔊CogniFiber has successfully raised $5 million in funding to develop groundbreaking processors designed to perform AI tasks 1,000 times faster than Nvidia. 🔊This ambitious venture aims to revolutionize the AI industry by significantly enhancing computational speed and efficiency. 🔊CogniFiber’s innovative technology promises to unlock new possibilities for AI applications, from advanced machine learning models to real-time data processing, transforming how businesses and industries leverage AI. 🔊This funding milestone marks a pivotal moment for CogniFiber, providing the resources needed to accelerate their research and development efforts. 🔊The company's vision of pushing the boundaries of AI performance underscores their commitment to driving technological advancement and setting new industry standards. 🔊CogniFiber's achievement is a testament to the potential of disruptive innovation in the tech sector. 🔊Their pioneering work is poised to inspire other startups and established companies alike to pursue bold ideas and breakthrough technologies. 🔊As CogniFiber moves forward, the AI landscape stands on the brink of a new era of unprecedented speed and capability. ✅️Share Your Startup Story 🚀📲 https://lnkd.in/ggir3Qap ✅️visite our website🌐 https://lnkd.in/dad4nH #CogniFiber #AIInnovation #TechFunding #DisruptiveTechnology #ArtificialIntelligence #StartupSuccess #ProcessorDevelopment #HighPerformanceComputing #scoopearth

3

There is a performance-flexibility tradeoff in GPU networking. RDMA solutions (e.g., RoCE/IB) support terabit throughput but cannot be easily tuned to the communication patterns in your apps and datacenters. Software solutions, the Linux TCP stack or your favorite user-level solution, can be easily tuned and quickly adopt new ideas from research (e.g., DCTCP). But their performance is lacking, especially with terabit networking for accelerators. We just presented ZeroNIC at OSDI'24, a hardware/software co-design that gets you the best of both worlds. The zero-copy throughput to your GPUs with any protocol and any implementation you want, including the Linux TCP stack. ZeroNIC can support innovation in network protocols and topologies for large-scale accelerated computing. Athinagoras Skiadopoulos Mark Zhao Zhiqiang Xie Shrijeet Mukherjee Rachit Agarwal Carlo Contavalli Saksham Agrawal Qizhe Cai Mike Goldflam

311

8 תגובות

GDDR7: The Ideal Memory Solution in AI Inference. The generative AI market is experiencing rapid growth, driven by the increasing parameter size of Large Language Models (LLMs). This growth is pushing the boundaries of performance requirements for training hardware within data centers... Read more in the Cadence Tech Talk article here>> https://ow.ly/SIay50TmxsW

1

You've seen those long context Google Gemini demos? Here's us cheekily unlocking Long Context Workflows on AMD MI300X with our Flywheel Inference Engine on TensorWave. No doubt that long context is making some buzz around LLM inference. It is kinda wild that you can just dump your entire doc set into the context and then... just chat with it. No RAG, no finetuning or anything. Just a chat with a looooong prompt.

32

1 תגובה