Muli Ben-Yehuda

During this hands-on workshop, our experts will guide you through best practices for optimizing your Kubernetes clusters, leveraging the power of PerfectScale. You'll learn how to: - Identify and safely remove wasted resources - Pinpoint and remediate issues causing latency and downtime - Evaluate the efficiency of your autoscalers - Select the most cost-effective node types based on real-time data https://hubs.la/Q02WgM2c0 #k8s #devops #kubernetes #aws #sre

4

You should take full advantage of modern processors as an engineer. This article helps you do that with .Net https://lnkd.in/dpZ8i6HN

4

Give a talk about PerfXLM, a high performance large language model inference engine, at RISC-V Summit Europe 2024

1

Intel submits Gaudi 2 results on MLCommons’ newest benchmark by fine-tuning Llama 2 70B using low-rank adapters and training MLPerf GPT-3 model with 1,000+ Gaudi 2s on Intel Tiber Developer Cloud. #IAmIntel https://bit.ly/3KRop7O

4

Leading a large R&D organization across multiple groups and regions requires more than just technical skills. It's about building strong relationships, setting clear expectations, and fostering a culture of open communication. I enjoyed sharing my experiences with the amazing Dr. Shani Horowitz-Rozen on her podcast. We discussed practical strategies for navigating complex challenges and creating high-performing teams. If you're a leader looking to level up your communication game, this episode is for you! #leadershipdevelopment #teammanagement #R&D #podcast #leadership

41

6 תגובות

Nvidia has agreed to acquire Israeli startup Run:ai in a deal estimated to be worth around $700 million, though the exact terms were not disclosed. Run:ai, which has been a close collaborator with Nvidia since 2020, specializes in software for managing AI computing resources. ➜ Check the news at: https://lnkd.in/dHsxrhSi ➜ Vendorvise's official Instagram: https://lnkd.in/ebK2KQ2C ➜ Vendorvise's Linktree: https://lnkd.in/dQdbMdth 🤖 If you enjoy reading my posts about AI and tech news, hit that like button and share with your friends to support my content! #ai #artificialintelligence #ainews #aipowered #news #blog #nvidia #runai #700million #generativeai #llms #tokens #aitech #tech #technology #hardware #content #socialmedia #future #new #launch #business #techtrends #innovation #success #markets #people

1

"NVIDIA to acquire AI startup Deci AI for $300 million - The chip giant has spent $1 billion purchasing two Israeli startups in one week after announcing a deal to buy Run:ai for $700 million last week - Nvidia has agreed to acquire Israeli artificial intelligence (AI) company Deci. According to estimates obtained by Calcalist, the value of the deal is $300 million. In Deci's last fundraising round in July 2022, it was valued at an estimated $200 million. The acquisition is subject to regulatory approvals and is expected to close during the second quarter. Nvidia confirmed the deal to Calcalist, stating that 'NVIDIA’s expertise in AI will be complemented by Deci’s capabilities in NAS, which automates the design of deep learning models to improve performance. Deci’s engineering team has knowledge in NAS, Foundational Models, Inference and in developing complex algorithms.' With this acquisition, Nvidia concludes the purchase of two Israeli AI companies in one week, totaling a value of $1 billion. This comes after it announced the purchase of Run:ai last Wednesday in a deal worth $700 million. Deci was founded in 2019 by Yonatan G., Ran El-Yaniv, and Jonathan Elial. Deci’s end-to-end deep learning development platform helps AI developers build, optimize, and deploy AI models to any environment such as cloud, edge, and mobile. The purchase of Run:ai was Nvidia’s biggest acquisition in Israel since buying Mellanox for $6.9 billion in March 2019. Since the acquisition, Nvidia’s local R&D center in Israel, which was first established in 2016, grew by about 100%, reaching 4,000 employees in the country. Nvidia also announced the acquisition of the Tel-Aviv based startup Excelero, provider of high-performance software-defined storage, in February 2022. Deci raised $25 million in a Series B funding round in July 2022 led by global software investor Insight Partners, with participation from existing investors Square Peg, Emerge Ventures, Jibe Ventures, and Fort Ross Ventures, as well as new investor ICON. The investment came just seven months after Deci secured $21 million in Series A funding, also led by Insight Partners, bringing Deci’s total funding to $55.1 million. Deci’s deep learning platform helps data scientists eliminate the AI efficiency gap by adopting a more productive development paradigm. With the platform, AI developers can leverage hardware-aware Neural Architecture Search (NAS) to quickly build highly optimized deep learning models that are designed to meet specific production goals. The Israeli startup announced the launch of its Generative AI platform in March designed to meet the efficiency and privacy needs of enterprises. Deci’s Gen AI platform features a series of proprietary, fine-tunable large language models (LLMs), an inference engine, and an AI inference cluster management solution." Philippe Schwartz Assaf Jacobi Liad Rubin Danny Hadar Eugenia (Bejar) Lustgarten David Mann Stark Daniel Aronovitz Evan Hahn George Mathew

318

12 תגובות

Here is a short blog post on why SambaNova's RDU is the Best Inference Solution! https://lnkd.in/dd3hiSN8 (The numbers are taken from https://lnkd.in/duqgAuEj)

8

Has anyone tried AMD's MI300X GPUs for inference? I read a blog post about them on Hacker News and it keeps coming back to me. In short, TensorWave benchmarked AMD’s MI300X vs NVIDIA’s H100s for LLM inference. They then shared that MI300X outperformed H100 by 1.22x - 2.94x. The audience on Hacker News noted that TensorWave makes money by selling AMD GPUs and then ripped apart some of the performance claims. For me, the more interesting part wasn’t on the blog post. H100 retails at $30K+ and MI300X sells at $10-15K apiece. So, even if those two cards had the same performance, price/performance would be 2-3x higher with the AMD cards. Then again, maybe CUDA vs ROCm also makes a difference. The problem is that AMD’s cards don’t seem to be available anywhere on-demand. Azure announced AMD MI300X VM availability during its Build conference - they are presumably now available in Canada Central and Sweden Central (if you want to run inference close to the Arctic circle, Azure has you covered). However, online comments indicate that no one has been able to get a hold of them yet. So, if you’ve ever tested with MI300X, could you chime in? Or, if you have some theoretical knowledge on this topic, please also drop a note in the comments (Aditya Rajagopal PhD, Diederik Vink, PhD)? I’m looking to learn. In case anyone's interested, here's TensorWave's blog post on this: https://lnkd.in/ePuGGCsw Here's the discussion on Hacker News: https://lnkd.in/exVgAiSC

30

15 תגובות

12 techniques to reduce your LLM API bill and launch blazingly fast products Cost-efficiency and minimal latency are critical for deploying production-ready LLM applications.   Sahar Mor's article sheds light on 12 innovative techniques to slash your LLM API bills, with gems like Semantic Caching and Summarizing Conversations leading the charge. This isn't just about saving tokens; it's a strategic playbook for scaling smartly in an AI-dominated landscape.  #MachineLearning #AI #NLP #CostOptimization #Innovation #TechTrends https://lnkd.in/ec3qst-M

19

1 תגובה

How to get higher performance than GPUs with lower cost for your business? #Sambanova SN40L RDUs and COE architecture is the answer. Check it out!

7

ExectOS is a preemptive, reentrant multitasking operating system that implements the XT architecture which derives from NT™ architecture. It is modular, and consists of two main layers: microkernel and user modes. Its’ kernel mode has full access to the hardware and system resources and runs code in a protected memory area. It consists of executive services, which is itself made up on many modules that do specific tasks, a kernel and drivers. Unlike the NT™, system does not feature a separate Hardware Abstraction Layer (HAL) between the physical hardware and the rest of the OS. Instead, XT architecture integrates a hardware specific code with the kernel. The user mode is made up of subsystems and it has been designed to run applications written for many different types of operating systems. This allows us to implement any environment subsystem to support applications that are strictly written to the corresponding standard (eg. DOS, or POSIX). Thanks to that ExectOS will allow to run existing software, including Win32 applications.

2

Very interesting webinar just started - How LLMs will impact different industries

1

This is simply incredible. Sambanova just launched a cloud inference platform allowing developers access to this blazing fast AI platform that has the best inference speeds on Llama models in the market. What is unique here is that all models are run with 16 bit precision as well as being extremely cost efficient. Start developing #FastAI ➡ http://cloud.sambanova.ai

48

Throughout the academic year, I have been working on a project to build a hardened multi-level OS based on the Linux kernel. We've been in discussions with many members of the community, including hardware manufacturers, AI labs, cloud vendors, and Linux vendors, and there's considerable excitement surrounding our approach. We have just received hardware from Ampere to support our operating system research with Dr. Eggert (maintainer of GNU Grep)! I'm eager to continue working on this project through fall and beyond. We plan to experiment with container-native, next-generation immutable container host operating systems like Microsoft's Flatcar Linux, Fedora CoreOS, and AWS Bottlerocket. The project will also involve developing AI tools to automate security policy generation. Access control policy management is an increasingly challenging problem from both security and verification perspectives. SELinux, a Linux Security Module, implements a mandatory access control mechanism. As SELinux policies, developed and maintained by security administrators, often become quite complex, it's crucial to analyze them carefully to ensure their correctness. I believe these distributions will attest to that. One of the only Linux distributions with a comprehensive set of policies is Android. This research is being led by Dr. Amir EAMAN, Joann S. from CSUF, and Pranav Kapoor and Shinyoung (Mark) Lee from UCLA. Ampere is one of the leading ARM chip manufacturers. Their Altra processors are used by Linus Torvalds for Arm64 Linux builds! Many thanks to Ampere for supporting our efforts! We're excited to share our future developments! cc: Folks who may be interested in this project (some working on different approaches than our own) Dan Walsh , Kinvolk (acquired by Microsoft) , Google Cloud Security , Red Hat , Amazon Web Services (AWS) , VMware , Canonical , SUSE , Dongsu Park , Thilo Fromm , Andrew Randall , Mathieu Tortuyaux , Sayan Chowdhury Huge shoutout to the Universal Blue project. Bringing the container-native model to the desktop! Using OCI as a delivery mechanism is awesome :) Jorge Castro

52

7 תגובות

Good listen: Mark Suster, the guest host, is joined by Dan Senor to discuss the remarkable growth of NVIDIA in #Israel, which now boasts over 4,000 employees and a global CTO who is Israeli. The conversation shifts to Israel’s startup ecosystem, highlighting its impressive statistics: 500 venture capital firms, 120 accelerators, 35 incubators, 400 multinationals with major R&D operations, and 7,000 startups. These figures underscore Israel’s position as having the highest density of startups globally, attracting significant global venture capital, leading in R&D spending relative to GDP, and ranking fourth in NASDAQ-listed public companies. Regardless - I'll recommend following This Week in Startups podcasts which brings interesting conversations with leaders from the startup community... https://lnkd.in/eGv3fr-H #iwork4dell

7

1 תגובה

🌟 2024: The Year of Generative AI Deployment! 🌟   If 2022 was the year the world discovered the potential of generative AI (GenAI) and 2023 marked breakthroughs in large-scale training, then 2024 is set to revolutionize how organizations deploy these powerful systems. Companies are now actively testing GenAI in production, unlocking unprecedented benefits. However, this shift isn't without its challenges—managing complexity and ensuring rapid response times are crucial for a seamless, human-like experience.   🚀 Introducing NVIDIA NIM™   NVIDIA has launched NIM™ inference microservices, designed to streamline GenAI deployment and significantly reduce time to market. Learn more about NIMs here https://lnkd.in/dyFkBfbi.   But what if the specific model you need isn't available as a NIM yet? You can still be extremely performant, and we’ll show you how.   📚 Unlocking Potential with Optimized Inferencing   In our latest blog , we explore how to leverage NVIDIA's cutting-edge inferencing technologies—Triton Inference Server and TensorRT-LLM—to enhance large language model (LLM) inferencing in production. We showcase the DictaLM-2.0 model with a custom tokenizer (which has recently led the Hugging Face Open Leaderboard for Hebrew LLMs) and demonstrate how to create optimized engines for both FP16 and quantized INT4 versions, all served through Triton Server. This setup allows for efficient interaction with the LLM, delivering streaming outputs with minimal latency.   🔗 Dive Deeper!   Check out our blog and explore the accompanying Jupyter notebook that details each step of the process. We’d love for you to try it out and share your feedback! Blog - https://lnkd.in/d68QYepb Unofficial Jupyter notebook - https://lnkd.in/digcgqQh   Big shoutout to the incredible team behind this initiative and the guys over at Dicta for the amazing collaboration: Shaltiel Shmidman Amit Bleiweiss Zvi Lapp   #GenerativeAI #MachineLearning #NVIDIA #TritonInferenceServer #TensorRT #LLM

18

1 תגובה