Data Phoenix’s Post

[Data Phoenix News] Deploying LLMs to any cloud or on-pre, with NIM and dstack With dstack's latest release, it's now possible to use NVIDIA NIM with dstack to deploy LLMs to any cloud or on-prem—no Kubernetes required. Read more 👉 https://buff.ly/3VnNbCm ------------- Have exciting news? Share your story with the AI & Data world today -> https://buff.ly/4hTlw5K

Extreme Performance Series 2024: Enabling and Optimizing GenAI Workloads with LLMs How do you run Generative AI workloads based on Large Language Models on VMware Cloud Foundation with optimized performance? Any tips? https://dy.si/uRNWAC

Nutanix Unified Storage is now a leader in AI storage performance, establishing NUS as a gold standard for AI and machine learning applications: * A single NUS cluster can serve 1056 accelerators, the highest of all vendors listed in the benchmark * Performance scales linearly with the 32-node cluster supporting 4X accelerators as the 8-node cluster * Similar performance per node is observed irrespective of the location, on-premises or in the cloud

Embrace the #GenAI era with confidence at #AWSreInvent 🚀 We're excited to host a pivotal session for anyone scaling #AI in the cloud. Our Zak Harabedian takes the helm to guide you through the operational labyrinth of scaling GPU workloads for #generativeAI. Kubernetes might be the buzzword, but how does it fare when the stakes are high, and the data is vast? Jot down the session details to gain insights on fine-tuning resource requests while saying goodbye to cost overruns and suboptimal resource use: https://ntap.com/3OdWeSc

Fastly doing their bit to improve responsiveness of AI queries by caching most used queries. This plays to a much wider issue with LLMs, which how they will be served from a cloud platform as a Service. There is a cold start issue of loading LLMs do to the size which will need to be fixed before a true on-demand Inference-as-a-Service can be deployed. Something Kontain.ai have solved. https://buff.ly/3zahaVY

At #VMwareExplore, all conversations are focused on VMware Cloud Foundation (VCF) and the Advanced Services that VMware is building to provide a full cloud experience for the consumers of a private cloud. I will do a full summary of the strategy and announcements from Explore soon; in the meantime, have a look at one of these Advanced Services - Private AI. Please keep in mind this was written before Explore.

In #healthcare there are continuous cybersecurity threats everywhere. We also want the benefits of #AI to improve patient care. These LLM models leverage public data and are expensive to test. Now consider having a ‘private’ AI model leveraging your own data and avoiding the Internet. Could this be an answer for healthcare? Link to the article on Private AI below. #CDW #strategy

Rob Sims, CDW’s Chief Technologist for Hybrid Platforms, explores VMWare’s current offerings in the Software-Defined Datacentre space. In this article Rob explores Private AI; one of the innovations VMWare is building onto its cloud foundation. In this article Rob covers: - Why do we need VMware Private AI Foundation with NVIDIA? - What is it? - Key features, and how it can deliver scalable, performant, and compliant AI-driven outcomes at the pace required by the business. 👉 https://hubs.ly/Q02W_dx80

📣 Don't miss out on our exclusive session at #AWSReInvent, where we'll dive deep into optimizing GenAI workloads! As the GenAI revolution continues to unfold, organizations are confronted with the task of scaling GPU workloads in the cloud. While Kubernetes presents an enticing solution for AI inference and analysis of new data, it also poses its own set of challenges. Join us for an enlightening session where we'll show you how to harness the power of Kubernetes in conjunction with AWS and NetApp to overcome these hurdles and optimize your GPU infrastructure. Discover the secrets to efficient resource utilization, cost savings, and unparalleled performance! Make sure to add this session to your #AWSReInvent schedule right away and position yourself at the forefront of the GenAI revolution: https://ntap.com/48GYUkU

Eyal covering the work our teams on running batch workloads on kubernetes and how to get scarce resources on GKE