Eliuth Triana’s Post

On a roll! Check out this awesome blog post that showcases how to optimize and scale NVIDIA NIM deployments on EKS using • Horizontal Pod Auto Scaler (HPA) + Cluster Auto Scaler (CAS) • Kubernetes Event Driver Auto Scaling (KEDA) + Karpenter

Learn how to deploy NVIDIA NIM on EKS cluster in part 1 of the blog! Stay tuned for part 2 with more details on scaling, load balancing & benchmarking NIMs on EKS.

🚀 Launch day: The Vantage Kubernetes agent now allows customers to view GPU efficiency metrics for AI intensive workloads. There's no doubt that we're in the wave of AI and as a result, a multitude of companies, both startups and enterprise, are orchestrating corresponding GPU workloads - typically using Kubernetes. These GPU workloads can be among the most expensive workloads a company ultimately runs. Thanks to a relatively new capability exposed by NVIDIA, the Vantage Kubernetes agent now automatically collects GPU usage information. This allows customers to view GPU idle costs for rightsizing Kubernetes workloads within Vantage Kubernetes efficiency reports. Given how outsized these workloads are, any efficiency gained by rightsizing these workloads can be fairly material to a company's overall cloud costs. Read the full post with all the details on our blog here: https://lnkd.in/e66bBAxY #finops #ai #kubernetes

Autopilot GPU pricing is a great example of taking a practical approach even if it deviates from past precedent. Having fully managed GPU compute with competitive pricing is a compelling reason to put your ML inference applications on Autopilot cc William Denniss

Running AI on fully managed GKE, now with new compute options, pricing and resource reservations Google Kubernetes Engine (GKE) in Autopilot mode now offers new features for running AI workloads. The addition of Accelerator and Performance compute classes improves GPU support, reduces prices for most GPU workloads, and enhances high-performance workload capabilities. These changes allow easier management and resource optimization, and can be accessed by upgrading to version 1.28.6-gke.1095000 or later.

The new NVIDIA Blackwell GPU platform is coming to AWS. Together, AWS and NVIDIA deliver the infrastructure and tools that enable customers to build and run real-time inference on multi-trillion parameter large language models (LLMs) faster, at massive scale, and at a lower cost than previous-generation NVIDIA GPUs on Amazon EC2. #aws #nvidia #genai #aiandml #cloudcomputing

Unlocking Next-Level Performance: Amazon EC2 G6e Instances Now Available for High-Performance AI and Spatial Computing Amazon EC2 G6e instances, powered by NVIDIA L40S Tensor Core GPUs, are now generally available. These instances deliver up to 2.5x better performance than G5 instances and offer up to 20% lower inference costs compared to P4d instances. Ideal for deploying large language models with up to 13B parameters, generating immersive 3D simulations, and running advanced AI inference workloads, G6e instances push the boundaries of what’s possible in machine learning and spatial computing. With up to 8 NVIDIA L40S GPUs, 192 vCPUs, 1.536 TB of system memory, and 7.6 TB of local NVMe SSD storage, G6e instances provide the power and flexibility needed for demanding workloads. Available now in US East (N. Virginia and Ohio) and US West (Oregon) regions, G6e instances can be accessed through On-Demand, Reserved, Spot Instances, or Savings Plans. To start using G6e instances, visit the AWS Management Console, AWS CLI, or AWS SDKs. For more details, check out the G6e instance page. (https://lnkd.in/gwd9FfGp) #AWS #CloudComputing #MachineLearning #HighPerformanceCompute #HPC

Short walkthrough to deploy a GPU-enabled kubernetes cluster on Azure AKS using Terraform: https://lnkd.in/edvmqKGj

To run many AI/ML workloads, you need GPUs or other specialized hardware. However, GPU access in the cloud is more complex than simply spinning up a new node type. Kubernetes offers scalability and dynamic resource allocation for AI/ML workloads running on GPU worker nodes. Fairwinds fully supports artificial intelligence (AI) and machine learning (ML) workloads in Kubernetes by providing GPU-enabled nodes. Fairwinds manages drivers for GPU vendor support, as well as autoscaling, which makes it trivial for our users to utilize GPU resources for use by AI/ML workloads. Learn more https://bit.ly/4c9ob7V