Tianshu Cheng’s Post

We’re excited to introduce Custom Servers on Baseten! To run customers’ mission-critical inference workloads, Baseten had to be great at 2 things: 1: Performance optimizations at the model level 2: Massive-scale infrastructure with cross-cloud horizontal scaling All wrapped in an expressive DevEx. We've built extensive tooling for performance optimizations—like our optimized Engine Builder in Truss. However, some of our customers come to us with pre-optimized models, mainly wanting to take advantage of the seamless autoscaling Baseten provides. With the launch of Custom Servers, you can bring your Docker image—untouched—and turn on cross-cloud autoscaling, fast cold starts, and low-latency chaining of models for compound AI systems. This makes it easier than ever to leverage our world-class infra, with blazing-fast inference and effortless autoscaling for any demand, coupled with the most expressive DevEx. Shoutout to our engineers Tianshu C., Sidharth Shanker, and Bola Malek across our Forward-Deployed and Core Product teams for this feature! Learn more in the launch blog: https://lnkd.in/e_6_GyAh Check out the demo by the lead engineer behind Custom Servers, Tianshu C.!

Leverage Karpenter for clusters with workloads experiencing fluctuating resource demands or diverse compute requirements. For static workloads, consider Managed Node Groups or Autoscaling Groups. https://lnkd.in/guCRX75i

Karpenter is a flexible, efficient, and high-performance Kubernetes compute management solution that helps improve application availability, reduce operational overhead, and increase cluster compute utilization. Alongside the graduation from beta, this 1.0 release includes three new features for Karpenter: 1/ the ability to specify disruption reasons, e.g. underutilization, emptiness, drift, for disruption budgets, 2/ a forceful disruption mode that helps customers balance application availability against security requirements, and 3/ an expansion of consolidateAfter which lets customers better tune Karpenter’s consolidation feature to meet their cost-efficiency and application availability requirements. https://lnkd.in/drDi3rXY

Who wants to run AI workloads in Kubernetes? 🙋 With Syself Autopilot and the new GPU servers from Hetzner, we make it possible to train AI models in a GDPR-compliant location. With the scalability of Kubernetes, fully managed. 🚀🤖   Check out the GPU servers here: https://lnkd.in/eqYv_VCK   Contact us today to get your trial: https://syself.com/demo New to this? 🔶 We offer a declarative, fully managed approach on Kubernetes, so you can focus on using Kubernetes. 🔶 No need for provisioning, maintaining, patching servers, or ensuring compatibility while upgrading Kubernetes. 🔶 We even test upgrades extensively, so you don’t have to spend any time on them. 🔶 No need to use Terraform—just use Kubernetes to manage Kubernetes and your tooling (e.g., kubectl, ArgoCD, Flux). 🔶 Our intelligent software manages your infrastructure, heals itself, and removes unhealthy servers.

Serverless computing can reduce overhead and increase scalability. What’s your experience with serverless solutions, and what are the main advantages or challenges? #Serverless #CloudArchitecture #ScalableTech

Serverless Computing: Is it Right for Your Project? Wondering if serverless is the perfect fit for your next project? This blog post breaks down the pros and cons. Key takeaways: • Cost-effective: Pay only for what you use. • Scalable: Easily handle fluctuating workloads. • Faster development: Focus on code, not infrastructure. Click here to read the full blog post: https://lnkd.in/eYvQGBs5 #serverless #cloudcomputing #softwaredevelopment #blog #blogpost #sinapiblog

#Azure - We recently announced the public preview of Standby Pools for Virtual Machine Scale Sets with Flexible Orchestration. Standby Pools is a new service that enables you to increase your scaling performance by creating a pool of pre-provisioned virtual machines from which your scale can pull from when scaling out. Standby pools reduce the time to scale out by performing various initialization steps such as installing applications/ software or loading substantial amounts of data. These initialization steps are performed on the virtual machines in the standby pool before to being moved into the scale set. #Compute #VirtualMachines #Scaling

Why does Lambda get all the attention? Watch Sheen Brisals explain how Lambda reaches its limits, drives new use cases, and when containers become the ideal choice for heavier workloads. Catch the full conversation here: https://bit.ly/3MEFR0k #TechTrends2024 #AIForBusiness #CloudComputing #Serverless #DigitalTransformation #AutomationTools #Productivity #TechTalks #WebTech #AntStack #AppModernization

Serverless Computing: Is it Right for Your Project? Wondering if serverless is the perfect fit for your next project? This blog post breaks down the pros and cons. Key takeaways: • Cost-effective: Pay only for what you use. • Scalable: Easily handle fluctuating workloads. • Faster development: Focus on code, not infrastructure. Click here to read the full blog post: https://lnkd.in/eRwPc35z #serverless #cloudcomputing #softwaredevelopment

Unleash Your Development Potential: Dive into Serverless Computing Serverless computing is revolutionizing the way applications are built and deployed. Say goodbye to managing servers and hello to a world of scalability, cost-efficiency, and faster development cycles. Focus on Code, Not Infrastructure: Serverless removes the burden of server management, allowing developers to focus on writing great code. Pay-Per-Use Model: Only pay for the resources your application uses, making serverless ideal for event-driven workloads and short-lived applications. Scalability Made Simple: Serverless infrastructure scales automatically based on demand, ensuring your application can handle any traffic surge. Ready to break free from the chains of traditional development? Explore serverless computing and see how it can empower your development team! #ServerlessComputing #CloudNative #CloudDevelopment #KritechTechnologies