PyTorch’s Post

Amazon Web Services (AWS) has optimized the PyTorch torch.compile feature for AWS Graviton3 processors. Congratulations, Sunita, on this impressive work! Stay tuned for more details on leveraging these optimizations for faster ML workflows! #MachineLearning #PyTorch #AWS #Graviton3

See below for an opportunity to leverage the PyTorch torch.compile feature to achieve up to 2x better performance for Hugging Face model inference and up to 1.35x better performance for TorchBench model inference compared to the previous PyTorch eager mode. See below for a detailed blog - happy inferencing! #AWS #Graviton #PyTorch #AIML

Performance by default on #Neoverse with improvements to the Open Source PyTorch implementation from AWS. torch.compile is a simple interface to unlock more performance for machine learning on Arm.

If you haven’t tried running your #pytorch inference workloads on #graviton, read below and see why you should. #huggingface

It was a crazy lesson today, I learned how to set up Auto Scaling Groups in AWS. It's fascinating how you can use Machine Learning to predict your future traffic and have it scale your instances automatically. #1dayatatime

Hello. #AWS and #LLM topic. It seems we are observing different performance between US and EU hosted Anthropic/Bedrock models (supposedly the same sonnet models). Not only the inference yields different results, also the speed is different. Anyone? edit: And now it's confirmed by quality and quantity tests. Be aware: Bedrock in AWS is not offering the same models in US and EU, even though they are named the same. Too bad AWS.

Graviton Guardians 🛡️, the “Machine Learning Inference Model at Scale with Graviton” event is almost here! 🗓️Join us on August 12th to dive into how Graviton processors can enhance your ML workloads. Hemant Javeri from 2bcloud will be doing a demo as well showing you how you can get started. It’s FREE, so don’t miss out: Register NOW https://lnkd.in/exmsjXg6 🌟 #AWS #MachineLearning #ZoKnowsML

Skimming through the latest AWS updates and missed this one! Amazon Bedrock's Knowledge Bases now support Claude 3 Sonnet. Check out the details here: https://lnkd.in/gb-p94Mz #AWS #AmazonBedrock #KnowledgeBases #Claude3Sonnet #Anthropic

In high-usage production settings using LLMs can be costly, bad for the environments, and sometimes not scalable (those RPMs/TPMs in closed source models). Task-oriented Small Language Models are gaining traction in these settings to solve for cost, latency, and even achieving higher performance relative to their large counterparts. Don’t take it from me, take it from the great folks at Amazon Web Services (AWS) and Invisible / Lydia Andresen who organized an amazing workshop on Fine-Tuning and Hosting SLMs. Follow their workshop tutorial notebook yourself here: https://lnkd.in/gMqWYDv2

🌟 Day 2 of AWS re:Invent 2024—wrap up. There is a massive list of new announcements, so it’s hard to follow. AWS is streamlining and removing the undifferentiated heavy lifting on so many fields. 🚀 🧑💻I ended the day with a hands-on workshop on multimodal (text, image, video->audio) retrieval-augmented generation with LangChain on top of Amazon Bedrock and OpenSearch. I learned some new tricks, but the best part is that you could chat with the AWS experts about the topic. Huge thanks to speaker Pratul Mathur and Saurav Mohanty! #awsreinvent #heromakers