Valerio Morelli

🚀 Looking to streamline fine-tuning for models like Llama 3, Mistral, and Mixtral? Our latest blog shows how Anyscale simplifies LLMs fine-tuning with Ray: 🔑 Key takeaways: • Scalable fine-tuning with LLM-Forge • Model Deployment with RayLLM Read more: https://lnkd.in/gQwD4A2U

34

Easily generate one billion embeddings in under a day... and with limited budget! How? With Ray Data from Anyscale and Pinecone! Cheng Su and Roy Miara present this can't-miss session during the #DataAISummit tomorrow. Catch it in-person or online.

43

2 Kommentare

In the lead-up to #Raysummit, we have a guest blog on Building a RAG Batch Inference Pipeline with Anyscale and Union.ai 🚀 #Flyte is an open-source orchestrator facilitating production-grade data & ML pipelines. When combined with Ray’s distributed computing power, it unites data, platform, infrastructure, and ML engineers to boost productivity and scale across evolving use cases. Union.ai platform makes orchestrating these pipelines seamless. In the RAG Batch Inference pipeline exactly there are actually 2 pipelines that Union orchestrates: 🔹 Embedding Generation Pipeline 🔹 Batch Inference Pipeline #Flyte Deck on Union.ai lets you preview responses without downloading, ensuring instant validation. Anyscale’s Ray platform optimizes the execution of these pipelines, helping deliver leading performance and cost efficiency. Read more here 👉 https://lnkd.in/gjDKRPmu

47

Let's evaluate Llama3's RAG performance on an arXiv paper (about RAG evaluation, this is a very meta notebook, puns intended)! In this tutorial, we build on our work earlier this week in combining Unstructured's API for pdf document preprocessing with OpenAI's GPT-4o + ragas for evaluation, Hugging Face for Llama3, and LangChain to integrate all of these systems to evaluate Llama3. Llama3's metrics on RAG for our quick example: context precision: 0.9867, faithfulness: 0.8297, answer relevancy: 0.8643, context_recall: 0.9733 Try this out with your favorite unstructured data by swapping the arXiv pdf URL in the notebook! Colab Notebook: https://lnkd.in/gSndzTuC

26

1 Kommentar

It’s day 4 of our 11 Days of Inference Acceleration Techniques. Today, we’re moving on to runtime level optimization best practices. 𝐅𝐨𝐮𝐫𝐭𝐡 𝐭𝐢𝐩: 𝐓𝐚𝐤𝐞 𝐚𝐝𝐯𝐚𝐧𝐭𝐚𝐠𝐞 𝐨𝐟 𝐠𝐫𝐚𝐩𝐡 𝐜𝐨𝐦𝐩𝐢𝐥𝐚𝐭𝐢𝐨𝐧 📈 Graph compilers such as TVM, Tensor-RT, and OpenVino work by getting a computation graph of a specific model and generating an optimized code adjusted for the target hardware. Graph compilation can optimize the graph structure by merging redundant operations, performing kernel auto-tuning, enhancing memory reuse, preventing cache misses, and more. But few things to be aware of: 📝 Not all models compile equally. 📝 The impact of compilation on model performance can vary. Hence, make sure to check the architecture's ability to compile early on in the process to avoid wasting time and resources on training a model that can’t be optimized for fast inference. – What’s the #11DaysofInferenceAccelerationTechniques? The Deci team is posting, for 11 days, a series of inference acceleration techniques for deep learning applications. If you’re looking for practical tips and best practices for improving inference, follow Deci AI so you won’t miss an update. #deeplearning #machinelearning #neuralnetworks #computervision

9

Yann LeCun: The problem isn't that it is a transformer. The problem is that it is an auto-regressive LLM. Auto-regressive LLMs that compute each token with a fixed number of computational steps can't reason, regardless of the details of the architecture. https://lnkd.in/gBffFgbq

1

Learn how you can convert a PyTorch model, trained on the web, to a mobile-optimized format. Madhur Zanwar describes the process using a few lines of code. He also discusses his journey and how one approach didn't yield satisfactory results while trying to run it in a React Native app. #machinelearning #pytorch #computervision #edgeml #reactnative #objectdetection #yolov5

14

🚀 Want to employ the power of #PyTorch for your scientific models without a steep learning curve? 👀 Check out our latest blog, where Travis Oliphant discusses how PyTorch's integration with the Array API makes it easy to use with familiar libraries like NumPy and scikit-learn.🔥 https://lnkd.in/dDRkEY3v #Innovation

10

🚀 Check out the latest Medium post by our student Tianyunxi Yin: "Quantization Fundamentals for Model Compression". Explore the essentials of quantization and learn how it helps in compressing machine learning models for better efficiency without compromising performance. Read more here: https://lnkd.in/gHbbR_J4 #USFMDS #DataScience #MachineLearning #ModelCompression #Quantization

15

New Blogpost: Fine-tuning LLMs for Multi-turn Conversations In this deep dive we cover: 🎛️ How to fine-tune with conversation data. 🎭 Implement loss masking to improve model performance. 🐍 Full code example fine-tuning Llama 3.1 8B on CoQA dataset. Read here: https://lnkd.in/gFxbpia6

18

Pivistation 5: Harness the Power of PyTorch https://lnkd.in/g2BYQ-Wn Discover the power of pre-installed software with Pivistation 5. With PyTorch already integrated into the camera, implementing object recognition has never been easier. Pivistation 5 is the world's first Raspberry Pi 5-based all-in-one camera kit coming with rich pre-installed camera drivers and an extensive range of software, including essential tools like OpenCV-Python, libopencv-dev (C++ version), libcamera, rpicam-apps, C++ essential tools (building environment), TensorFlow (Python), PyTorch, camera streamer, Arducam camera server, and more. To know more about Pivistation 5, visit here: https://kck.st/4dK9l9o

1

🌐 Exciting Announcement! We are thrilled to host Fangru Lin for a discussion on her groundbreaking work, "Graph-enhanced Large Language Models in Asynchronous Plan Reasoning," recently presented at ICML 2024! 🎉🧠 Join us on August 24, 2024, at 10:00 AM (EST) as Fangru Lin walks us through the innovative "Plan Like a Graph (PLaG)" approach. This method revolutionizes LLM performance by breaking down tasks into sub-tasks, forming an execution graph that is either solved in parallel or sequentially by the LLM. This structured approach not only enhances results but also opens new pathways for future research in prompt engineering. 📜 Paper Abstract: Planning is a fundamental property of human intelligence. Reasoning about asynchronous plans is challenging since it requires sequential and parallel planning to optimize time costs. Can large language models (LLMs) succeed at this task? Here, we present the first large-scale study investigating this question. We find that a representative set of closed and open-source LLMs, including GPT-4 and LLaMA-2, behave poorly when not supplied with illustrations about the task-solving process in our benchmark AsyncHow. We propose a novel technique called Plan Like a Graph (PLaG) that combines graphs with natural language prompts and achieves state-of-the-art results. We show that although PLaG can boost model performance, LLMs still suffer from drastic degradation when task complexity increases, highlighting the limits of utilizing LLMs for simulating digital devices. Our study is an exciting step towards using LLMs as efficient autonomous agents. Our code and data are available at https://lnkd.in/e49YXamU. Check out the full paper: https://lnkd.in/eCdsdhE4. 📅 Save the Date: 🗓️ August 24, 2024 🕙 10:00 AM (EST) 📍 Location: 🌐 Virtual Event Join via Zoom: https://lnkd.in/eSq8PKNe 🚀 Don't miss this opportunity to gain insights into the future of LLMs and their application in complex tasks! 🌟 #ICML2024 #LLMs #MachineLearning #GraphLearning #AIResearch Looking forward to your participation! Spread the word and let's dive into the future of LLMs together! 🚀

13

𝗣𝗮𝗿𝗮𝗺𝗲𝘁𝗲𝗿-𝗘𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝘁 𝗙𝗶𝗻𝗲-𝗧𝘂𝗻𝗶𝗻𝗴 (𝗣𝗘𝗙𝗧) PEFT is an optimization technique for fine-tuning large language models (LLMs) by only modifying a small subset of parameters, instead of adjusting the entire model. This significantly reduces computational resource requirements while maintaining high performance. Popular PEFT techniques include 𝗟𝗼𝗥𝗔 (𝗟𝗼𝘄-𝗥𝗮𝗻𝗸 𝗔𝗱𝗮𝗽𝘁𝗮𝘁𝗶𝗼𝗻), 𝗣𝗿𝗲𝗳𝗶𝘅 𝗧𝘂𝗻𝗶𝗻𝗴, 𝗮𝗻𝗱 𝗣-𝗧𝘂𝗻𝗶𝗻𝗴. Common PEFT techniques: 1 .𝗟𝗼𝗥𝗔 (𝗟𝗼𝘄-𝗥𝗮𝗻𝗸 𝗔𝗱𝗮𝗽𝘁𝗮𝘁𝗶𝗼𝗻): LoRA works by "freezing" most of the original model’s parameters and only training a pair of low-rank matrices. This reduces the number of trainable parameters from tens of thousands to just a few thousand, saving resources while maintaining good performance on specific tasks like text classification and language translation. 𝟮. 𝗣𝗿𝗲𝗳𝗶𝘅 𝗧𝘂𝗻𝗶𝗻𝗴 𝗮𝗻𝗱 𝗣-𝗧𝘂𝗻𝗶𝗻𝗴: These techniques add parameters into different layers of the model or within embeddings. They are optimized for tasks such as Natural Language Understanding (NLU) without the need to fine-tune the entire neural network. https://lnkd.in/gTM7r_tf #PEFT #FineTuning #LLMs

1

Torch Compile is here to optimize inference in large language models (LLMs) with minimal overhead. In Benjamin Marie's latest article, explore how adding just one line of code can improve decoding performance. #Llama LLM

14