Weights & Biases’ Post

Hi everyone, three new LLAMA 3.1 LLMs have been released: 8B, 70B, and 405B. You can explore LLAMA 3.1 on Databricks and an optimized version on UNSLOTH (note: the 70B model requires an 80GB GPU and has a reduced context length of 48K instead of 128K). 🔎 The most intriguing part might be the technical paper here. It offers valuable insights into building and optimizing AI models. Check out page 54 for an interesting vision section and page 56 for video pre-training details. 📌 However, only the text version (LLM) will be publicly available for now, as the vision model is considered too risky for use outside META’s platform.

We’re excited to announce that we have released our SDK and web console! Using our SDK, everyone can now fine-tune LLMs like Llama 3.1 with just a few lines of code! You don’t have to set up compute instances, build ML infra, or pay ridiculous fees for GPUs. Our web console also enables anyone to fine-tune LLMs, even if you don’t know how to code. All you need to do is pick a LLM, upload a dataset, and hit start! To learn more about our SDK and web console, please visit: https://docs.luminolabs.ai or https://app.luminolabs.ai. This is the first step in our mission to let everyone in the world build AI in a safe, affordable, and quick manner! AI should not be something just accessible to conglomerates. If you’re interested in joining us our journey, apply at https://lnkd.in/g-2z2qse 

Extracting structured data from documents used to be extremely hard. With the rise of great open source models with vision capabilities this suddenly gets much easier and cheaper. Just some weeks ago, you would've needed to purchase 500k in GPUs and massively train & fine tune a full pipeline of ML models just to get some data from documents (if you had multiple docs with multiple layouts for example). Today, however, you can simply use an off-the-shelf LLM with vision capabilities - and you have the best parser you can imagine. Read more about how this works in our latest post https://lnkd.in/dJ85k_fE

A customised LLM augmented with local files. Looking forward to trying this, thanks Nomic AI

Fine-tune the latest Phi-3-128K-Instruct on a single GPU! I have built a notebook using which you can fine-tune Phi-3-128K-instruct on your own custom data and unlock Phi-3's full potential! With this notebook, you'll discover how to fine-tune Phi-3-128K-instruct on your custom data using PEFT's QLoRA – on a single GPU. Google Collab link in comments. 💼 #Microsoft #Finetuning #LLM #PEFT #QLoRA #AI #MachineLearning #GoogleColab

It's quite incredible to see how far open-source LLMs have come. I still remember 2 years ago fighting to download the 300GB of weights required to run BLOOM on a cluster of 8xA100. Today, I'm excited to share a guide that demonstrates how you can deploy your own private open-source state-of-the-art code generation model that rivals GPT-4o in performance, for just $0.3/hour! By combining: • Qwen2.5-Coder-32B impressive performance • Quantization techniques compression abilities • llama.cpp's efficient inference server • vast.ai's affordable GPU infrastructure You can get: ✅ Enterprise-grade coding performance ✅ ~40 tokens per second generation speed ✅ Complete privacy and control ✅ Single consumer grade GPU deployment ✅ No dependency on commercial API providers Running high-performance LLMs do not require massive infrastructure investments anymore. This is a game-changer for individual developers and companies wanting to maintain complete data privacy while leveraging the cutting-edge AI capabilities. Check out the detailed guide attached. PS: it also shows how to achieve Cursor like user experience by integrating your code generation model in continue.dev or PearAI. * currently limited to ~15k tokens context window on RTX4090

💰Let's examine the pricing of Llama 3.1 405B. ⚡️405B is revolutionizing AI accessibility and affordability so far. As industrial leaders deploy the Llama 3.1 405B model on servers, they offer remarkably competitive pricing. 👇 Check it out: 🟢 Llama 3.1 405B: $3 / $3 per million tokens for both input and output (https://lnkd.in/d9efkw_6) 🔵 Claude 3.5 Sonnet: $3 input / $15 output per million tokens 🟣 GPT-4: $5 input / $15 output per million tokens

Are you looking for a better way to manage GPU-enabled VMs for your AI and machine learning tasks? In his latest demo, Diego Ciangottini explores how to use Dagger to streamline this process. He demonstrates how to: 🔧 Build a Dagger module to deploy and manage VMs on demand 🤖 Integrate seamlessly with AI frameworks and Kubernetes clusters ⚙️ Optimize infrastructure for machine learning and AI workloads If you're tackling GPU-enabled workloads and want to simplify your deployment process, this video will show you how Dagger can help! 📺 https://lnkd.in/gx3RMT7c

OpenAI's GPT-4 used $78 million worth of compute to train, while Google DeepMind's Gemini Ultra used $191 million and 50 billion petaFLOPs for compute. Explore this blog to understand the implications of the rising costs on machine learning and our future: 🧠 https://bit.ly/47ID8LP