Rui Malheiro

🚀 Looking to streamline fine-tuning for models like Llama 3, Mistral, and Mixtral? Our latest blog shows how Anyscale simplifies LLMs fine-tuning with Ray: 🔑 Key takeaways: • Scalable fine-tuning with LLM-Forge • Model Deployment with RayLLM Read more: https://lnkd.in/gQwD4A2U

34

In this Technical Deep Dive, we share our approaches in using Large Language Models (#LLMs) to make numerical-type data(sets) like DHS, JMP, HDX more accessible with the example of WASH data. In this 15 min. video Olivier Mills and team cover: 1️⃣ Basic scripting to extract core data from datasets from API endpoints 2️⃣ How to use LLMs to generate LLM-friendly schema definitions 3️⃣ How to structure a AI-friendly database store that can scale to thousands of datasets and millions of data indicators 4️⃣ How to use vector querying to retrieve relevant indicators from the store based off of a natural language (english) query 5️⃣ How to build a prompt that generates valid SQL query using LLMs with the above retrieved data schema. The various components form a single AI system combining data ingestion and data exploration using a software layer, a data layer and multiple LLM layers. ⏭️ Next steps will be to expand the datasets, automate other data source ingestion and further test the system with more complex plain english queries. We plan to incorporate this as a new feature for WASH AI after testing. Feel free to reach out if you would like a copy of the Notebooks, sample prompts and data schema definitions. We are #opensource The initial work was done in collaboration with Data for Good YYC (see https://lnkd.in/g3AijS93)

3

1 Comment

Easily generate one billion embeddings in under a day... and with limited budget! How? With Ray Data from Anyscale and Pinecone! Cheng Su and Roy Miara present this can't-miss session during the #DataAISummit tomorrow. Catch it in-person or online.

43

2 Comments

This week in #TheBatch: 🎨 All about Claude’s Artifacts feature 🌱 AI growth hinders carbon emissions goals 🧠 GaLore, a memory-efficient method for fine-tuning and pretraining Plus: Andrew Ng shares his concerns about the proposed California regulation SB 1047. Read The Batch now: https://hubs.la/Q02G5YpM0

49

Combining the power of TensorFlow, NeuralForecast, and zero-shot LLMs, Jinhang Jiang shows how we can generate accurate multi-step quantile forecasts.

11

Benchmarking Federated Learning for Large Language Models with FedLLM-Bench Quick read: https://lnkd.in/g2qf5YBp Paper: https://lnkd.in/gV4JCdYG

26

Here's a snippet demonstrating how you can use Mirascope to make your LLMs take conditional actions based on the classification of an input. Check out the full example at https://lnkd.in/eV8wE5C2

11

2 Comments

Introducing RouteLLM: a sophisticated routing framework developed with Berkeley-LMSys in collaboration with Anyscale. RouteLLM optimizes query handling by dynamically selecting between high-performance proprietary LLMs and cost-effective open-source models, cutting costs by over 2x without sacrificing quality. Using human preference data and LLM-as-a-judge for data augmentation, our routers evaluate query complexity to choose the appropriate model. Rigorous testing on benchmarks like MMLU and GSM8K confirms our cost-efficient, high-quality performance. Explore our open-source code, models, and preference data on GitHub: https://lnkd.in/gJyijZRx, and try our online demo: https://lnkd.in/gME-GC2a Explore More about RouteLLM and learn how it can revolutionize your LLM applications: Read our Blog here: https://lnkd.in/gfPD-u-y LLMsys Blog here: https://lnkd.in/ga_MgERE Full research paper here: https://lnkd.in/gqRy7Pjy

81

1 Comment

GO COMMENTARY WEEKLY: FRAMEWORK & TOOLS FOR THE LLM ERA Welcome to Go Commentary by Phat Nguyen (backend engineer), exploring the latest developments in the Go programming ecosystem. Here we take a closer look at how Go is adapting to the rise of Large Language Models (LLMs). New frameworks like LangChainGo and Genkit are emerging to help developers integrate LLMs into their applications, expanding Go’s role in AI. - Go Survey: The Go team is seeking feedback from developers on how Go is used, challenges, and desired features to improve community support. - LLMs in Go: Go is being positioned as a solid choice for building LLM-powered applications, thanks to its strengths in concurrency and networking. - RAG Server Example: A simple Retrieval-Augmented Generation (RAG) server implementation demonstrates Go’s straightforward approach to LLM integration. - LangChainGo, Genkit, Ergo: Introducing tools for LLM tasks, production-ready features, and distributed systems, raising questions about balancing simplicity and innovation in Go's ecosystem. Read Dwarves Go Weekly at: https://lnkd.in/g5VKE9vf #GoLang #LLM #dwarves #software _ Dwarves Notes (https://memo.d.foundation/) combine our team’s collective know-hows, R&Ds, and operation approaches. Connect and learn alongside other tech fellows: - Discord: discord.gg/dwarvesv - Github: https://lnkd.in/gZZ2eZMu - Website: https://d.foundation

3

New Post: More Agents Is All You need This paper introduces a method that significantly enhances Large Language Model (LLM) performance by scaling up the number of agents using a sampling- https://buff.ly/44Lmmvi

1

Curious about the instruction fine-tuning process in LLMs? Check out this notebook by Sebastian Raschka, PhD for a step-by-step guide! Share your thoughts in the comments ⬇

1

Ever wonder how Large Language Models (LLMs) go from being general-purpose text processors to experts in tasks like translation, sentiment analysis, or question answering? 🤔 That's where fine-tuning comes in. 👍 🔧 How LLM Fine-tuning Works (Traditional Approach) 1. Model Selection: Start with a pre-trained LLM. 2. Dataset Preparation: Clean and format your task-specific dataset. 3. Set Hyperparameters: Define key settings like learning rate, batch size, and number of epochs. 4. Train the Model: Fine-tune the model by adjusting its parameters based on the new dataset. 5. Evaluation: Measure the model’s performance using metrics like accuracy or F1 score. While this process works, it has its challenges. 🚫 Limitations of Traditional LLM Fine-tuning 🛠️ It’s Complex: Setting up fine-tuning requires deep knowledge of frameworks like PyTorch, which is a widely-used machine learning library developed by Facebook’s AI Research (FAIR) team.   💸 It’s Resource-Intensive: The process can demand high computational power, often requiring powerful GPUs.   ⚙️ Code-Heavy: Tuning strategies and adjusting parameters often means digging deep into the code, which can slow down experimentation. 💪 Introducing Torchtune: A Simpler Way to Fine-tune Developed on top of PyTorch, torchtune makes fine-tuning large language models much more accessible. It simplifies the process and takes advantage of PyTorch's flexibility while minimizing the need for heavy coding and expensive hardware. What Makes torchtune Special? 📋 YAML Configurations: You can easily modify training parameters using YAML files without changing the code. 🧩 Modular Design: Experiment with different fine-tuning strategies and techniques with minimal coding. 💻 Memory Optimization: torchtune is designed to run efficiently on consumer-grade GPUs—no need for expensive hardware! 🤝 Seamless Integration: Works well with popular tools like Hugging Face Datasets and Weights & Biases, making it easy to add to your existing workflow. 👨‍🔧 How torchtune Works (In a Nutshell) 1. YAML Setup: Define your model and training parameters in a simple YAML file. 2. Easy Initialization: Load the configuration and start the training process with just a few commands. 3. Efficient Training: torchtune optimizes memory usage, making training smoother even on standard hardware. 4. Evaluate Quickly: Evaluate the model's performance right after training without needing to save it first. 🔎 Why Choose torchtune? 🛠 Easy to Adjust: YAML configurations make it simple to try out different setups without needing to touch the code. ✈ Flexible and Fast: Quickly iterate and experiment with various fine-tuning techniques. 💸 Cost-Effective: Designed to work even on less powerful GPUs, making it accessible for researchers, students, and professionals alike. You can go through in-detail about this torchtune, here is the GitHub link https://lnkd.in/dDW7gwiw

5

1 Comment

🚀 Want to employ the power of #PyTorch for your scientific models without a steep learning curve? 👀 Check out our latest blog, where Travis Oliphant discusses how PyTorch's integration with the Array API makes it easy to use with familiar libraries like NumPy and scikit-learn.🔥 https://lnkd.in/dDRkEY3v #Innovation

10

GenAI iteration speed is growing exponentially. One outcome is that the context window — the number of tokens a large language model (LLM) can use at one time to generate a response — is also expanding rapidly. https://lnkd.in/dr5njXhM #Database #AI #LargeLanguageModels by Usama Jamil thanks to MyScale

2

Enhancing Self-Supervised Learning with Automatic Data Curation: A Hierarchical K-Means Approach Quick read: https://lnkd.in/giitcGQy Paper: https://lnkd.in/gyN-2sWa

18

Normal Computing has been collaborating with DSPy, a cutting-edge framework developed by the Stanford NLP group aimed at algorithmically optimizing language model (LM) prompts. Thomas Ahle, Head of Machine Learning at Normal Computing, was excited to highlight our team's recent contributions enabling the DSPy compiler to take advantage of typing constraints. This provides a powerful way to create robust and type-safe predictors while seamlessly integrating with the existing DSPy ecosystem. This combination of type safety, flexibility, and optimization capabilities makes Typed Predictors a valuable addition to the DSPy toolset, empowering developers to build more reliable and efficient AI pipelines. We continue on the mission to build reliable and secure AI at Normal Computing! https://lnkd.in/dScSBqgu

22

🤔 Facing challenges with traditional Retrieval-Augmented Generation systems in your LLM projects? 🚀 Introducing GraphRAG: A Solution to Power Up LLMs with Knowledge Graphs: https://lnkd.in/gUpyKgSy Traditional retrieval systems often struggle with the complexity and volume of data, leading to suboptimal integration of information into LLMs. GraphRAG, developed by Microsoft Research, revolutionizes this by leveraging knowledge graphs to enhance data structuring, relevance, and retrieval efficiency. 📅 Join us at the DataHack Summit 2024 for an in-depth session: Topic: "GraphRAG: Powering Up LLMs with Knowledge Graphs" The speaker of this session will be Siddhant Agarwal, Developer Relations Lead - APAC at Neo4j. In this session, Siddhant will explore how GraphRAG addresses these challenges by: ➡️ Creating Knowledge Graphs: Extract entities and relationships to form a structured knowledge base. ➡️ Semantic Clustering: Organize the graph into semantic communities for better data cohesion. ➡️ Enhanced Query Processing: Generate comprehensive, diverse answers relevant to user queries. #AnalyticsVidhya #DataHackSummit2024 #DHS2024

132

4 Comments

When *open* isn't *open source* 🙋🏼‍♂️👀 Reflecting on the open-source landscape for Large Language Models, I explore the essentials: permissive licensing, transparency in training data and model architecture & more #LLM #LLMs #AI #GenerativeAI #opensource https://lnkd.in/gSJuK_J4

1