Aravind Parameswaran’s Post

Struggling to optimize your RAG setup? Not sure if you chunked your data right to enable optimal context retrieval? Not sure which embedding model will work best for your data? Don’t worry, you’re not alone.. A RAG has several moving parts: data ingestion, retrieval, re-ranking, generation etc.. Each part comes with numerous options. If we consider a toy example, where you could choose from: 5 different chunking methods, 5 different chunk sizes, 5 different embedding models, 5 different retrievers, 5 different re-rankers/ compressors 5 different prompts 5 different LLMs That’s 78,125 distinct RAG configurations! That's 271 days of non-stop trial-and-error effort (assuming you can build & evaluate each option in just 5 mins)! In short, it’s kinda impossible to find your optimal RAG setup manually. So, how do you determine the most optimal RAG configuration for your data and use-case? Use hyperparameter tuning - an ML technique for identifying the optimal values for your parameters when there’s a large set of possible values. But, how do you do it without writing a bunch of code to do hyperparameter tuning? Use RAGBuilder -> RagBuilder takes your data as an input, and runs hyperparameter tuning on the various RAG parameters (like chunk size, embedding etc.) evaluating multiple configs, and shows you a dashboard where you can see the top performing RAG setup, and in 1-click generate the code for that RAG setup. So you can go from your RAG use-case to production-grade RAG setup in just minutes. Best part, it’s open source with active contributors. Github Repo link: https://lnkd.in/g_igANZy What challenges have you faced in optimizing your RAG setup? Let me know in the comments below. #AI #LLM #RAG #DataScience #Optimization

DataStax and Unstructured — a POWERFUL partnership! This blog shows how Unstructured + DataStax Astra DB can quickly convert common document types into vector data for highly relevant GenAI similarity searches ⤵️ https://ow.ly/xPGm50SyW4s #DataStax

DataStax and Unstructured — a POWERFUL partnership! This blog shows how Unstructured + DataStax Astra DB can quickly convert common document types into vector data for highly relevant GenAI similarity searches ⤵️ https://ow.ly/bBoB50SwyVz #DataStax

DataStax and Unstructured — a POWERFUL partnership! This blog shows how Unstructured + DataStax Astra DB can quickly convert common document types into vector data for highly relevant GenAI similarity searches ⤵️ https://ow.ly/b6VK50SwUG9 #DataStax

DataStax and Unstructured — a POWERFUL partnership! This blog shows how Unstructured + DataStax Astra DB can quickly convert common document types into vector data for highly relevant GenAI similarity searches ⤵️ https://ow.ly/5pV850SxOPs #DataStax

DataStax and Unstructured — a POWERFUL partnership! This blog shows how Unstructured + DataStax Astra DB can quickly convert common document types into vector data for highly relevant GenAI similarity searches ⤵️ https://ow.ly/CLxx50Sy9Jb #DataStax

DataStax and Unstructured — a POWERFUL partnership! This blog shows how Unstructured + DataStax Astra DB can quickly convert common document types into vector data for highly relevant GenAI similarity searches ⤵️ https://ow.ly/hkXT50SyMRz #DataStax

DataStax and Unstructured — a POWERFUL partnership! This blog shows how Unstructured + DataStax Astra DB can quickly convert common document types into vector data for highly relevant GenAI similarity searches ⤵️ https://ow.ly/Q4E250SxhyR #DataStax

DataStax and Unstructured — a POWERFUL partnership! This blog shows how Unstructured + DataStax Astra DB can quickly convert common document types into vector data for highly relevant GenAI similarity searches ⤵️ https://ow.ly/OWB950SyCqV #DataStax

DataStax and Unstructured — a POWERFUL partnership! This blog shows how Unstructured + DataStax Astra DB can quickly convert common document types into vector data for highly relevant GenAI similarity searches ⤵️ https://ow.ly/wM8V50SyA3E #DataStax