trufflepig’s Post

🔍 New Insights Unlocked! Exploring Pandas Query Engine vs. Pandas AI🔍 In the ever-evolving world of data analysis, new tools are rewriting the rules. Pandas Query Engine(LlamaIndex) enables users to interact with data through natural language queries, making it easier to interpret large, private datasets at scale. Built on top of the Pandas DataFrame, this engine translates natural language prompts into DataFrame operations, providing a seamless way to query and manipulate data without needing SQL knowledge. Meanwhile, PandasAI (YC W24) offers AI-driven insights and natural language querying for a more intuitive, predictive experience. 🧠✨ In my latest blog, I dive into how these two approaches stack up, and which one might be the secret weapon for your next project. Whether you're after speed or intelligence, this breakdown will help you choose your data warrior! 🎙️ Podcast audio credits go to NotebookLM — an incredible research assistant from Google that not only summarizes complex sources but now also provides “Audio Overviews” for AI-generated discussions. It’s like having a personal analyst on demand! Curious to see the face-off? Check it out and join the conversation! ⚔️📊: https://lnkd.in/gmSaT3vr #DataScience #AI #Pandas #DataAnalysis #Innovation

🤖 Want to leverage LLMs to transform natural language into structured data? Here's a working example using the 𝐌𝐮𝐥𝐭𝐢-𝐀𝐠𝐞𝐧𝐭 𝐎𝐫𝐜𝐡𝐞𝐬𝐭𝐫𝐚𝐭𝐨𝐫 framework! While you'll want to adapt it for your specific needs, this sample gives you a running start for building: ✨ Convert natural language queries into structured JSON ✨ Transform complex search criteria into clean parameters ✨ Handle advanced filtering with ease Tired of cluttered e-commerce filters with endless checkboxes? This sample shows how natural language search should work. Instead of clicking through multiple filters, with this sample users can simply type: 1 - "𝑆ℎ𝑜𝑤 𝑚𝑒 𝑃𝑟𝑖𝑚𝑒-𝑒𝑙𝑖𝑔𝑖𝑏𝑙𝑒 ℎ𝑒𝑎𝑑𝑝ℎ𝑜𝑛𝑒𝑠 𝑢𝑛𝑑𝑒𝑟 $100 𝑤𝑖𝑡ℎ 𝑔𝑜𝑜𝑑 𝑟𝑒𝑣𝑖𝑒𝑤𝑠" → And get a clean, structured JSON ready for your database or API! 2 - Or ask "𝐻𝑜𝑤 𝑙𝑜𝑛𝑔 𝑑𝑜𝑒𝑠 𝑖𝑡 𝑡𝑎𝑘𝑒 𝑡𝑜 𝑔𝑒𝑡 𝑎 𝑟𝑒𝑓𝑢𝑛𝑑?" → get accurate responses from a knowledge-base powered agent! See the code in action: https://lnkd.in/ed9U6Dtj What would you build with this? Let me know in the comments! 👇 #LLM #AI #OpenSource #AWS #DeveloperTools #Innovation #mao #multiagentorchestrator #genai Anthony Bernabeu

Semantic similarity isn't the only imporant factor for retrieval. Embedding time data can help you get better results. Let's talk about recency embeddings. What's that, you ask? Basically, your datetime or timestamp data mapped onto a high-dimensional vector space. While embedding tabular data isn't new, I rarely see it utilized. fast.ai veterans might remember using some of the very first LLMs to embed tabular data for regression and classification tasks. But the practice never really took off. Why now? Well, vector databases and search engines are everywhere these days. Similar to how semantic search tends to be more nuanced than regex, recency embeddings can be more nuanced than simple filtering. Check it out yourself and don't forget to follow my new Medium account: https://lnkd.in/eCDRtxqB

I've recently shared a new article on Towards Data Science, where I explore a method for guiding generative AI to produce specific tokens. By leveraging this approach, we can effectively shape the generative process to adhere to desired formats such as JSON, SQL, or pandas code. This has profound implications for improving our ability to translate human language into computer-readable formats. https://lnkd.in/d2JViyCD

In the world of data and analytics, the ability to converse with data using natural language can seem almost like science fiction. But it’s possible. Check out BCG X’s real-life implementation of a #GenAI solution utilizing a technique known as text-to-SQL, that enables business users to access data using natural language: https://on.bcg.com/4amEsou #DataScience #DataScientist #LLM

In the world of data and analytics, the ability to converse with data using natural language can seem almost like science fiction. But it’s possible. Check out BCG X’s real-life implementation of a #GenAI solution utilizing a technique known as text-to-SQL, that enables business users to access data using natural language: https://on.bcg.com/4amEsou #DataScience #DataScientist #LLM

A Self-Healing AI Query Generator for Large Scale Tabular Data! 📊🤖 Thrilled to share table-rag+—an AI-powered framework transforming natural language queries into SQL, now with self-healing capabilities! 🌟✨ Built on TableRAG research published by Google DeepMind, table-rag+ leverages Mistral AI (mistral-nemo) via Ollama to make querying large-scale tables more intuitive and resilient. Exciting features: - Self-Healing Queries: Automatically heals failed SQL queries using an LLM and retries up to 3 times. - Foreign Key Awareness: Utilizes foreign key relationships for intelligent queries. - Contextual Query Repair: Dynamically adjusts queries based on errors. - Cell Database Sampling: Builds a database of distinct values for optimized results. The current version works with SQLite databases but stay tuned for: - Integration with SQLAlchemy. - Retrieval-Augmented Generation (RAG) for context-aware responses. - Auto-Prompt Generation for dynamic prompts across different data sources. Explore the full README (https://lnkd.in/gD5YjD6s) for installation details and the future roadmap. Let's make data more accessible and insights more discoverable together! Excited to hear your thoughts and feedback! 💬🔗 #AI #DataScience #SQL #RAG #LLMs #SLMs

Two weeks ago, I received an email about DataGemma and I thought, "I need to write a blog on this". There was a slight delay in plans but finally got to it. In this Medium blog post, I explore Retrieval Interleaved Generation (RIG)—a promising method that builds upon the well-known Retrieval Augmented Generation (RAG) approach. I examine how RIG differs from RAG and discuss its potential implications for the future. 📄 Read the original DataGemma paper here: https://lnkd.in/gGG9gUmE 📝 Check out my blog post on RIG: https://lnkd.in/geqjZksj Excited to hear your thoughts! #GenerativeAI #DataGemma #Google #RAG #LLM

Great article! Ryan Oattes. This helps explains the complexities of RAG in a simple way for everyone to understand. Thanks to our partners at Databricks and AVEVA, we are able to demonstrate the power of how Graph-driven RAG can greatly improve LLM based responsive outcomes for our customers. #RAG #LLM #GenAI #bigdata #AI Shiv Trisal Tim Garritty Anthony Sheaffer Rashesh Mody Vishal Mahna Amit Verma

In the world of data and analytics, the ability to converse with data using natural language can seem almost like science fiction. But it’s possible. Check out BCG X’s real-life implementation of a #GenAI solution utilizing a technique known as text-to-SQL, that enables business users to access data using natural language: https://on.bcg.com/4amEsou #DataScience #DataScientist #LLM