Vivedha Elango’s Post

Demo video of using natural language to query a SQL database using LlamaIndex and OpenAI. This was adapted from a LlamaIndex example to include foreign keys between tables. Repo in the comments. #llm #llamaindex #openai #sql

The true power of LLMs isn't in stand alone chat bots, it's in integrating LLMs into existing systems to augment their efficiency. Learn how we can use LLMs to automate some aspects of data analytics to save time internally and create fresh offerings for customers.

Talk to Your Data: Learn how natural language prompts change the data analytics game. This blog post explores using AI to automate SQL query generation, from database context loading to result presentation. Click the link to discover how this technology can benefit your business!

I'm excited to share that I developed a LLM based Text to SQL Tool 🎉 Can't stop sharing how accurately the queries were written starting from simple to complex. Thinking how vastly it could simplify the life of a developer! From spending hours writing complex queries to now accomplished instantly! Awesome response when RAGGED with Finetuning. What's next? Are you an SQL expert? If Yes - Here is the query you need. Utilize your time to focus on other critical tasks. If No - Don't Worry! I can generate it for you. Use your time to learn and grow. #SQL #AI #TechInnovation #DeveloperTools #NaturalLanguageProcessing

Some general musings: Output parsers are your friend. Sometimes using the llm to simply determine intent and then passing a predetermined class of intent to your own custom code or document query is the move rather than slaving away at complicated prompts. I've also found that oftentimes people are looking for the model to do all of the work for them. It's not magic. Inject yourself into the chain for the win!

Killer new vector database! These are the most important components of a RAG system, and we just got a very promising open-source alternative that brings something different: A SQL interface + an AI-first approach. This is a big deal. https://lnkd.in/em7iQdaP Check the open-source project here: https://lnkd.in/ebnfxPtd MyScaleDB was built from scratch for production AI applications, and you can use it using SQL. SQL is the most popular language to deal with data, and now we have a vector databases that uses it as well! MyScaleDB is faster and cheaper than other vector databases. Check this comparison: https://lnkd.in/eCpS7BcX This is a huge win for the open-source community!

Vector databases do much more than just store vector embeddings: As you can imagine, retrieving data based on similarity requires a lot of comparing between objects and thus can take a long time. In contrast to other types of databases that can store vector embeddings, a vector database can retrieve data fast. To enable low-latency search queries, vector databases use specific algorithms to index the data. Additionally, some vector databases, like Weaviate, store the vector embeddings and the original data object, which lets you combine traditional search with modern vector search for more accurate search results.

Wondering if Claude 3 is good at SQL query generation? Rajesh Balamohan has rigorously tested Anthropic's Claude 3 Opus, the behemoth of the Claude 3 series, against text-to-SQL tasks, comparing its capabilities with GPT-4-turbo. For straightforward, Spider-like SQL queries, Opus is on par with GPT-4-turbo's performance. However, it falls short by 20-30% for complex queries involving semi-structures or intricate joins, sub queries, etc. Primarily due to errors that are syntactically correct but semantically inaccurate. We're eager to see further advancements in LLMs for SQL generation, especially for complex schemas. The progress of models like Claude 3 is promising, and we hope to see others catch up or even surpass GPT-4 in SQL query generation soon. A side note: I've also put Opus to the test in other areas such as classification, data generation, and found it to be more versatile, generating a wider array of outputs compared to GPT-4's more concise and uniform responses. Blogpost: https://lnkd.in/dQnx4pjn #SQL #LLM #Claude3 #GPT4 #Anthropic #OpenAI

Check out the article by Waii on how to address challenges in agentic Text2SQL with vector databases! Text2SQL is becoming increasingly popular, and name enumeration works well with low-cardinality data, such as a column for U.S. states with 50 unique values, which are manageable. However, this approach is not practical when dealing with high-cardinality data, like a product catalog with millions of unique product names. This is where vector databases and semantic search come into play. See how this works in the real world: https://lnkd.in/giWc83xx #Vectordatabase #Milvus #SQL