Karteek Pradyumna Bulusu

Microsoft's open-source repo on GraphRAG is gaining significant traction, If you're wondering if it suits your needs, here's an overview. What? GraphRAG uses LLMs to construct knowledge graphs from data and answer user queries based on private datasets. Unlike traditional RAG methods that rely on vector similarity, GraphRAG leverages LLM-generated knowledge graphs to significantly enhance question-and-answer performance, especially for complex document analysis. Why graphs? Graphs excel at connecting different pieces of information through their relationships, making them ideal for synthesizing insights and performing complex analytical tasks, unlike flat, unconnected data. When should you use it? GraphRAG could be ideal for scenarios requiring deep information discovery and analysis across multiple noisy documents or datasets containing mis/ dis-information. GraphRAG can also be beneficial in scenarios where a straightforward semantic search based solely on the query may not suffice. The answer may reside in related information rather than the query term alone. When to skip it? GraphRAG's effectiveness depends on well-structured indexing, particularly in unique datasets with domain-specific concepts. Indexing can be resource-intensive, requiring careful setup and testing before extensive use. GraphRAG's effectiveness also hinges on its ability to accurately identify nodes related to a query, a task that is super straightforward in semantic search. You can always combine both semantic and graph-based methods to leverage the strengths of each approach. link - https://lnkd.in/gFqzRp4i Microsoft #graphrag #microsoft

7

Dear companies, take care of your AI talent, especially the researchers. Researchers? Really? Aren't researchers the least influential people in the company? - They are the cost-centers. They do not directly sell to customers & so, do not generate revenue. And company has to fund them from profits - They live far away from reality. They do not know the "ground-truth". - They are not enterprising or charismatic like the #marketing & #sales' dashing & street-smart folks. - They cannot converse in normal language but are deep inside their narrow, un-understandable world. - Even if they do not exist in the company, the current product portfolio can be enough to sustain sales for years, or even decades. Look at ROLEX. Their watches are as mechanical as they were decades ago! Clearly their R&D budgets for adopting consumer-side emerging technologies are near-zero. - They help achieve company's Reduction in Force (RiF) targets. This is probably the only reason a company & its other employees may want them - for own survival. As they get hit first. So, why should companies care about their AI researchers, if at all they have them? The world has changed. They are one of the most influential people in the tech world and its only likely to increase with time. Why? We do not need every researcher to come up with a newer neural architecture or a different LLM, VLM or LMM. Right? That's true. But their role has just expanded and we still do not know. They are some of the best equipped people to manage the speed of current AI changes, naturally. Look, the AI world has been so rapidly advancing in the past 2 years that leaders are finding it difficult to keep track of them. Even if they succeed to keep track of them, their AI developments & implementations are becoming obsolete within 6-9 months! Ask a leader, and you'll invariably hear this. So? How will AI researchers help? They know the next in-thing (or likely to be one) at least a year before the world. And so, they can guide the next #AI #development or implementation within an organization that can sustain itself for a little over an year and a half, at least. Even if they are not implemented, at least the leaders & organizations can plan their 3-year strategies (Short Range Planning) based on their inputs. Are there any examples? AI Agents. They have been in existence for over an year. But organizations are still implementing LLMs. In spite of #Devin going viral! So tell me, what's the next in-thing that companies should think of? While there are nuanced answers to this question, some of the prominent ones, in general, would be - Natural Conversations, Reinforcement Learning & #Humanoids. But not every future in-thing actuates in future. No? True. And hence, we even need a Discriminator & not just a Generator AI researcher ^⁠_⁠^ #GAN #Future #HLM

21

3 Comments

When it comes to "Retrieval" phase of RAG, choosing the right strategy makes all the difference. Sparse retrieval (like BM25/TF-IDF) is great for finding exact keyword matches, while Dense retrieval (like BERT) understands the meaning behind words for more accurate results. Hybrid retrieval combines both, offering the best of both worlds. Each method has its strengths. Understanding when to use them is key to better search results. 

10

4 Comments

LangChain's modularity and flexibility enable the seamless orchestration of various LLM components, making it an ideal framework for constructing intricate conversational flows. Paired with LangGraph's visual design capabilities for managing these flows, development processes are notably more intuitive and streamlined. Read my take. #machinelearning #LLM #Langchain #Langgraph

13

Check out this highly recommended course for those interested in optimizing their models: - Introduction to Quantization - Symmetric and Asymmetric Quantization Modes - Hands-on Implementation from Scratch using PyTorch - covering Per Tensor, Per Channel (Axis), and Per Group techniques - Weights Packing: Exploring 2 or 4 Bits Quantization Methods https://lnkd.in/gMAuWXHm

43

Data Scientists, This is BIG! Snowflake's Arctic LLM is Here – and Open-Source! 🤯 Get your hands on it now: Interactive Demo: Explore Arctic's capabilities with this Streamlit app → https://lnkd.in/gJ7y38Ng Replicate: Ready to experiment? Find it on Replicate → https://lnkd.in/gyDXiMju Why This Matters: Arctic lets you quickly build demos of your models. This is a game-changer for showcasing your work and making your models more accessible! Huge thanks to Snowflake for making this happen! Are you excited to try Arctic? Let me know in the comments! 👇 #LLM #OpenSource #DataScience #Streamlit #Snowflake

8

Question : Why does an AI product manager need to monitor #AI model❓ Once AI model is developed and deployed, the job doesnt end there. Model encodes pattern of the data it has seen during training phase. There may be several patterns or edge cases which are not part of training data. If model encounters any such patterns in testing or production phase, it will produce results which wont make sense without giving any warning about it. 👉 Data drift is a common challenge faced by AI-powered products. Over time, the distribution of input data can shift, leading to a mismatch between the model's training data and the real-world data it encounters. This can result in degraded model performance, unexpected outputs, and even complete model failure. 👉 The consequences of data drift can be severe, ranging from suboptimal user experiences to regulatory compliance issues. Product managers must be vigilant in detecting and addressing these challenges to ensure the continued success of their AI-powered offerings. 👉 Sudden changes in metrics can indicate input data drift. Additionally, monitoring the distribution of model outputs can reveal output drift, which may signal a need to retrain or fine-tune the model. Detecting data drift early is crucial. Product managers can leverage techniques such as A/B testing, anomaly detection, and continuous monitoring to identify and address these issues. Once detected, the product team can take appropriate actions, such as retraining the model, adjusting the data pipeline, or even considering a model architecture change. 💡 Quick Tip: Integrate model monitoring into your product roadmap and allocate dedicated resources to ensure the long-term success of your AI-powered offerings. Establish a comprehensive model monitoring dashboard to gain real-time visibility into the health and performance of your AI models. #artificialintelligence #productmanagement #datascience #productmanager

5

🎯 Unlocking the Power of Retrieval Metrics in Information Retrieval Systems 🎯 In the fast-evolving world of AI and machine learning, understanding and optimizing retrieval metrics is crucial for building effective information retrieval systems. Whether you're working on search engines, recommendation systems, or Retrieval-Augmented Generation (RAG) pipelines, these metrics provide the key insights needed to enhance performance and user satisfaction. Let's dive into the essential retrieval metrics you should know! 📊 📌 Precision@k Precision@k measures the proportion of relevant documents in the top-k retrieved documents. A high Precision@k indicates that the system is good at retrieving relevant results. 📌 Recall@k Recall@k evaluates the proportion of relevant documents retrieved out of all relevant documents available. It's a measure of completeness. 📌 Mean Reciprocal Rank (MRR) MRR is the average of the reciprocal ranks of the first relevant document. It rewards systems that return relevant documents at higher ranks. 📌 Normalized Discounted Cumulative Gain (NDCG) NDCG measures the ranking quality by considering the position of relevant documents. It accounts for the fact that retrieving relevant documents at higher ranks is more valuable. Why Are These Metrics Important? These metrics help us understand how well our retrieval system is performing and where improvements are needed. By optimizing these metrics, we can ensure that users find the most relevant information quickly and efficiently. 🚀 Implementation Tips: Collect Data: Prepare a dataset with queries and corresponding relevant documents. Run Retrieval: Execute the retrieval step and collect the top-k results. Compute Metrics: Calculate Precision@k, Recall@k, MRR, and NDCG for an in-depth performance analysis. 🔍 Stay ahead in the game by mastering retrieval metrics and transforming your information retrieval systems! #AI #MachineLearning #InformationRetrieval #DataScience #Metrics #Precision #Recall #MRR #NDCG #Innovation #Tech

1

1 Comment

Evaluating Large Language Model (LLM) performance in production involves monitoring various metrics to ensure the model meets business requirements and user expectations. Here are key metrics to evaluate LLM performance: 1. _Accuracy_: Measures the model's ability to generate correct responses. 2. _F1-score_: Evaluates the model's balance between precision and recall. 3. _Perplexity_: Assesses the model's ability to predict the next token in a sequence. 4. _ROUGE score_: Measures the quality of generated text compared to reference text. 5. _BLEU score_: Evaluates the model's ability to generate fluent and coherent text. 6. _Latency_: Monitors the model's response time to ensure it meets performance requirements. 7. _Throughput_: Measures the model's ability to handle multiple requests simultaneously. 8. _Error rate_: Tracks the frequency of errors or exceptions. 9. _User satisfaction_: Collects feedback through surveys, ratings, or user interviews. 10. _Business metrics_: Aligns with business objectives, such as revenue, engagement, or conversion rates. Additionally, consider: 1. _A/B testing_: Compares the performance of different models or configurations. 2. _Continuous monitoring_: Regularly tracks metrics to detect performance degradation. 3. _Model interpretability_: Analyzes feature importance, attention weights, or visualizations. 4. _Adversarial testing_: Evaluates the model's robustness to edge cases or attacks. 5. _Data quality_: Monitors the quality and relevance of training data. By evaluating these metrics, you can ensure your LLM model performs optimally in production, meets user expectations, and drives business value.

4

𝐂𝐡𝐚𝐭𝐆𝐏𝐓 𝟒𝐨 can make 𝐛𝐢𝐠, 𝐮𝐧𝐢𝐧𝐭𝐞𝐥𝐥𝐢𝐠𝐞𝐧𝐭 mistakes 😱! Today, I utilized ChatGPT to calculate the correlation between two binary variables. While it may seem trivial (Fig 1 - blue), the result was unexpected. The tool initially calculated 2.48 (Fig 2 - red), highlighting an error in the square root computation. Upon correction, the accurate correlation coefficient of 0.795 was obtained (Fig 3; green). Here is the link to my GPT chat: https://lnkd.in/djB5Jbra Reflecting on this, the question arises - Can we truly trust ChatGPT for complex tasks when it falters on basic calculations? Perhaps, as I encountered, a cautious approach is beneficial. I would recommend this prompt "𝐂𝐡𝐚𝐭𝐆𝐏𝐓 𝐦𝐚𝐤𝐞𝐬 𝐦𝐢𝐬𝐭𝐚𝐤𝐞𝐬. 𝐂𝐡𝐞𝐜𝐤 𝐞𝐯𝐞𝐫𝐲𝐭𝐡𝐢𝐧𝐠," instead of "𝐂𝐡𝐚𝐭𝐆𝐏𝐓 𝐜𝐚𝐧 𝐦𝐚𝐤𝐞 𝐦𝐢𝐬𝐭𝐚𝐤𝐞𝐬. 𝐂𝐡𝐞𝐜𝐤 𝐢𝐦𝐩𝐨𝐫𝐭𝐚𝐧𝐭 𝐢𝐧𝐟𝐨." Have you encountered similar instances of AI discrepancies? I invite you to share your experiences. Thank you.

66

3 Comments

We had this coming: MoA LLM -> Mixture of Experts (MoE) Agents -> #Mixture of Agents (MoA) Let's understand them in simple terms. #MoE: Within one large model, sub-models are created on specific parts of data to develop an expert on that part of data. E.g. One sub-model specializes in predicting health conditions for Native Hawaiians while another sub-model is specialized for Asians. MoE reduces training time, computational costs while maintaining (or improving) accuracies of #LLMs. Twitter / X's #Grok-1, Mistral AI's #Mixtral 8x7B (8 experts of 7 billion parameters each), Databricks's #DBRX are based on this architecture. Ensemble models (EM): Why am I suddenly jumping to EM? Because, in your discussions, this may come up as a concept similar to MoE at #LLM-level, just like EM is at traditional #ML-level (e.g. Gradient Boosting Random Forest). But there is one distinction - In EM, each model is trained on all the data while in MoE, each sub-model (expert) is trained on a subset of data. #MoA: While MoE combines multiple neutral networks (#experts), MoA combines outputs from multiple Agents (potentially accessing multiple LLMs internally). Advantage: More accurate response with improved quality. Disadvantage: Longer inference time; to the time order of 5x. Solution: Groq. It reduces inference time to as low as 1x! #ArtificialIntelligence #AI

20

Hello all, I and my friend Aditya Sahu have been working lately on Automatic Redaction / Replacement tool for PDFs (scanned PDF & text-readable) and for Images which uses large language models (LLMs) in the backend. Different Formats / templates of PDFs cause a major problem in the extraction process. *What it does: 1. It removes the editing panel for your redaction / replacement work and instead you can just explain what you need to redact / replace in normal keywords or plain sentences in a comma separated fashion. 2. It uses LLMs in the background - you can use your own privately deployed model like llama 3 70b fine-tuned version or you can also use models like GPT-4o, Claude etc. which are commercially available. It can be configured based on your requirement. 3. It uses OCR for Image / Scanned Image case, which can be configured to any Cloud Server OCRs as well as open-source OCRs. 4. Can ingest multiple images and PDFs and work sync. 5. You can run multiple iterations on the same file to make many small small changes that can be possible in the PDF. Note: The process is not fully 100% accurate but it is well in range of 85-90+ , the accuracy can vary based on the choice of LLM model with good context length and choice of OCR as well as the quality of image. [*imp] *PDFs are quite complex for handling the formatting issues, so this tool does have limitations on what length of words can be replaced because it might lead to formatting issues in the provided PDF. There would be HIL [human in the loop] for evaluation. You can go through the Demo once; it has 2 parts in it - one for scanned PDF and one for text readable pdf (built using Python) 😀. Right now, it is just a Streamlit demo but parallelly in the backend work is going on... Please DM to me Mohit Singh Pawar or Aditya Sahu for more details or clarification related to its availability. Audio credits: ElevenLabs free credits. I guess will have to upload part 1 here and part 2 in a separate post. part 2 : https://lnkd.in/gEi-DUzb #redact #llm #replace #pdf #genai #generativeai

56

5 Comments

Learn what's new with RAGs in my latest article #LLM #RAGs #MachineLearning

8

1 Comment

Databricks has just announced the launch of Databricks AI/BI, a revolutionary business intelligence product designed to democratize analytics and insights across organizations. Featuring AI-powered dashboards and the Genie conversational interface, AI/BI allows users to generate and interact with complex data insights through natural language. This new product, powered by a compound AI system, continuously learns from usage and provides accurate, real-time answers without the need for extensive setup or modeling. This marks the beginning of a new era where data science, data analysts, and data engineers can effortlessly generate and answer questions about data. This technology makes accessing and understanding data insights from high-level leadership much easier, even for those without deep technical expertise. https://lnkd.in/gAGcApe8 #DataScience #Analytics #BusinessIntelligence #AI #MachineLearning #Databricks

10

Hallucination in Large Language Models (LLMs) refers to the generation of factually incorrect or nonsensical content. To overcome hallucination: 1. _Improve training data quality_: Ensure data accuracy, diversity, and relevance. 2. _Increase model capacity_: Larger models can better capture complex relationships. 3. _Regularization techniques_: Apply techniques like dropout, weight decay, or data augmentation. 4. _Fact-checking and verification_: Integrate external knowledge sources or human evaluation. 5. _Adversarial training_: Train the model to detect and correct hallucinations. 6. _Uncertainty estimation_: Quantify the model's confidence in its outputs. 7. _Human feedback and iteration_: Incorporate human feedback to refine the model. 8. _Knowledge graph integration_: Incorporate structured knowledge to ground the model. 9. _Multi-task learning_: Train the model on multiple tasks to improve generalization. 10. _Continuous learning and updating_: Regularly update the model with new data and techniques. Additionally, consider using: 1. _Constrained generation_: Limit the model's output to a specific domain or format. 2. _Post-processing filters_: Apply filters to detect and correct hallucinations. 3. _Evaluation metrics_: Use metrics that reward factual accuracy and penalize hallucinations. By implementing these strategies, you can reduce hallucinations and improve the reliability of your LLM.

1

Want to perform hyperparameter tuning on your ML and DL algorithms? Learn about how you can do it with Optuna and neptune.ai

15