Anish Nitin Somaiah

Understanding Conversation Summary Memory vs Conversation Summary Buffer Memory in LangChain For detailed sample code, check out the Google Colab notebook(https://lnkd.in/gRsarpNX) In the evolving field of conversational AI, memory mechanisms play a critical role in maintaining coherent and contextually relevant interactions. Two key approaches, Conversation Summary Memory and Conversation Summary Buffer Memory, offer distinct ways of handling conversational context. While Conversation Summary Memory focuses on summarizing entire dialogues, making it ideal for long conversations where historical context is essential, Conversation Summary Buffer Memory retains a rolling window of recent exchanges, suited for short, focused conversations. Understanding the differences between these memory types and their appropriate use cases can significantly improve chatbot performance. For a deeper dive into how these memory systems function and their practical applications using LangChain, you can explore the full pdf article below. #ConversationalAI #AI #LangChain #Chatbots #ArtificialIntelligence #MemoryMechanisms #MachineLearning #NLP #OpenAI #AIChatbot #Technology

92

🌟 How to get started with GenerativeAi (RoadMap for Beginners) 🌟 💡Here’s a 5 Week RoadMap for anyone who would like to start their Generative AI Journey 1️⃣ Week 1: LLM Basics and Foundation Introduction to Large Language Models, covering their architecture, training techniques, and real-world applications. 📕Course: - Transformer course by HuggingFace - Introduction to GenerativeAi by DataCamp 2️⃣ Week 1-2: Prompt Engineering Exploration of Large Language Models, covering core concepts and advanced techniques for effective prompt design and optimization 📕Course: - ChatGPT PromptEngineering for Developers by DeepLearning.ai - Prompt Engineering with Llama2 by DeepLearning.ai 3️⃣ Week 2-3: Building LLM Application Developing powerful AI applications using Large Language Models and the versatile LangChain framework. 📕Course: - LangChain chat with your data by DeepLearning.ai - LangChain for LLM Application Development by Deeplearning.ai - Building Applications with Vector Databases by DeepLearningAI 4️⃣ Week 3-4: RAG and FineTuning Master the art of enhancing LLM performance through Retrieval-Augmented Generation and custom model fine-tuning for specialized tasks. 📕Course: - Augment your LLM using RAG by Nvidia - Knowledge Graphs for RAG by DeepLearning.ai - FineTuning LLM by DeepLearning.ai 5️⃣ Week 4-5: Evaluation & Implementation Putting your LLM model into production 📕Course: - LLM Ops by Deeplearning.ai - Building and Evaluating AdvancedRAG Applications by DeepLearning.ai (Link to courses in 1st comment) 🤔 The best way to update is to create real-life applications So here’s a Github repository containing GenerativeAI project :- (Link in 2nd comment) #GenAI #GenerativeAi #llm #chatgpt #learngenerativeai #generativeairoadmap #generativeaiguide #learngenai #ChatGPT  #chatbots #chatgpt4 #chatgpt3 #GenerativeAI  #ArtificialIntelligence #AI #MachineLearning #ml #AIroadmap #DataScience

31

4 Comments

Want to perform hyperparameter tuning on your ML and DL algorithms? Learn about how you can do it with Optuna and neptune.ai

15

Excited to share our interactive BlogApp on Generalized Linear Models (GLMs)! In collaboration with Venkatachalam Subramanian Periya Subbu, Satya Lasya N, and Pooja Baralu Umesh, we've developed an intuitive tool that makes understanding GLMs both engaging and accessible. If you are an aspiring Data Scientist, Actuary, Statistician, Economist, Epidemiologists or an Engineer, you definitely need to check it out at: https://lnkd.in/gMqf3M5Y View the source code: https://lnkd.in/g9HUm_7r Special thanks to Professor Cody Carroll for the guidance throughout this project. Let's discuss GLMs! What's your experience with these powerful statistical tools? Share your thoughts below. #DataScience #Statistics #MachineLearning #GLM #Analytics #Programming #USF #MSDS

34

2 Comments

Question : Why does an AI product manager need to monitor #AI model❓ Once AI model is developed and deployed, the job doesnt end there. Model encodes pattern of the data it has seen during training phase. There may be several patterns or edge cases which are not part of training data. If model encounters any such patterns in testing or production phase, it will produce results which wont make sense without giving any warning about it. 👉 Data drift is a common challenge faced by AI-powered products. Over time, the distribution of input data can shift, leading to a mismatch between the model's training data and the real-world data it encounters. This can result in degraded model performance, unexpected outputs, and even complete model failure. 👉 The consequences of data drift can be severe, ranging from suboptimal user experiences to regulatory compliance issues. Product managers must be vigilant in detecting and addressing these challenges to ensure the continued success of their AI-powered offerings. 👉 Sudden changes in metrics can indicate input data drift. Additionally, monitoring the distribution of model outputs can reveal output drift, which may signal a need to retrain or fine-tune the model. Detecting data drift early is crucial. Product managers can leverage techniques such as A/B testing, anomaly detection, and continuous monitoring to identify and address these issues. Once detected, the product team can take appropriate actions, such as retraining the model, adjusting the data pipeline, or even considering a model architecture change. 💡 Quick Tip: Integrate model monitoring into your product roadmap and allocate dedicated resources to ensure the long-term success of your AI-powered offerings. Establish a comprehensive model monitoring dashboard to gain real-time visibility into the health and performance of your AI models. #artificialintelligence #productmanagement #datascience #productmanager

5

Thrilled to share that 2️⃣ / 2️⃣ of our works have been accepted EMNLP'24. 1️⃣. CorrSynth - A Correlated Sampling Method for Diverse dataset Generation from LLMs. (accepted at EMNLP'24 Main) The work is primarily lead by Abhishek Divekar and Suhas S Kowshik where we introduce a novel method to perform correlated sampling from LLMs to improve the distillation performance of student models. Kudos to both of them for their outstanding work! 2️⃣. PEARL: Preference Extraction with Exemplar Augmentation and Retrieval with LLM Agents (accepted at EMNLP'24 Industry Track) In this work, we propose a multi-agent framework leveraging the collaboration between large language models for providing product recommendations to customers in a conversational setting. Big thanks to Akshay Jagatap, Vinayak Puranik and Anirban Majumder for their immensely valuable contributions! Looking forward to presenting our works in person at the conference. See you in Miami, Florida 🌊 🛫 #EMNLP #ML #Research #LLM #NLP

96

Many of ML practitioners might have heard the term "Extending context length" for LLMs. In case if you want to dive deeper into this topic along with its code implementation, give this blog a read. I have described about Position Interpolation, NTK aware and Dynamic NTK method, which are widely used for extending context length.

68

Check out this highly recommended course for those interested in optimizing their models: - Introduction to Quantization - Symmetric and Asymmetric Quantization Modes - Hands-on Implementation from Scratch using PyTorch - covering Per Tensor, Per Channel (Axis), and Per Group techniques - Weights Packing: Exploring 2 or 4 Bits Quantization Methods https://lnkd.in/gMAuWXHm

43

Question : Sampling Techniques : What are they❓ As we saw recently in exit polls on Indian elections, choosing samples from a population can have a significant impact on predictions. #AI enthusiasts need to have a solid understanding of statistical sampling techniques. Statistical sampling is essential for AI product managers because it helps them: 1️⃣ Understand the Target Population: Accurate sampling ensures that the data used to train and evaluate AI models is representative of the target population, reducing the risk of biased or skewed results. 2️⃣ Optimize Data Collection: Sampling techniques can help AI product managers determine the appropriate sample size and sampling method, ensuring efficient and cost-effective data collection. 3️⃣ Validate Model Performance: Sampling is crucial for testing and validating AI models, as it allows for the creation of representative test sets that accurately reflect the real-world scenarios the model will encounter. AI enthusiasts should be aware of the different types of sampling methods, including: 🔺 Probabilistic Sampling Techniques: methods where randomisation is used instead of deliberate choice. 👉 Simple Random Sampling: Each member of the population has an equal chance of being selected. 👉Stratified Sampling: The population is divided into subgroups (strata), and samples are drawn from each stratum. 👉Cluster Sampling: The population is divided into clusters, and a random sample of clusters is selected. 👉Systematic Sampling: Every nth member of the population is selected, starting from a random point. 🔺 Non-Probabilistic Sampling Techniques: methods where some selection criteria exists 👉Convenience Sampling: Selecting participants based on their availability and accessibility. 👉Quota Sampling: Selecting participants to match predetermined proportions in the population. 👉Purposive Sampling: Selecting participants based on specific characteristics or criteria. 👉Snowball Sampling: Selecting participants through referrals from initial participants. The choice of sampling method depends on the specific goals of the AI project, the available resources, and the characteristics of the target population. Choosing the wrong sampling technique can lead to biased results, skewed data, and ultimately, the development of AI models that fail to perform well in real-world scenarios. 💡 Quick Tip: When selecting a sampling technique, consider the diversity and representativeness of the target population, the level of precision required, and the resources available for data collection.  #AI #ArtificialIntelligence #productmanagement #productmanager #DataScience 

7

🌟 **Decoding Vector vs. Graph Databases for RAG & RAFT – Which One Wins?** When it comes to enhancing large language models through Retrieval Augmented Generation (RAG) or Retrieval Augmented Fine-tuning (RAFT), the choice of database—vector or graph—plays a pivotal role. 🔹**Vector Databases:** Data is partitioned into chunks and each chunk is transformed into a vector using a language model. These vectors are then indexed in a vector database, facilitating quick nearest neighbor searches to find the most relevant information chunks in response to a query. 🔸**Graph Database:** This database shines in storing relationships between entities. By parsing and indexing these connections, graph databases excel at constructing detailed subgraphs for specific queries, ensuring that only the most relevant, contextually connected information is retrieved. It’s about understanding the network of relationships.An LLM is good at extracting the below kind of triplet information. [Entity A] -> [Relationship/Context] -> [Entity B] Both have their strengths—vector databases are great for fuzzy matching large data sets quickly, while graph databases excel in precise context and relationship extraction. 🚀 **The cutting edge? Combining both!** : This hybrid model takes advantage of the quick searching capability of vector databases with the deep relational insights of graph databases. For RAG or RAFT, this means not only retrieving the most relevant information quickly but also understanding its context within a broader knowledge base. For example, if a model is answering a query about "healthy eating," a vector database can quickly fetch documents mentioning health and nutrition. A graph database then elaborates on how these concepts connect with diet plans, medical advice, and recipes, providing a comprehensive, context-rich response. #GraphRAG #RAG #RAFT #GenerativeAI #GraphDB #VectorDB

13

Text chunks in retrieval may risk losing context. The below article introduces contextualizing each chunk using an LLM. The added cost and latency required for contextualizing is partially addressed through a lightweight LLM (haiku) and prompt caching. Along with other known techniques (BM25, reranking, etc.), a significant reduction in retrieval failure is achieved. Overall, a good article to read. https://lnkd.in/g_5fgAN8

10

It’s been a while since I have shared my experiences of working on AI projects using LLM’s that I have worked on in last 2 years. Hope these experiences will help folks. 😊 so here is the first one   Language Classification with GPT – We were working on a problem scenario where 2 languages to be classified in separate classes were very much similar in terms of words available in them and up to a good extent semantically also, but the languages were actually different. As the languages were domain specific, it was expected that we have to empower GPT. We used different models starting with GPT 3.5, GPT4 Turbo 0125, and then GPT 4 Omni and at last GPT-4 Turbo 2024-04-09. Of all the models we got the best accuracy with the last one. The last 2 are the latest ones available and as per Microsoft recommendation the go-to GPT models. However with GPT-4 Turbo 2024-04-09 also we didn’t got the desired result by the same pipeline we were using (words filter + vector search + RAG). While searching for the method to resolve the issue, I came across a research paper around ‘Feedback loop’ and how it can improve the results. (Sorry didn’t remember the paper name now , will try to find and post that).  We applied the same with some experimentations and it worked. So the focus here is – if you have a use cases where GPT/LLM’s is getting confused in language classification after application of latest techniques like vector search and RAG, feedback loop can be introduced after the final output as a last step. It can enhance the accuracy in very confusing but important cases. Hope this will help community. 😊 #NLP #LLM #GPT #RAG #DataScience #AI #OpenAI #Chatgpt OpenAI

40

2 Comments

Good folks at Google and Harvard University have introduced FRAMES (Factuality, Retrieval, and Reasoning MEasurement Set), a comprehensive evaluation dataset designed to test RAG capabilities across factuality, retrieval accuracy, and reasoning. • 824 challenging multi-hop questions requiring integration of info from 2-15 Wikipedia articles • Tests 5 reasoning types: numerical, tabular, multiple constraints, temporal, and post-processing • Rigorous quality checks, including temporal disambiguation and large output space • Baseline results with state-of-the-art LLMs like Gemini-Pro-1.5 and Gemma2 models • Novel multi-step retrieval pipeline using BM25 scoring • Best performance: 0.66 accuracy with (k,n,n_docs) = (5,5,10) configuration • Significant improvement over single-step 0.40 accuracy They discuss a unified framework for evaluating end-to-end RAG performance, addressing limitations in existing benchmarks that test components in isolation. Excited to see how this advances the development of more robust RAG systems! What are your thoughts on comprehensive RAG evaluation?

18

Data Scientists, This is BIG! Snowflake's Arctic LLM is Here – and Open-Source! 🤯 Get your hands on it now: Interactive Demo: Explore Arctic's capabilities with this Streamlit app → https://lnkd.in/gJ7y38Ng Replicate: Ready to experiment? Find it on Replicate → https://lnkd.in/gyDXiMju Why This Matters: Arctic lets you quickly build demos of your models. This is a game-changer for showcasing your work and making your models more accessible! Huge thanks to Snowflake for making this happen! Are you excited to try Arctic? Let me know in the comments! 👇 #LLM #OpenSource #DataScience #Streamlit #Snowflake

8

🚀 Bengaluru-based Conversational AI Startup Gnani AI Raises $3M 💰 Gnani AI, India's first voice-first small language model (SLM) for vernacular languages, has secured $3M in funding from tech holding company Info Edge. The investment will fuel the startup's expansion plans. Key Highlights: • Gnani AI is building a series of voice-based SLMs, trained on millions of hours of proprietary audio datasets and billions of conversations in 12 Indian languages • The startup has over 100 customers in banking, financial services, insurance, telecom, automotive, and healthcare industries • Gnani AI's models run on the edge, reducing costs and enabling quick fine-tuning based on enterprise needs • The startup specializes in creating customized models for individual companies, adding real value to their data CEO Ganesh Gopalan emphasizes the importance of multimodal models, focusing on voice and text fusion. By taking their models to enterprises and helping them build on top of it, Gnani AI is revolutionizing the conversational AI landscape in India. #ConversationalAI #VoiceFirst #IndianLanguages #Funding #Expansion

12

1 Comment

One Reason Why the Bigger Picture is Important when Modelling ECL Some context ... ECL has many moving parts, which makes it challenging (and also what I love about it). You have PD, EAD, LGD, SICR, FLI, term structures, overlays, MGL, and whatnot. Here's the thing ... modellers are model-focused. They want to build good models that pass aaaallll the validation tests. This creates a tendency to focus on each component in isolation ... and creates a risk that we celebrate a model before we know its impact on the ECL. However ... at the end of the day, the movement in ECL is all that matters. If it goes up, a negative asset on the balance sheet increases, and the income statement decreases. And vice versa. (Note that I'm not an accountant 😅.) The point I'm trying to make is this: don't go into meetings showing only how great the models are. Translate these models into ECL impacts and explain them simply. But this is easier said than done (for many reasons).

57

5 Comments

🚀 What Happens to a Contextual Embeddings Model if a Word is Not Present in the Vocabulary? 🌟 In contextual embeddings models like BERT, GPT-3, etc., handling out-of-vocabulary (OOV) words is crucial for maintaining performance and accuracy. These models typically address OOV words by breaking them down into subword units using techniques like WordPiece or Byte-Pair Encoding (BPE). This means that even if the entire word isn't present in the vocabulary, its parts likely are. Here's how it works: 1. Subword Tokenization: When a word is not in the vocabulary, it is split into smaller subword units. For example, the word "newword" might be broken down into "new" and "word", or even into smaller units like "ne", "##w", "##word" in the case of BERT. This approach ensures that the model can still generate embeddings for parts of the word, capturing some of its meaning and allowing it to be processed effectively. 2. Retaining Semantic Information: By breaking down OOV words into known subwords, the model can retain and leverage semantic information from the subwords. This allows the model to handle a wide range of words, even those it hasn't seen during training. For instance, the subwords "new" and "word" in "newword" carry meaningful information that the model can use to infer the context and meaning of the OOV word. 3. Handling Complex Words and Morphology: Subword tokenization is particularly useful for handling complex words and morphological variations. It reduces the number of OOV words and improves the model's ability to generalize across different forms of words. For example, the word "unhappiness" might be broken down into "un", "##happi", and "##ness", allowing the model to understand its structure and meaning more effectively. 4. Contextual Understanding: Contextual embedding models like BERT and GPT-3 are pre-trained on large corpora, which helps them understand and generate embeddings for subwords based on their context. This allows them to handle OOV words dynamically and accurately. Even if a word is rare or newly coined, the model can use its context to generate meaningful embeddings and produce relevant responses. By leveraging subword tokenization techniques, contextual embeddings models can effectively manage OOV words, ensuring robust performance and accurate language understanding. Stay tuned for more insights on NLP and AI advancements! 💡 #NLP #AI #MachineLearning #DataScience #LanguageProcessing #TechInnovation #BERT #GPT3 Please follow my posts for more updates: Tejas Nanaware on LinkedIn #machinelearning #deeplearning #artificialintelligence #datascience #statistics Feel free to connect and share your experiences or ask any questions!

5

Ran a quick test on the new #OpenAI #o1mini to solve IIT-JEE Advanced 2024 maths questions. It accurately solved 13 out of 17 questions with an impressive accuracy of ~76%. Quick thoughts: 1. The hidden chain of thought is quite detailed and extensive. Breaks the problem, solves and verifies and iterates multiple times. This might increase the output token cost by a big margin. 2. Seems there are tools integrated for calculations. Much needed as #gpt4o does a bad job at it. 3. Latency is quite high. Some questions took more than 50 seconds to get answered. This needs to come down for production user-facing use cases. 4. Hallucination still exists. Check the 1st question in the link, it had the correct step-by-step reasoning but hallucinated at generating the final answer. 5. Overdoing self-thought. Saw a couple of instances where it did generate correct answers in the process but eventually made wrong answers. Would give a clearer picture once the rate limits are increased and benchmarking is done on a bigger sample. Let me know any interesting findings and thoughts 😀 https://lnkd.in/giSCezyG #GenAI #OpenAI #PhysicsWallah

138

5 Comments

Hello all, I and my friend Aditya Sahu have been working lately on Automatic Redaction / Replacement tool for PDFs (scanned PDF & text-readable) and for Images which uses large language models (LLMs) in the backend. Different Formats / templates of PDFs cause a major problem in the extraction process. *What it does: 1. It removes the editing panel for your redaction / replacement work and instead you can just explain what you need to redact / replace in normal keywords or plain sentences in a comma separated fashion. 2. It uses LLMs in the background - you can use your own privately deployed model like llama 3 70b fine-tuned version or you can also use models like GPT-4o, Claude etc. which are commercially available. It can be configured based on your requirement. 3. It uses OCR for Image / Scanned Image case, which can be configured to any Cloud Server OCRs as well as open-source OCRs. 4. Can ingest multiple images and PDFs and work sync. 5. You can run multiple iterations on the same file to make many small small changes that can be possible in the PDF. Note: The process is not fully 100% accurate but it is well in range of 85-90+ , the accuracy can vary based on the choice of LLM model with good context length and choice of OCR as well as the quality of image. [*imp] *PDFs are quite complex for handling the formatting issues, so this tool does have limitations on what length of words can be replaced because it might lead to formatting issues in the provided PDF. There would be HIL [human in the loop] for evaluation. You can go through the Demo once; it has 2 parts in it - one for scanned PDF and one for text readable pdf (built using Python) 😀. Right now, it is just a Streamlit demo but parallelly in the backend work is going on... Please DM to me Mohit Singh Pawar or Aditya Sahu for more details or clarification related to its availability. Audio credits: ElevenLabs free credits. I guess will have to upload part 1 here and part 2 in a separate post. part 2 : https://lnkd.in/gEi-DUzb #redact #llm #replace #pdf #genai #generativeai

56

5 Comments