Edwin Lungatso’s Post

Exploring the dynamic world of Retriever-Augmented Generation (RAG)! RAG combines the power of language models with a retrieval system to generate responses based on a vast array of information. - Inspired by a video tutorial on building a RAG application using Langchain and OpenAI, I chose to use an extended dataset close to my heart - the timeless classic "Alice in Wonderland”. - Utilizing OpenAI’s embeddings function and ChromaDB, I transformed these text chunks into a vector database. In the screenshot below you can see that it returns the 3 most relevant chunks in the text that it thought best match the questions asked (query) and the LLM model will use these chunks to provide the final response along with the sources. https://lnkd.in/gtjv6rye #rag #llm #generatieveai #openai #llm #nlp #deeplearning #conversationalai

Hello LinkedIn people... I am excited to share my experience on the project "Summarizing Text with Ease using Hugging Face!" I recently explored the power of text summarization using Hugging Face's transformers library, and I'm blown away by the results! With just a few lines of code, I was able to: - Load text data - Preprocess text - Create a summarization pipeline - Get accurate summaries! The model was able to condense long pieces of text into concise, meaningful summaries, saving me hours of time and effort. The potential applications are vast - from news article summaries to automated report generation, and even chatbots! #HuggingFace #TextSummarization #NLP #AI #MachineLearning #Saisatish #AIMERsociety

𝐎𝐩𝐭𝐢𝐦𝐢𝐳𝐢𝐧𝐠 𝐈𝐧𝐟𝐨𝐫𝐦𝐚𝐭𝐢𝐨𝐧 𝐑𝐞𝐭𝐫𝐢𝐞𝐯𝐚𝐥 𝐟𝐨𝐫 𝐁𝐞𝐭𝐭𝐞𝐫 𝐈𝐧𝐬𝐢𝐠𝐡𝐭𝐬 We aim to move beyond the fundamentals of Retrieval-Augmented Generation (RAG) and explore what an advanced RAG system truly entails. Going beyond the 101-level understanding means integrating the following steps: 1️⃣ Query: We start by enhancing the user’s query, sharpening it to ensure we get more precise and useful results. 2️⃣ Retrieval: Then, we dive into the search for relevant data. 3️⃣ Re-ranking: It's not enough to just find relevant content. We re-rank it, bringing the most valuable insights to the top. 4️⃣ Generation: Using these prioritized results, our language model generates a comprehensive response. 5️⃣ Response Validation: Before delivering the answer, we verify it. Does it make sense? Is it accurate and aligned with the original question? 6️⃣ Response: Once validated, the solution is ready to be shared with the user. #AI #MachineLearning #InformationRetrieval #NLP #DataInsights #LLM #RAG

I developed a Movie Genre Classification model that enables users to accurately predict the genre of a movie based on its plot summary or other textual information. The model leverages techniques such as TF-IDF or word embeddings for feature extraction, paired with classifiers like Naive Bayes, Logistic Regression, and Support Vector Machines. Users can input the movie plot, and the model predicts the genre based on the learned patterns from the training data. The program continues refining predictions with real-time input, making it an intuitive tool for classifying movies. #MachineLearning #MovieGenreClassification #NLP #AI CodSoft #codsoft

𝐎𝐩𝐭𝐢𝐦𝐢𝐳𝐢𝐧𝐠 𝐈𝐧𝐟𝐨𝐫𝐦𝐚𝐭𝐢𝐨𝐧 𝐑𝐞𝐭𝐫𝐢𝐞𝐯𝐚𝐥 𝐟𝐨𝐫 𝐁𝐞𝐭𝐭𝐞𝐫 𝐈𝐧𝐬𝐢𝐠𝐡𝐭𝐬 We aim to move beyond the fundamentals of Retrieval-Augmented Generation (RAG) and explore what an advanced RAG system truly entails. Going beyond the 101-level understanding means integrating the following steps: 1️⃣ Query: We start by enhancing the user’s query, sharpening it to ensure we get more precise and useful results. 2️⃣ Retrieval: Then, we dive into the search for relevant data. 3️⃣ Re-ranking: It's not enough to just find relevant content. We re-rank it, bringing the most valuable insights to the top. 4️⃣ Generation: Using these prioritized results, our language model generates a comprehensive response. 5️⃣ Response Validation: Before delivering the answer, we verify it. Does it make sense? Is it accurate and aligned with the original question? 6️⃣ Response: Once validated, the solution is ready to be shared with the user. #AI #MachineLearning #InformationRetrieval #NLP #DataInsights #LLM #RAG

Getting Started with LangChain Tools and Agents! I recently explored LangChain’s built-in tools and created my own custom tool to enhance the quality of the questions being asked. ✨ Here's how I did it: Utilized built-in tools like Arxiv, Wikipedia, and DuckDuckGo for seamless search and retrieval. Built a custom tool on top of my retriever in a RAG system I developed. Combined everything to create a powerful agent using Llama3 as my LLM. Integrated HuggingFace embeddings and Chroma as my vector database for efficient searches. #LangChain #AI #MachineLearning #LLM #NLP #CustomTools #HuggingFace

I recently started few projects with Hugging Face, and I’ve been blown away by the possibilities their open-source models unlock! 🌐💡 Exploring their library, I quickly realized how many cutting-edge solutions we can create with minimal effort thanks to their Transformers and Pipelines. Tasks like text summarization and translation are now simpler than ever — requiring only a few lines of code to implement. Here’s a notebook I put together showcasing these capabilities. It’s amazing how we can build powerful tools ourselves using open-source models! Big thanks to the amazing contributors at Hugging Face 🙌. Check it out below and start creating! 👇 #MachineLearning #NLP #OpenSource #AI #HuggingFace #Transformers #TextTranslation #TextSummerization

Scikit-LLM, an integrating LLMs like GPT directly into traditional scikit-learn workflows! 🤖 Whether you're working on text classification, sentiment analysis, or just looking to harness the power of LLMs without disrupting your existing pipelines, Scikit-LLM makes it simple and efficient. In this article, I dive deep into: The design and architecture of Scikit-LLM How it integrates seamlessly with scikit-learn Practical examples with code to get you started quickly! Check out the full article here and discover how you can elevate your ML workflows with LLMs. https://lnkd.in/dfE3-x-T #AI #MachineLearning #NLP #DataScience #LLM #ScikitLearn #GPT #Innovation

𝐀 𝐒𝐢𝐦𝐩𝐥𝐞 𝐑𝐞𝐭𝐫𝐢𝐞𝐯𝐚𝐥-𝐀𝐮𝐠𝐦𝐞𝐧𝐭𝐞𝐝 𝐆𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐨𝐧 𝐌𝐨𝐝𝐞𝐥. A basic retrieval-augmented generation model begins with a trigger, such as a user query or instruction. This trigger is sent to a retrieval function, which fetches relevant content based on the query. The retrieved content is then merged back into the context window of the LLM, along with the input prompt and the query itself. Care is taken to leave enough space for the model's response. Finally, the LLM generates an output based on the combined input and retrieved content. This simple yet effective approach often yields impressive results, demonstrating the value of grounding in practical applications. #LLM #LanguageModel #AI #ArtificialIntelligence #MachineLearning #DeepLearning #NaturalLanguageProcessing #NLP

📜 On short of “Tree of Thoughts: Deliberate Problem Solving with Large Language Models” 📎 Keywords: LLMs, prompt engineering, AI reasonings, search algorithms. Language models are increasingly used for problem-solving, but they often struggle with tasks requiring exploration or strategic lookahead. A new framework called “Tree of Thoughts” (ToT) is introduced to improve these models. ToT allows deliberate decision-making, considering multiple reasoning paths and self-evaluating choices. Experiments show ToT significantly enhances language models’ problem-solving abilities on tasks like Game of 24, Creative Writing, and Mini Crosswords, with a 74% success rate. As a part of “📜 On short of…”, today’s journey is to gain a crisp and clear understanding of the Tree of Thoughts. During the voyage, we will look at the underlying notion of ToT to address important issues such as: What is ToT and why do we need it? Following that, we test our knowledge by doing an experiment using the ToT schema. The content of this writing is structured as follows: 📍Tree of Thoughts: we capture several key points of this architecture. 📍Code Implementation: we conduct an experiment that our prompting technique follows the ToT schema. 📍Conclusion: we review what we have done during our journey, also, we discuss about advantages and disadvantages of the technique. Let’s dive in! #ai #nlp #llm #openai #gpt4 #prompting #treeofthoughts #tot #deeplearning #machinelearning #searchalgorithms