Sanyam Jain

🚀 **DAY 42: Stanford CS224N: NLP with Deep Learning** Today's lecture dives into syntactic structure and the intricacies of dependency parsing. 🔍 **Lecture 4: Syntactic Structure and Dependency Parsing** 1. **Syntactic Structure: Consistency and Dependency**   - Exploration of syntactic structures in natural language, focusing on constituency and dependency.   - Understanding how sentences are broken down into their syntactic components and the relationships between words. 2. **Dependency Grammar and Treebanks**   - Introduction to dependency grammar, where syntactic structure is represented as dependencies between words.   - Overview of treebanks, annotated datasets used to train and evaluate parsing models. 3. **Transition-Based Dependency Parsing**   - Explanation of transition-based parsing methods, which use a sequence of actions to construct dependency trees.   - Discussion on the efficiency and effectiveness of transition-based parsers. 4. **Neural Dependency Parsing**   - Introduction to neural network models used in dependency parsing.   - How these models leverage word embeddings and other features to improve parsing accuracy. 🛠 **Skills and Concepts Learned:** - **Syntactic Structure:** Gained insights into the syntactic structure of sentences, with a focus on constituency and dependency. - **Dependency Parsing:** Understanding the concepts of dependency grammar and how it is applied in natural language processing. - **Treebanks:** Learned about treebanks and their role in training syntactic parsers. - **Neural Models:** Introduction to neural models used in dependency parsing, highlighting their advantages in handling syntactic complexity. REF-https://lnkd.in/gfaRde3B This lecture is crucial for understanding how NLP models handle the syntactic aspects of language, which is foundational for tasks like parsing, translation, and more. 🌟📖 #NLP #DeepLearning #DependencyParsing #StanfordCS224N #MachineLearning

1

𝐔𝐍𝐞𝐭'𝐬 𝐎𝐫𝐢𝐠𝐢𝐧𝐚𝐥 𝐀𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞 𝐡𝐚𝐬 𝐭𝐰𝐨 𝐩𝐫𝐨𝐛𝐥𝐞𝐦𝐬.  This code solves the following two issues in the original paper. 1. It cannot handle odd middle/input dimensions of the features. 2. It doesn't output masks of the same size as input. I couldn't find the solution online, after enormous searching. Some places solve the problem 2, but not the problem 1. Hence, I decided to write my code. I present the solution with small perturbations to the original architecture in Github. Let me know if you have any suggestions or improvements. I am still learning how to code in a better way. haha. Follow Srijit Mukherjee for more. #statistics #deeplearning #machinelearning #datascience #dataanalytics https://lnkd.in/gFVYKtRP

16

1 Comment

Here's my blog on the evolution of attention mechanisms. From Bahdanau's additive attention (2014) to Luong's multiplicative attention (2015) and Vaswani's self-attention in Transformers (2017). Blog:- https://lnkd.in/guWW5Eyb

31

2 Comments

📝 A Curated Set of NLP Primers: Attention, Autoregressive vs. Autoencoder Models, Transformers, BERT 🔹 Attention: http://attention.aman.ai - Overview (The Attention Mechanism, The Bottleneck Problem, The Context Vector Bottleneck, How Attention Solves the Bottleneck Problem) - Origins of Attention - Attention: Under the Hood - Classic Sequence-to-Sequence Model - Sequence-to-Sequence Model with Attention - Context Vector - Compared: Attention Mechanism v/s Fixed-length Context Vector Approach - Extensions to the Classic Attention Mechanism - Self-Attention (Why Have Multiple Attention Layers?) - Comparative Analysis: Additive vs. Scaled Dot-Product Attention (Computational Efficiency, Theoretical Complexity, Usage and Performance, Implementation Details) - Multi-head Attention - Cross Attention - Ghost Attention - Linear Attention 🔹 Autoregressive vs. Autoencoder Models: http://autoregenc.aman.ai - How do Decoder Models Work? - How do Encoder Models Work? - Encoder-Decoder Models - XLNet: the Best of Both Worlds (Permutation Language Modeling) 🔹 Transformers: http://transformer.aman.ai - Mathematical background (Vectors, Matrix Multiplication, Dot Product, Masking, Sampling) - Attention (Additive/Multiplicative Attention, Self/Cross-Attention, Multi-Head Attention) - Core components of the Transformer Architecture (Embeddings, Positional Encoding, Skip Connections, Layer Normalization, Softmax) - Top-level Transformer Architecture (Encoder and Decoder stack) - Implementation details (Byte-Pair Encoding, Teacher Forcing, Label Smoothing) - Lessons learned (What are Transformers learning? Why is training them so hard?) - Pros/cons of Transformers relative to CNNs/RNNs - Relation between Transformers and Graph Neural Networks 🔹 BERT: http://bert.aman.ai - Background: Pre-Training - Contextualized Embeddings - Masked Language Modeling (MLM) - Next Sentence Prediction (NSP) - BERT’s Encoder Architecture vs. Other Decoder Architectures - The Strength of Bidirectionality - Supervised Finetuning #artificialintelligence #deeplearning #nlp

66

4 Comments

🚀 DAY 40: Stanford CS224N: NLP with Deep Learning** Lecture 2 - Neural Classifiers Today's lecture expands on our understanding of word vectors, introduces the GloVe model, and delves into neural network classifiers. 🔍 **Lecture Topics Covered:** 1. **Course Organization:**   - Brief overview of the course structure and today's agenda. 2. **Finish Looking at Word Vectors and Word2vec:**   - Complete the discussion on Word2vec and its applications.   - Detailed examples of how word vectors represent semantic relationships. 3. **Capturing the Essence of Word Meaning by Counting:**   - Discussion on whether counting word occurrences can effectively capture word meanings.   - Comparison between frequency-based methods and prediction-based methods like Word2vec. 4. **The GloVe Model of Word Vectors:**   - Introduction to the GloVe (Global Vectors for Word Representation) model.   - Explanation of how GloVe combines the advantages of count-based and prediction-based methods. 5. **Evaluating Word Vectors:**   - Methods for evaluating the quality of word vectors.   - Intrinsic evaluation (e.g., word similarity tasks) and extrinsic evaluation (e.g., performance on downstream tasks). 6. **Word Senses:**   - Discussion on polysemy and how words can have multiple senses.   - Methods for distinguishing between different word senses. 7. **Review of Classification and How Neural Nets Differ:**   - Overview of classification problems in machine learning.   - Key differences between traditional classifiers and neural network classifiers. 8. **Introducing Neural Networks:**   - Basic concepts of neural networks.   - Explanation of neurons, layers, activation functions, and how neural networks learn. 🛠 **Skills and Concepts Learned:** - **Word Vectors and GloVe:** Enhanced understanding of different word embedding techniques and their applications. - **Evaluation Methods:** Gained knowledge on how to evaluate word vectors effectively. - **Word Senses:** Learned about polysemy and techniques to handle multiple word senses. - **Neural Networks:** Acquired foundational knowledge of neural networks and their role in classification tasks. Excited to continue exploring the world of NLP and deep learning! 🌟📖 #NLP #DeepLearning #WordVectors #GloVe #NeuralNetworks #StanfordCS224N #MachineLearning

5

🚀 **DAY 48: Stanford CS224N: NLP with Deep Learning** Today's lecture dives into the intricacies of Transformers and the concept of model pretraining, which has become fundamental in modern NLP techniques. 🔍 **Lecture 10: Transformers and Pretraining** 1. **Quick Review of Transformer Model:**   - Recap of the Transformer architecture, focusing on its self-attention mechanism, multi-head attention, and encoder-decoder structure.   - Emphasized how Transformers handle sequence-to-sequence tasks efficiently by processing the entire sequence simultaneously rather than sequentially. 2. **Brief Note on Subword Modeling:**   - Discussed the importance of subword modeling, which allows models to handle out-of-vocabulary words and better represent the morphological structure of words.   - Techniques like Byte Pair Encoding (BPE) and WordPiece are commonly used in modern NLP models to split words into subwords, enabling more flexible and accurate representations. 3. **Motivating Model Pretraining from Word Embeddings:**   - Word embeddings like Word2Vec laid the groundwork for understanding word meaning through dense vector representations.   - Model pretraining extends this concept by training entire models (not just word embeddings) on large corpora before fine-tuning them on specific tasks, leading to better generalization and performance. 4. **Model Pretraining Three Ways:**   - **Decoders:** Pretraining models like GPT, wich use a unidirectional approach where the model predicts the next word in a sequence.   - **Encoders:** Pretraining models like BERT, which use a bidirectional approach, capturing context from both directions in the sequence.   - **Encoder-Decoders:** Pretraining models like T5, which combine the strengths of both encoders and decoders, making them versatile for various tasks. 5. **Very Large Models and In-Context Learning:**   - Explored the rise of very large models (e.g., GPT-3) that push the limits of what can be achieved with pretraining.   - In-context learning, where models perform tasks with minimal or no fine-tuning, leveraging the vast knowledge they have learned during pretraining. 🛠 **Skills and Concepts Learned:** - **Subword Modeling:** Understanding how subword techniques like BPE and WordPiece help models deal with out-of-vocabulary words and capture word morphology. - **Model Pretraining:** Gained insight into the different pretraining strategies (decoders, encoders, encoder-decoders) and their impact on NLP performance. - **Large-Scale Models:** Recognized the significance of very large models and their capabilities in in-context learning, which represent the cutting edge of NLP research. This lecture provides a deep dive into how pretraining and Transformers are revolutionizing NLP, setting the stage for more advanced techniques and applications. 🌟📖 #NLP #DeepLearning #Transformers #Pretraining #StanfordCS224N #MachineLearning

4

Gradients of Matrices: When we compute the gradient (or derivative) of an m×n matrix A with respect to a p×q matrix B, the resulting Jacobian would be a four-dimensional tensor J of size (m×n)×(p×q). Each entry Jijkl​ in this tensor is given by ∂Aij​​/ ∂Bkl​, which represents the partial derivative of the ij-th entry of A with respect to the kl-th entry of B. Vector-Space Isomorphism: Since matrices represent linear mappings, there exists a vector-space isomorphism (a linear, invertible mapping) between the space Rm×n of m×n matrices and the space Rmn of mn vectors. This means we can re-shape our matrices into vectors of lengths mn and pq, respectively. Working with Jacobian Matrices: In practical applications, it’s often desirable to re-shape the matrix into a vector and continue working with this Jacobian matrix. The chain rule boils down to simple matrix multiplication, whereas in the case of a Jacobian tensor, we need to pay more attention to what dimensions we need to sum out. #cs_internship #machine_learning #step3

6

1 Comment

TASK-2 🚀 Project Update: Customer Segmentation using K-Means Clustering I’m excited to share my recent project, where I implemented a K-means clustering algorithm to group customers of a retail store based on their purchase history. This project is a key step towards understanding customer behavior and enhancing personalized marketing strategies. This project involved analyzing customer data, including variables such as purchase frequency, average transaction value, and product categories, to identify distinct customer segments. By implementing the K-means algorithm, I was able to segment customers into meaningful clusters, which revealed valuable insights into different buying behaviors. These insights will enable the business to tailor its marketing strategies, optimize product offerings, and ultimately enhance customer satisfaction. This project not only demonstrated the power of clustering algorithms in uncovering hidden patterns but also paved the way for more personalized and effective customer engagement strategies. I'm excited about the potential applications of these insights in driving business growth and improving customer experiences. 🌐 Project Overview: The goal of this project was to segment customers into distinct groups based on their purchasing patterns. By identifying clusters of similar customers, businesses can tailor their marketing efforts, optimize product offerings, and improve customer satisfaction. #prodigyinfotech

8

#RecSys: Interesting work on Tackling the Cold Start Problem in Recommendation Systems. - Anima Singh Nikhil Mehta Trung Vu Xinyang Yi's paper on Better Generalization with Semantic IDs introduces the use of Semantic IDs instead of traditional random IDs for videos, items, and users. Unlike random IDs, which are unconnected by design, Semantic IDs are based on the semantic representation (embeddings) of an item's metadata (e.g., video metadata). This approach leads to better generalization, as similar items/videos are intrinsically linked, significantly improving cold start scenarios. - Jianling Wang Haokai Lu's paper on LLMs for User Interest Exploration showcases a hybrid approach combining LLMs and traditional collaborative filtering. LLMs are used for controlled generation of user interest clusters, leveraging their strength in reasoning and generalization, while collaborative filtering ensures strong personalization. By pre-computing these interest clusters, the approach offers low-latency recommendations and is especially useful for handling cold start cases. Live experiments have shown promising results. It is good example of how LLMs and traditional recommendation techniques are being used together in real-world use cases! Ed H. Chi Lexi Baugher Roger Cheng He Ma Panchapagesan Krishnamurthy Vijay Krishnan Sudarshan Sivaraman Phil Walsh #Turing

42

2 Comments

🚀 Unlocking the Power of Attention Mechanisms in Encoder-Decoder Models! 🔍✨ Have you ever wondered how translation apps like Google Translate deliver such accurate translations? The key lies in attention mechanisms that revolutionize the performance of encoder-decoder models! Let’s break it down: 🧠 What Are Encoder-Decoder Models? Imagine translating a sentence like “I am Shivasish” into French ("Je suis Shivasish"). Here's how it typically works: Encoder: Reads the entire input sentence and compresses it into a context vector—a summary that represents the input. Decoder: Uses this context vector to generate the output, one word at a time. The Catch? With longer or more complex sentences, a single fixed-length context vector often misses important details, leading to less accurate translations. 🔄 Enter the Attention Mechanism! Attention is like giving your model superpowers! Here's how it works: Dynamic Focus: Rather than using one static context vector, attention allows the model to “focus” dynamically on different parts of the input sentence. It identifies which words or phrases are most important at each step of the translation. Contextual Magic: At every decoding step, the model scores parts of the input sentence to determine relevance. This ensures it captures critical details and relationships between words, enhancing both meaning and accuracy. Enhanced Accuracy: By adapting its focus continuously, attention mechanisms lead to more fluent and contextually appropriate translations. 🌟 The Magic in Action Check out the attached image to see attention mechanisms in full swing! As the input sentence “I am Shivasish” gets translated to “Je suis Shivasish,” attention scores dynamically adjust to focus on relevant parts of the sentence. This results in a more nuanced and accurate translation! 📈 Why It Matters? Attention mechanisms are a breakthrough in Natural Language Processing (NLP), leading to models with significantly higher BLEU scores—meaning more human-like, fluent translations. Whether it’s understanding complex sentences or capturing subtle language nuances, attention ensures your model performs more like a human translator. 💡 Pro Tip: If you're working in NLP or building any model requiring sequence-to-sequence tasks (e.g., machine translation, speech recognition), integrating attention mechanisms is a must for boosting performance! . . #AI #MachineLearning #NLP #DeepLearning #Translation #AttentionMechanisms #EncoderDecoder #BLEU #ArtificialIntelligence #LanguageModels #Innovation

24

1 Comment

🚀 𝐌𝐞𝐦𝐨𝐫𝐲 𝐄𝐟𝐟𝐢𝐜𝐢𝐞𝐧𝐭 𝐕𝐞𝐜𝐭𝐨𝐫 𝐄𝐦𝐛𝐞𝐝𝐝𝐢𝐧𝐠𝐬 𝐟𝐨𝐫 𝐑𝐀𝐆 𝐀𝐩𝐩𝐥𝐢𝐜𝐚𝐭𝐢𝐨𝐧𝐬 𝐔𝐬𝐢𝐧𝐠 𝐔𝐈𝐍𝐓8 🔍 𝘌𝘹𝘱𝘭𝘰𝘳𝘪𝘯𝘨 𝘵𝘩𝘦 𝘍𝘶𝘵𝘶𝘳𝘦 𝘰𝘧 𝘕𝘢𝘵𝘶𝘳𝘢𝘭 𝘓𝘢𝘯𝘨𝘶𝘢𝘨𝘦 𝘗𝘳𝘰𝘤𝘦𝘴𝘴𝘪𝘯𝘨: In the world of natural language processing, vector embeddings play a crucial role in capturing semantic meaning. However, the high memory consumption of embedding models often hampers their scalability and practicality. Today, I'm excited to share insights from my recent article on a groundbreaking solution to this challenge: memory-efficient UINT8 vector embeddings. 𝐊𝐞𝐲 𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬: Memory Reduction: By leveraging the compactness of unsigned 8-bit integers (UINT8), we can significantly reduce the memory footprint of embeddings while maintaining their semantic integrity. This means more efficient use of computational resources and faster processing times. Practical Implementation: The article details how to implement these efficient embeddings in the Retrieval Augmented Generation (RAG) framework using the Qdrant vector store. This approach transforms RAG into a more agile and accessible solution across various domains. Real-World Applications: From large-scale question-answering to enhanced information retrieval, this innovation paves the way for streamlined operations and improved performance. Technical Deep Dive: Learn how to generate and upsert quantized embeddings into Qdrant, with step-by-step code examples and performance comparisons between float32 and uint8 embeddings. 𝐖𝐡𝐲 𝐓𝐡𝐢𝐬 𝐌𝐚𝐭𝐭𝐞𝐫𝐬: This advancement is not just about cutting costs or saving memory—it's about unlocking the potential of NLP applications at scale. Whether you're working in research, developing AI solutions, or looking to optimize your existing systems, memory-efficient UINT8 vector embeddings could be the game-changer you need. 🛠️ 𝑻𝒆𝒄𝒉𝒏𝒊𝒄𝒂𝒍 𝑯𝒊𝒈𝒉𝒍𝒊𝒈𝒉𝒕𝒔: Embedding Quantization: Convert high-dimensional float64 Cohere embeddings to uint8 embeddings, reducing dimensionality and memory usage. Qdrant Integration: Create collections and upsert data efficiently with native UINT8 support. I'm eager to see how this innovation will impact the NLP community and drive forward the capabilities of AI. Let's continue pushing the boundaries of what's possible! 🔗 𝐑𝐞𝐚𝐝 𝐭𝐡𝐞 𝐟𝐮𝐥𝐥 𝐚𝐫𝐭𝐢𝐜𝐥𝐞 𝐡𝐞𝐫𝐞:  https://lnkd.in/ghBKS489 #NLP #MachineLearning #ArtificialIntelligence #DataScience #MemoryEfficient #RAG #Embeddings #TechInnovation #Qdrant

70

14 Comments

Evaluation of RAG results is as important as RAG itself 💯 Let's understand 4 different ways to evaluate RAG results - But before that lets revise what is "Top k" is for the sake of simplicity - Top-k Sampling: Here, the LLM only considers the k most probable tokens(words) in its vocabulary for the next step. This focuses the generation process and reduces randomness. It's useful for tasks where you need specific terminology or want to maintain a consistent style. For example, using a low k value (k=3) might limit the LLM to choosing the top 3 most likely words for the next sentence, leading to more focused and controlled outputs 🎯 Now the 4 evaluation metrics .. 📌 Precision at k - ​What it measures: How good the system is at finding relevant documents among the top k results.   ​Example: If you search for “best italian restaurants” and out of the top 10 results, 7 are actually good Italian restaurants, the Precision at 10 is 7/10 = 70%   📌 Recall at k - ​What it measures: How good the system is at finding all the relevant documents within the top k results?   ​Example: Suppose there are 20 good Italian restaurants in total, and your search system found 8 of them in the top 10 results. The Recall at 10 (R@10) is 8/20 = 40%   📌 Mean Reciprocal Rank (MRR) - ​What it measures: How quickly the system finds the first relevant document.   ​Example: If you search for something and the first relevant result is at position 3, the reciprocal rank is 1/3 = 0.33   📌 Normalised Discounted Cumulative Gain (NDCG) - ​What it measures: How good the system is at ranking relevant documents higher in the results list, giving more credit to documents found earlier.   ​Example: • You give higher scores to relevant documents found at the top of the list. • If a relevant document is found at position 1, it gets a high score. • If it's found at position 10, it gets a lower score. • NDCG combines these scores to show how well the system ranks relevant documents overall, with higher NDCG values indicating better performance. Hope you have learnt something new from this! Be curious and Keep learning! Follow Bhavishya Pandit for more posts on GenAI! #python #ml #ai #datascience #genai #llms #artificialintelligence #machinelearning #generativeai #gpt4 #rag #ibm #google #amazon #openai

136

14 Comments

Post 1/2: General guidelines for your ML Technical interviews: Before Applying: Research the company and team in advance. Check out their blogs and the prospective team members. Try to align and highlight your skills and key projects that are aligned to the company or the team or the role you are applying to. For example: Let's say you have have completed a variety of ML projects in both computer vision and NLP. If you are applying for an LLM engineer role, you might want to highlight your courseworks, projects, etc. related to NLP. Try to highlight those libraries that you have used for such projects. While writing CVs, try to write concisely in 1-2 lines and use bullet points if possible, instead of writing long sentences and paragraphs. Don’t Inflate your skills and contributions: I have noticed that a few candidates mention their group projects, without having core contributions and passing it as their own, while their contributions are almost marginal. During interviews, when asked about such group projects and their details, the candidates bail out saying they don’t know because they were not working on that particular aspect and they had worked on other aspects. This may bring out a bad impression to your interviewer. Now, the question is should you include or not such projects in your CV? Personally, I think it is fine to add such projects, as long as you have good understanding and you are able to explain the underlying concepts and all the aspects of the projects, while clearly mentioning your contributions. Putting lots of jargon and keywords: Your resume might get noticed in the initial stages, when someone might be searching for potential candidates and the keywords in your resume might get you lucky. However, on the long run they might hurt The more keywords you add and highlight, there is a high chance the interviewer may assume that you have thoroughly understood or applied those concepts. Unless you are prepared to explain those concepts in detail along with their intuition and understanding, I would suggest not using unnecessary jargon. Continue reading in Post 2/2... https://lnkd.in/gQGjMGJ9 P.S. This post does not generalize the interview process and varies from interviewer to interviewer. Each organization may have their own interview policies and may have their own style of interview process. #machinelearning #interviews #interviewpreparation #interviewtips

60

2 Comments

A while back I had written a survey paper on knowledge graph embedding models. Finally, I decided to put it in arxiv. The survey paper covers from basics of KG to different embeddings methods to application in social media . Here is the link to survey paper. https://lnkd.in/gi6pP_ux

37

1 Comment

ML's foundation is maths. > From hypothesis creation to selecting proper priors > From choosing which loss function to use to which regularizer to use > From choosing the correct metric for performance to hyperparameter optimization > From optimization functions to explainability > From feature engineering to dimensionality reduction, > From MLE to MAP

121

2 Comments

🌟 Excited to Share My Latest Project: RAG Research Paper Analayzer 📄 Effortlessly Analyze Research Papers Key Features: 1️⃣ Upload PDFs and watch the app split content into searchable chunks & store embeddings in a vector database. 2️⃣ Get instant answers to your queries along with highlighted passages directly from the paper for maximum transparency. 3️⃣ Pinpoints the most relevant pages and sections based on your prompt, helping you focus on critical content. 4️⃣ Provides both the answer and supporting evidence directly from the source material, empowering you to uncover insights faster. 5️⃣ Powered by Retrieval-Augmented Generation (RAG), this tool turns academic literature into actionable insights with just a click. Technologies Behind the Magic: 1️⃣ LangChain: Integrates advanced workflows for querying and summarization. 2️⃣ Streamlit: Offers an interactive, user-friendly interface. 3️⃣ ChromaDB: Handles vector embeddings for efficient similarity search. 4️⃣ Custom Embedding API: Converts text into dense vector representations. 5️⃣ Docker: Ensures seamless deployment and integration into any workflow. 📦 Why I Built This: As a passionate AI developer, I wanted to create a tool that empowers researchers and students to easily extract meaningful insights from complex academic papers. This project combines cutting-edge technologies to deliver precision and convenience, all within a containerized environment. 🔗 Try It Out: Get the Docker Image: https://lnkd.in/dAN7J2TA Explore the Code on GitHub: https://lnkd.in/dtRqMXpw I’d love to hear your thoughts and feedback! 💡 #Docker #Streamlit #LangChain #ResearchTools #ArtificialIntelligence #NLP #AcademicResearch #ChromaDB #GenerativeAI

20

3 Comments

Retrieval-Augmented Generation for Large Language Models: A Survey 🔹 Challenges of LLMs: LLMs face issues like hallucinations, outdated knowledge, and non-transparent reasoning. 🔹 RAG as a Solution Retrieval-Augmented Generation (RAG) integrates external knowledge, improving accuracy and credibility, especially for knowledge-intensive tasks. 🔹 RAG Paradigms ◾ Naive RAG: Basic indexing, retrieval, and generation process, facing challenges with precision and recall. ◾ Advanced RAG: Improves indexing and retrieval with pre-retrieval and post-retrieval strategies, such as query rewriting and context compressing. ◾ Modular RAG: Introduces flexible modules (e.g., search, memory, predict) and new patterns (e.g., rewrite-retrieve-read), allowing dynamic adaptation. 🔹 Core Components of RAG: ◾ Retrieval: Utilizes unstructured, semi-structured, and structured data sources; employs fine and coarse retrieval granularity; optimizes indexing and queries. ◾ Generation: Focuses on context curation through reranking and compression; fine-tunes LLMs for specific tasks and scenarios. ◾ Augmentation: Employs iterative, recursive, and adaptive retrieval processes to enhance robustness and efficiency. 🔹 Evaluation of RAG: ◾ Targets: Assesses retrieval quality and generation quality. ◾ Metrics: Includes context relevance, faithfulness, answer relevance, noise robustness, negative rejection, information integration, and counterfactual robustness. ◾ Tools and Benchmarks: Uses RGB, RECALL, RAGAS, ARES, TruLens, CRUD, and RAGQuestEval for evaluation. Reference : https://lnkd.in/g5NF3mmE #LLM #GenAI #NLP #AI #datascience #machinelearning #data

7

2 Comments

Turbo Sparse: Achieving LLM SOTA Performance with Minimal Activated Parameters Exploiting activation sparsity helps accelerate inference process of large language models (LLMs) without compromising performance. However, activation sparsity is determined by activation functions, and commonly used ones like SwiGLU and GeGLU exhibit limited sparsity. To address these challenges, this paper proposed a novel dReLU function, which is designed to improve LLM activation sparsity, along with a high-quality training data mixture ratio to facilitate effective sparsification. 𝗞𝗲𝘆 𝗰𝗼𝗻𝘁𝗿𝗶𝗯𝘂𝘁𝗶𝗼𝗻𝘀: - Efficient dReLU activation function: proposed method utilizes fewer than 150B tokens, representing less than 1% of typical pretraining tokens (commonly 15T tokens). - Sparse activated models: released sparsely-activated TurboSparse-Mistral7B and TurboSparse-Mixtral-47B models, which shows better performance compared to original versions - Practical inference speedup: Evaluation shows that with released models, 2-5× speedup can be achieved, with up to 10 tokens/s even without a GPU on TurboSparse-Mixtral-47B. 𝗔𝗻𝗮𝗹𝘆𝘀𝗶𝘀 i) Limitations of existing ReLUfication - sparsity evaluation of ReLULlama-7B and original Llama-2-7B, shows that existing ReLUfication methods can only improve the sparsity from 40% to 67% - This can be attributed to the ReLU activation function applied after gate weight, which masks all negative activations to zero ii) dReLU - newly proposed activation function, where ReLU is applied after both the up- and gate-projection - for an experiment comparing models using dReLU and SwiGLU, results shows that dReLU structure exhibit similar convergence compared to those using SwiGLU -further evaluation results for various sparsity levels demonstrate dReLU activation function enables high sparsity without significant degradation in performance, maintaining competitive performance even at 90% sparsity - In contrast, the SwiGLU-based model experiences a severe increase in perplexity as sparsity reaches 80%. 𝗘𝘅𝗽𝗲𝗿𝗶𝗺𝗲𝗻𝘁𝗮𝗹 𝗥𝗲𝘀𝘂𝗹𝘁𝘀 - TurboSparse-Mistral-7B outperforms Gemma-2B by far, while only activating 3B parameters. - TurboSparse-Mixtral-47B outperforms the original Mixtral-47B with only 4.5B parameters activated - for activations with a value of zero, TurboSparse-Mistral-7B, on average, has 90% of the neurons inactive in each layer, and TurboSparse-Mixtral-47B with 85% on average for each expert FFN 𝗜𝗻𝗳𝗲𝗿𝗲𝗻𝗰𝗲 𝗦𝗽𝗲𝗲𝗱𝘂𝗽 𝗘𝘃𝗮𝗹𝘂𝗮𝘁𝗶𝗼𝗻 - On Pure CPU Inference, ReLUfied models achieve 2.08-2.28× speedup over original model. - On Hybrid GPU-CPU Inference, demonstrate significant speedups ranging from 2.52 to 4.64× compared to the baseline llama.cpp - For Deploying LLMs on mobile phones, achieves a 22.2× speedup using TurboSparse-Mixtral-47B inference compared to llama.cpp with original Mixtral-47B 𝗣𝗮𝗽𝗲𝗿: https://lnkd.in/eeHCWhUt 𝗠𝗼𝗱𝗲𝗹𝘀: https://lnkd.in/etHpZUju

24

1 Comment