Komal Khetlani’s Post

Anchoring bias describes the common human tendency to rely too heavily on the first piece of information offered (the "anchor") when making decisions. In decision-making, anchoring occurs when individuals use an initial piece of information to make subsequent judgments. In AI, anchoring bias can happen in various ways, primarily through the data used to train machine learning models and the initial parameters set by the developers. When setting up AI models, the initial parameters and assumptions can act as anchors. Example: In a recommendation algorithm, if the initial algorithm overemphasizes certain user behaviors, it might create a feedback loop where the user is only recommended similar items, reinforcing initial consumption patterns. #DataScientist #MachineLearning #NLP #LLM #DeepLearning #DataScience #Statistics #AI #LanguageAI #GenerativeAI #AIforHealth #ResponsibleAI #BiomedicalResearch #BiasInAI

Note #29 on Generative AI and foundation models: Attention vs Self-attention Layers in Transformers: Attention mechanism: enables models to focus on crucial parts of the input, mimicking human perception. They facilitate understanding by capturing long-range context and relationships within data focusing on relationships between input and output sequences. Despite their effectiveness, attention mechanisms face scalability challenges. Self-attention mechanism: focuses on a single sequence, allowing a model to learn about itself. Unlike regular attention, it relates input to output within the same sequence. Self-attention can be realized through diverse kernel (score) functions, offering a range of types. Among these, the softmax function stands out as a commonly employed choice, referred to as scaled-dot product attention (SDP) (for more details see [Vaswani et al. Proc. NeurIPS 2017] and [Lin et al: arXiv:2106.04554v2]). To avoid the complexity of SDP, we note that some tokens may are less crucial; not all parts of the input sequence requires equal attention and thus there are some methodologies to simplify SDPs (more details in medium.com): 1- Sparse attention: sparsifies the attention matrix 2- Matrix factorization: decomposes and approximates the attention matrix with a lower-rank matrix without losing too much information. 3- Locality-sensitive hashing: computes nearest-neighbor search from the attention matrix to choose related tokens. 4- Kernel attention: considers the softmax function as a kernel for computing the self-attention matrix more efficiently. pic ref: medium.com #machinelearning #deeplearning #neuralnetwork #nlp #generatieveai #llm #llms #datascience #dataanalytics #foundationmodels #genai #ml #cloudcomputing #cloudai #nlp #ai #transformers #attention #selfattention #attentionmatrix #sdp

𝐎𝐩𝐭𝐢𝐦𝐢𝐳𝐢𝐧𝐠 𝐈𝐧𝐟𝐨𝐫𝐦𝐚𝐭𝐢𝐨𝐧 𝐑𝐞𝐭𝐫𝐢𝐞𝐯𝐚𝐥 𝐟𝐨𝐫 𝐁𝐞𝐭𝐭𝐞𝐫 𝐈𝐧𝐬𝐢𝐠𝐡𝐭𝐬 We aim to move beyond the fundamentals of Retrieval-Augmented Generation (RAG) and explore what an advanced RAG system truly entails. Going beyond the 101-level understanding means integrating the following steps: 1️⃣ Query: We start by enhancing the user’s query, sharpening it to ensure we get more precise and useful results. 2️⃣ Retrieval: Then, we dive into the search for relevant data. 3️⃣ Re-ranking: It's not enough to just find relevant content. We re-rank it, bringing the most valuable insights to the top. 4️⃣ Generation: Using these prioritized results, our language model generates a comprehensive response. 5️⃣ Response Validation: Before delivering the answer, we verify it. Does it make sense? Is it accurate and aligned with the original question? 6️⃣ Response: Once validated, the solution is ready to be shared with the user. #AI #MachineLearning #InformationRetrieval #NLP #DataInsights #LLM #RAG

What is the difference between Self-Attention and Cross-Attention? They are actually very similar! The self-attention computes the interactions between the different elements of an input sequence (for example, the different words in a sentence), and the cross-attention computes the interactions between the elements of 2 different input sequences (for example, how words in one sentence influence words of another sentence). Both of those attentions can be computed by the same process. We have 3 matrices, Wk, Wq, and Wv, and they project the input vectors into Keys, Queries, and Values vectors. The self-attentions are computed by using the same input vectors, whereas the cross-attentions are computed by using vectors coming from 2 different sources. Those input vectors in the case of self-attention can be internal hidden states within a Transformer, for example, and they can be the encoder output and the internal hidden states of a decoder in the case of an encoder-decoder Transformer for the cross-attentions. For the cross-attentions, the encoder output gets projected as Keys and Values, whereas the decoder hidden states get projected as Queries. Then, the softmax transformation of the matrix multiplication between Keys and Queries creates the attentions, self, or cross depending on the input vectors. The output of the attention layer is just the matrix multiplication between the attention matrix and the Values vectors. #ai #ml #dl #deeplearning #transformer #machinelearning #nlp #selfattention #neuralnetwork #europe #germany

And that’s a wrap for Day 3 of the #NLPSummit 2024! It’s been a day full of insights from #NLP and #GenerativeAI experts:  - Alina Dia Trambitas-Miron, PhD from John Snow introduced an advanced tool designed to automate key aspects of the literature review process. - Supriya Raman from JPMorganChase explored different quantization methods and techniques, the common libraries used and discussed evaluation of performance and quality of quantized #LLMs using standard metrics -  Chang She from LanceDB explained how Lance format works, and the value it delivers to AI teams training models or putting applications into production. - Petros Zerfos from IBM Research discussed #LLM app development with the open-source data prep kit You can still register to get an access to on-demand videos: https://lnkd.in/dnS8eFjb #GenerativeAI #LargeLanguageModels #LLM #ResponsibleAI #DigitalTransformation #RAG #RetrivalAugmentedGeneration #HealthcareAI #ClinicalLLM #DigitalHealth #PromptEngineering #MedicalChatbot #MedicalGPT

𝐎𝐩𝐭𝐢𝐦𝐢𝐳𝐢𝐧𝐠 𝐈𝐧𝐟𝐨𝐫𝐦𝐚𝐭𝐢𝐨𝐧 𝐑𝐞𝐭𝐫𝐢𝐞𝐯𝐚𝐥 𝐟𝐨𝐫 𝐁𝐞𝐭𝐭𝐞𝐫 𝐈𝐧𝐬𝐢𝐠𝐡𝐭𝐬 We aim to move beyond the fundamentals of Retrieval-Augmented Generation (RAG) and explore what an advanced RAG system truly entails. Going beyond the 101-level understanding means integrating the following steps: 1️⃣ Query: We start by enhancing the user’s query, sharpening it to ensure we get more precise and useful results. 2️⃣ Retrieval: Then, we dive into the search for relevant data. 3️⃣ Re-ranking: It's not enough to just find relevant content. We re-rank it, bringing the most valuable insights to the top. 4️⃣ Generation: Using these prioritized results, our language model generates a comprehensive response. 5️⃣ Response Validation: Before delivering the answer, we verify it. Does it make sense? Is it accurate and aligned with the original question? 6️⃣ Response: Once validated, the solution is ready to be shared with the user. #AI #MachineLearning #InformationRetrieval #NLP #DataInsights #LLM #RAG

🌟 Understanding Self-Attention vs. Multi-Head Self-Attention 🌟 In the world of deep learning, particularly with transformers, understanding the mechanisms of self-attention and multi-head self-attention is crucial for leveraging their full potential. Here’s a quick breakdown of the key differences: 🔍 Self-Attention: - Analyzes relationships between elements in a sequence. - Transforms each element into Query, Key, and Value vectors. - Computes compatibility scores to determine how much focus each element should have on others. - Creates a context-aware representation by weighting Value vectors. ✨ Multi-Head Self-Attention: - Runs multiple self-attention processes in parallel, allowing the model to focus on different aspects. - Projects input into multiple Q, K, and V vectors for each head. Calculates separate attention scores and outputs, then concatenates them for a richer representation. 💡 Analogy: - Self-attention is like understanding how each word in a sentence relates to others to grasp overall meaning. - Multi-head self-attention is akin to reading the sentence multiple times, focusing on different elements like grammar, context, and sentiment, and then combining insights for a deeper understanding. These mechanisms are foundational in tasks like machine translation, enabling models to capture long-range dependencies effectively. As we continue to innovate in deep learning, self-attention and multi-head self-attention will remain pivotal in advancing our capabilities in processing complex sequential data. #SelfAttention #MultiHeadSelfAttention #Transformers #DeepLearning #NLP #AI #MachineLearning

Exploring Bidirectional RNNs and Their Types:- Bidirectional RNNs are a unique type of recurrent neural network that process information in both forward and backward directions, capturing context from both past and future points in a sequence. This dual-direction processing allows them to understand surrounding information, making them highly effective for tasks where contextual awareness is critical. Why Use Bidirectional RNNs? Enhanced Contextual Awareness: By using information from both directions, bidirectional RNNs capture context that traditional RNNs might miss. This is particularly useful for predicting a word in between others or inferring meaning in sequences. Example: For instance, in a sentence like “The cat is on the [___] chasing a mouse,” a bidirectional RNN can predict “mat” by considering both previous words ("on the") and upcoming ones ("chasing a mouse"). Types of Bidirectional RNN- 1. One-to-One Description: Basic architecture where a single input maps to a single output. Example: Standard neural networks use this setup, not specifically designed for sequences. 2. One-to-Many Description: A single input generates a sequence of outputs. Example: Image captioning: One image input generates a sequence of descriptive words. 3. Many-to-One Description: A sequence of inputs converges into a single output. Example: Sentiment analysis: Multiple words or sentences lead to one sentiment label. Great for summarization tasks where the sequence is condensed into one 4. Many-to-Many Description: A sequence of inputs maps to a sequence of outputs, typically maintaining the same length. Example: Machine translation: Each word in a sentence translates into a corresponding word in another language. #ArtificialIntelligence #MachineLearning #DeepLearning #NeuralNetworks #RNN #BidirectionalRNN #NLP #NaturalLanguageProcessing #DataScience #AIResearch #TechInnovation #SequenceLearning #LanguageModeling #DeepLearningModels #MLAlgorithms #DataEngineering #DataAnalytics #SpeechRecognition #FutureOfAI #AIApplications #AICommunity #LearningAI #ComputationalLinguistics #AIForGood #TechTrends #AIExplained #PredictiveModeling #SmartAlgorithms #AIInsights #InnovationInAI

In the fast-paced world of medical research, efficient data access is key. Discover how tools like 3RDi Search are revolutionizing data access and analysis, making research faster and more accurate. Learn more about the benefits and future of intelligent search in the medical field. Read blog https://lnkd.in/dgU-B8na #IntelligentSearch #MedicalResearch #3RDiSearch #SemanticSearch #AI #NLP #FutureResearch #Research #blog

Choosing the right LLM in 3 Simple Steps. Various models are trained on different datasets and possess differing numbers of parameters. Selecting an inappropriate model can lead to significant and undesired consequences, such as biases originating from the training data or hallucinations that are simply incorrect. Here are the factors you need to consider while choosing the right model for you: Step1: Clearly define your use case – In this step you need to define what exactly you want your foundation model to do such as text summarization, image captioning, etc. Step2: Make a comparison chart- In this step, you need to list the various available model options, including their size, cost, performance, risk, and method of deployment. You can also use a model card for this step. Step3: Choose the model that provides most value – Run test and evaluate option based on the use case. Three factors that you need to consider while choosing your model are accuracy, reliability, and speed. #AI #LLM #NLP #CV #DeepLearning #MachineLearning