Subham Kundu

Shashank Sonkar and colleagues are back with another great paper and matching GitHub repo, "Pedagogical Alignment of Large Language Models". You can read the paper on arXiv here - https://lnkd.in/enZfCJgi, and browse the code here - https://lnkd.in/gyW5hSzp. They demonstrate an interesting approach to generating synthetic data to train open weights models to behave more pedagogically, as well as developing some new evaluation metrics. I think there's lots here for those of us interested in this kind of work to learn from and build on. (If synthetic data generation is new do you, check out this post from answer.ai earlier this week - https://lnkd.in/ecsG7R4b.)

38

4 Comments

Exploring Hybrid Retrieval-Augmented Generation (RAG) for Enhanced Information Retrieval In my recent research in the  realm of information retrieval, I came across Hybrid RAG which balances between semantic (vector similarity) and syntactic (keyword) search approaches for achieving accurate and comprehensive results. A cutting-edge solution that combines the strengths of both methods to revolutionize how we interact with data. Key Features: - Semantic Search (Vector Similarity): Ideal for handling queries with typos or nuanced intents, leveraging powerful embeddings like HuggingFace to capture semantic relationships. - Syntactic Search (Keyword Matching): Perfect for precise matches on keywords, abbreviations, and names, employing techniques such as BM25Encoder for effective keyword search. - Hybrid Approach: By blending vector and keyword search scores using advanced fusion techniques like Reciprocal Rank Fusion (RRF), Hybrid RAG ensures robust and context-aware retrieval. Why Hybrid RAG Matters: Semantic search (using dense embeddings) excels in capturing semantic similarity and context, making it effective for understanding the meaning behind queries and documents. On the other hand, syntactic search (using techniques like BM25) is better at capturing exact matches and structural similarities. Combining both ensures a more comprehensive coverage of relevant information. Implementing Hybrid RAG: -- Set up with Pinecone for indexing and retrieval. -- Utilize HuggingFaceEmbeddings for semantic search. -- Employ BM25Encoder for efficient keyword matching. -- Integrate PineconeHybridSearchRetriever for seamless fusion and retrieval. Unlock the power of Hybrid RAG to enhance your information retrieval capabilities. Explore more about how this innovative approach can elevate your data analytics and decision support systems. #HybridRAG #InformationRetrieval #SemanticSearch #SyntacticSearch #AI #MachineLearning #DataScience #TechnologyInnovation #PineconeAI #GenAI #MLOps 

7

int8 embeddings from Cohere (4x saving in memory, 30% latency for retrieval, 0.01% loss in search quality (yes, it's 10^{-4}) ) binary embeddings (yes, it's 1 bit, 32x saving in memory), led to 6-10% percent loss in search quality measured by MTEB that might be reasonable for some applications, where 5% retrieval noise is critical or might be reduced through other means there are technique how to improve it See details inside https://lnkd.in/gzEHh-Wq

22

OCR sucks! There I said it. Many people are building bandage solutions to try and patch it, like this open source project. There are still countless problems like analyzing images, forms, tables, charts, hand written notes, and more. I developed the Discrepancy AI Optic system specifically to replace OCR and fix all the issues with it. Optic AI gives you structured data directly from your files. One simple API to replace your entire OCR and data extraction pipeline. No pre-possessing no post-processing no creating templates, no data sanitation and standardization. With Optic AI you just define the data you want and we give you the data in the exact JSON structure you requested. Want to check out a demo? Book some time with me here: https://lnkd.in/gyGxBN3r. ///

11

3 Comments

🔸Here's Todays Leetcode Problem of the day : [ 🔥DAY 193 ] 🔸Name : Find if Path Exists in Graph 🔸Description : There is a bi-directional graph with n vertices, where each vertex is labeled from 0 to n - 1 (inclusive). The edges in the graph are represented as a 2D integer array edges, where each edges[i] = [ui, vi] denotes a bi-directional edge between vertex ui and vertex vi. Every vertex pair is connected by at most one edge, and no vertex has an edge to itself. You want to determine if there is a valid path that exists from vertex source to vertex destination. Given edges and the integers n, source, and destination, return true if there is a valid path from source to destination, or false otherwise.   Example 1: YInput: n = 3, edges = [[0,1],[1,2],[2,0]], source = 0, destination = 2 Output: true Explanation: There are two paths from vertex 0 to vertex 2: - 0 → 1 → 2 - 0 → 2 Example 2: oInput: n = 6, edges = [[0,1],[0,2],[3,5],[5,4],[4,3]], source = 0, destination = 5 Output: false Explanation: There is no path from vertex 0 to vertex 5.   Constraints: 1 <= n <= 2 * 105 0 <= edges.length <= 2 * 105 edges[i].length == 2 0 <= ui, vi <= n - 1 ui != vi 0 <= source, destination <= n - 1 There are no duplicate edges. There are no self edges. 🔸Link: https://lnkd.in/dd3Wj4YZ

6

Apple spilled the beans on Apple Intelligence Foundation Models (notes below): Architecture: > Dense - decoder only transformer architecture > RMSNorm & Query/ Key normalization > GQA (w/ 8 KV heads) > SwiGLU activation & RoPE (base_freq=500K for long context) Pre-training & Tokenisation: > Webpages crawled through the Applebot (web crawl) > Code & Math datasets (publicaly licensed) > BPE tokenizer w/ 100K vocab for server & 49K for on-device Three step pre-training: >Core (consumes most of the compute budget) AFM-server - 6.3T tokens + 4096 seq length AFM-on-device - initialised from a pruned 6.4B server model, trained for full 6.3T tokens along with distillation loss - Continued (down-weight lower quality data and increase code, math, licensed data weight) 1T tokens, w/ 8192 seq length no distillation loss for AFM-on-device in this phase - Context-lengthening with long sequence + synthetic data 100B tokens, w/ 32768 seq length Training Infrastructure: > Pre-trained v4 & v5p TPU clusters > Using AXLearn (JAX) with a combination of tensor, fsdp, and seq parallelism > AFM Server trained on 8192 TPUv4 chips > AFM On-device trained on 2048 TPUv5p chips Post Training: > Hybrid data - synthetic + human annotated > Synthetic data for Mathematics (problem rephrase & reversion + evolution), Tool use and coding > RLHF: Iterative Teaching Committee - Refresh online human preference data collection using a diverse set of best performing model > For above, collect pairwise human preference on responses sampled from the comittee Deployment: > Adapters for each task, adapter values represented using 16-bits, loaded on-the-fly based on the task > Quantised under 4-bit-per-weight (3.7 bpw), use accuracy recovering adapters for regaining the lost performance > Accuracy recovery adapter trains on 10B tokens across different ranks, 8, 16, 32 > Some layers (unimportant) pushed to 2-bit Evaluation: > On-device: SoTA in IFEval and competitive with Gemma 7B on AlpacaEval 2.0 > Server: SoTA in IFEval, comparable to Mixtral 8x22B in Arena Hard > Competitve with GPT 4/ Gemini 1.5 on Tools/ function calling, writing (summarisation, composition) benchmarks > On-device beats L3 8B on Math The report is quite feature packed, quite enjoyed skimming through it. Thanks, Apple, for being so open about your practices and spilling the beans on what would power the next gen of on-device ML. More notes, coming soon! 🤗

315

13 Comments

📊 Data Insights: Outlier Detection & Removal Techniques! 📊 Today, I explored multiple techniques for outlier detection and removal, each tailored for different datasets and goals. Here’s a quick breakdown of the methods used: Percentile Technique - This approach helped refine New York City hotel price data by removing the top and bottom 1% extremes. Reducing from 48,895 to 48,183 records highlighted the utility of percentiles in addressing business needs without skewing insights. Interquartile Range (IQR) - Perfect for identifying outliers in a distribution. The IQR approach on the same dataset (using Q1-1.5*QR and Q3+1.5*QR) enabled the removal of extreme values beyond 25th and 75th percentiles, resulting in a refined set of 45,918 records. Z-Score Technique - Leveraging the Z-score technique to clean data with a threshold of ±3, I removed values that deviated significantly from the mean. This method retained the most statistically relevant data for analysis. Standard Deviation - Applied on normally distributed height data, using a 3-standard deviation threshold, where we retained 9,993 out of 10,000 records. Great technique when the dataset follows a normal distribution! Each technique has its unique strengths in keeping our data accurate and meaningful, catering to varying business and analytical needs. 🔍 Curious to learn which method you find most effective for your data projects? #DataScience #MachineLearning #OutlierDetection #DataCleaning #DataAnalytics #Python GitHub link : https://lnkd.in/eJUdZn4i

19

A Survey of RAG and RAU: Advancing Natural Language Processing with Retrieval-Augmented Language Models Quick read: https://lnkd.in/gnGqk44W Paper: https://lnkd.in/g8-VftbX #llms #largelanguagemodels #ai

20

💡 Optimizing your AI's retrieval process can drastically reduce errors and boost performance. In this article, Atita Arora details advanced strategies for optimizing RAG systems through an evaluation-based methodology. Here are some of the insights we had using Qdrant and Quotient AI: ✅ Low chunk relevance and context relevance make the model more likely to hallucinate. ✅ Retrieving more documents and reducing chunk size boosts model performance. ✅ Dynamic retrieval adapts to query variations, improving response accuracy. ✅ Changing LLMs or prompts affects response quality, requiring fine-tuning for better performance. Follow along with her four-step experiment and try a pipeline in various ways: https://buff.ly/3zaTnVG Github Notebook: https://lnkd.in/dbKGgFTT Workshop on YouTube: https://lnkd.in/dqGWkqhm

87

1 Comment

Quotient AI helps developers measure the quality of their RAG systems - so they know how to improve them. Read the latest case study with Qdrant 👇

22

#part2 of #openbox conversation with Aditya Joshi & Dipankar Srirag. We discussed about their paper "Evaluating Dialect Robustness of Language Models via Conversation Understanding". Some of the key highlights from the conversation are: 1. Approach to evaluating the #dialect robustness of #LLM in the context of conversational understanding. 2. The limitations of existing datasets and the need for alternative #datasets that capture the nuances of different dialects. 3. #Masking approach to evaluate how language models fill in the masked words in conversations. 4. The performance results show that there is a degradation in performance for non-standard dialects of English, highlighting the need for dialect-specific customization. 5. Comparing open source and closed source language models and find that #finetuning open source models yields better results. And also examining the impact of conversation length on model performance and find that shorter conversations lead to more accurate predictions. 6. This work aims at contribute towards the localization of language models and address the challenges of dialect understanding in business and social settings. Future work includes exploring adaptation techniques to improve dialect generalization and addressing the limitations of fine-tuning and conversation lens. Link: https://shorturl.at/YQkZ2

9

2 Comments

#part2 of #openbox conversation with Aditya Joshi & Dipankar Srirag. We discussed about their paper "Evaluating Dialect Robustness of Language Models via Conversation Understanding". Some of the key highlights from the conversation are: 1. Approach to evaluating the #dialect robustness of #LLM in the context of conversational understanding. 2. The limitations of existing datasets and the need for alternative #datasets that capture the nuances of different dialects. 3. #Masking approach to evaluate how language models fill in the masked words in conversations. 4. The performance results show that there is a degradation in performance for non-standard dialects of English, highlighting the need for dialect-specific customization. 5. Comparing open source and closed source language models and find that #finetuning open source models yields better results. And also examining the impact of conversation length on model performance and find that shorter conversations lead to more accurate predictions. 6. This work aims at contribute towards the localization of language models and address the challenges of dialect understanding in business and social settings. Future work includes exploring adaptation techniques to improve dialect generalization and addressing the limitations of fine-tuning and conversation lens. Link: https://shorturl.at/YQkZ2

7

1 Comment

Best way to finetune LLMs for preference alignment Open Large Language Models (LLMs), like Llama and Mistral, have transformed natural language processing (NLP) by enabling tasks ranging from language translation to conversational AI. However, to make these models perform better and align them with user expectations, specific optimization techniques are essential. The most widely used optimization methods are Proximal Policy Optimization (PPO), Direct Preference Optimization (DPO), and Supervised Fine-Tuning (SFT). However, Objective Reinforcement Preference Optimization (ORPO) is a new method that outperformed all others in my experiments. The reason behind its success lies in its objective function. In the objective functions of other methods, while they maximize the likelihood of assigning the highest possible probability to the correct word, they neglect the rejected responses, as the ‘yi’ boolean term marks a zero for every other response. This oversight can increase the likelihood of higher probabilities being assigned to negative responses for generic capability of LLM. ORPO proposes a new objective function. LSFT is the original cross-entropy for maximizing the correct word’s probability, and LOR is the odds ratio between chosen and rejected responses. This approach helped me achieve the top score on the leaderboard, surpassing models like GPT, MedPaLM, and Gemini, by using only Llama 8B. Leaderboard Link: https://lnkd.in/dQq9uPD5 We at Jivi AI will continue to push boundaries for VLLM and LLM in medical domain. FInal model of jivi is under testing, will be released soon with guide for training LLM.

87

17 Comments

Classy Vision is a new end-to-end, PyTorch-based framework for large-scale training of state-of-the-art image and video classification models. https://lnkd.in/gSUswHBJ

3

torchtune is a PyTorch-native library for easily authoring, fine-tuning and experimenting with LLMs. We're excited to announce our alpha release! torchtune provides: Native-PyTorch implementations of popular LLMs using composable and modular building blocks Easy-to-use and hackable training recipes for popular fine-tuning techniques (LoRA, QLoRA) - no trainers, no frameworks, just PyTorch! YAML configs for easily configuring training, evaluation, quantization or inference recipes Built-in support for many popular dataset formats and prompt templates to help you quickly get started with training #largelanguagemodels #llm #finetune https://lnkd.in/gAj4iyDj

25

Golden Retrieval - MM-Embed: Universal Multimodal Retrieval with Multimodal LLMs (#NVIDIA, November 2024) Paper: https://lnkd.in/gpmQBsUe Abstract: "State-of-the-art retrieval models typically address a straightforward search scenario, where retrieval tasks are fixed (e.g., finding a passage to answer a specific question) and only a single #modality is supported for both queries and retrieved results. This paper introduces techniques for advancing information retrieval with multimodal large language models (MLLMs), enabling a broader search scenario, termed universal multimodal retrieval, where multiple modalities and diverse retrieval tasks are accommodated. To this end, we first study fine-tuning an MLLM as a bi-encoder retriever on 10 datasets with 16 retrieval tasks. Our empirical results show that the fine-tuned MLLM retriever is capable of understanding challenging queries, composed of both text and image, but underperforms a smaller CLIP retriever in cross-modal retrieval tasks due to modality bias from #MLLMs. To address the issue, we propose modality-aware hard negative mining to mitigate the modality bias exhibited by MLLM retrievers. Second, we propose to continually fine-tune the universal multimodal retriever to enhance its text retrieval capability while maintaining multimodal retrieval capability. As a result, our model, MM-Embed, achieves state-of-the-art performance on the multimodal retrieval benchmark M-BEIR, which spans multiple domains and tasks, while also surpassing the state-of-the-art text retrieval model, NV-Embed-v1, on MTEB retrieval benchmark. Finally, we explore to prompt the off-the-shelf MLLMs as the zero-shot rerankers to refine the ranking of the candidates from the multimodal retriever. We find that through #prompt-and-reranking, MLLMs can further improve multimodal retrieval when the user queries (e.g., text-image composed queries) are more complex and challenging to understand. These findings also pave the way to advance universal multimodal retrieval in the future." #genai #retriever #multimodal #search #information Zachary Huhn Summer Crenshaw Scott Phillips Lohith Srikanth Pentapalli Kelly Cohen

7

3 Comments

This is detailed guide to calculate GPU Memory 💾 requirement for serving LLMs. Ps - it doesn't take context length into consideration Website 📖- https://lnkd.in/gf7YzqJt

116

3 Comments