Towards Data Science’s Post

How can reranking models improve your RAG pipeline? This article by Daniel Klitzke explains how to integrate a reranking model with Huggingface Transformers to boost context quality. #RAG #LLM

🚀 Master Graph Traversal: DFS vs BFS Explained! Are you ready to dive deep into the world of Graph Algorithms? 🌐 Whether you're preparing for coding interviews 🎯 or looking to sharpen your skills, understanding Depth-First Search (DFS) and Breadth-First Search (BFS) is key! In this post, we cover: 🔍 DFS: Explore all paths in a graph by going as deep as possible before backtracking. Perfect for solving puzzles like Sudoku 🧩 and finding paths in mazes 🗺️. 🔗 BFS: Explore level by level, ideal for finding the shortest path in networks 🌐, crawling the web 🌍, or even AI decision-making 🤖. 💡 Applications: >Network routing 🛣️ >Web crawling 🚀 >Scheduling tasks 📅 >Game development 🎮 Check out the video to learn these key algorithms and apply them to real-world scenarios! 🚀 #GraphAlgorithms #DFS #BFS #CodingInterviewPrep #Pathfinding #TechSkills #AI

To summarize text using Hugging Face's BART model, load the model and tokenizer, input the text, and the model generates a concise summary.

Day 30 🔥 Binary Trees 1️⃣ Zigzag Level Order Traversal Observation: This is a variation of the standard level order traversal, but the direction of traversal alternates at each level (left to right, then right to left). Approach: Use a queue to store nodes level by level. For each level, use an array to store node values. Insert elements either left-to-right or right-to-left depending on the current direction. After processing all nodes in a level, flip the direction. Return the final list of level values after completing the traversal. --- 2️⃣ Max Path Sum Observation: The path in the binary tree can start and end at any node. The goal is to find the maximum sum path between any two nodes. Approach: Use recursion to explore left and right subtrees. At each node, calculate the maximum path sum through the left and right child. Keep track of the maximum path sum globally. The result at each node is the maximum of either the left or right child’s value plus the node’s value. --- 3️⃣ Binary Tree from Preorder and Inorder Traversal Observation: Preorder traversal gives the root node first, and inorder traversal helps identify the left and right subtrees. Approach: Build a hash map for the inorder traversal to quickly find the root index. Recursively build the left and right subtrees based on the root node's position in the inorder array. Repeat this process to construct the entire tree. --- #DSA #Algorithms #ProblemSolving #CodingPractice #InterviewPrep #TechInterviews #Placements #BinaryTree #DataStructures

🚀 Master Graph Traversal: DFS vs BFS Explained! Are you ready to dive deep into the world of Graph Algorithms? 🌐 Whether you're preparing for coding interviews 🎯 or looking to sharpen your skills, understanding Depth-First Search (DFS) and Breadth-First Search (BFS) is key! In this post, we cover: 🔍 DFS: Explore all paths in a graph by going as deep as possible before backtracking. Perfect for solving puzzles like Sudoku 🧩 and finding paths in mazes 🗺️. 🔗 BFS: Explore level by level, ideal for finding the shortest path in networks 🌐, crawling the web 🌍, or even AI decision-making 🤖. 💡 Applications: > Network routing 🛣️ >Web crawling 🚀 >Scheduling tasks 📅 >Game development 🎮 Check out the video to learn these key algorithms and apply them to real-world scenarios! 🚀 https://lnkd.in/dHZpwjKn #GraphAlgorithms #DFS #BFS #CodingInterviewPrep #Pathfinding #TechSkills #AI

🚀 Introducing a Learnable Bias Term, Δ𝑏, in LoRA! 🚀 ⚡️ Part 2 of the series "Breaking Down LoRA" ⚡️ After diving deeper into LoRA (Low-Rank Adaptation), implementing it from scratch in PyTorch (code: https://lnkd.in/dub47uBj), I experimented with a new idea that could potentially enhance its performance even further. Let’s take a quick look! 🤔 For a vanilla linear layer, the output is: 𝑦 = 𝑥⋅𝑊ᵀ + 𝑏 (Reference: https://lnkd.in/d5gff8zP). For a LoRA-adapted linear layer, the output becomes: 𝑦 = 𝑥⋅(𝑊 + (α/𝑟)⋅Δ𝑊)ᵀ + 𝑏 This works well, but I noticed something: during fine-tuning with LoRA, the bias term (𝑏) stays fixed. This means the transformation is still pivoting around the original pre-trained model's bias, which may limit flexibility. 🛠 My Solution? I introduced a learnable bias term, Δ𝑏, into the LoRA linear layer. However, it doesn't make sense to scale this term using ΔW's scaling factor. Hence, we add another hyper-parameter, β, making the output: 𝑦 = 𝑥⋅(𝑊 + (α/𝑟)⋅Δ𝑊)ᵀ + (𝑏 + β⋅Δ𝑏) Since the dimension of Δ𝑏 matches 𝑏, I could have simply unfrozen the original bias. However, to stay consistent with LoRA's core philosophy—preserving the original model and maintaining the ability to attach/detach adapters—I added it as a separate term. Here's the code: https://lnkd.in/dub47uBj 🏆 The Results? • A modest improvement in accuracy on average compared to conventional LoRA for my experiments (pretrained on flattened MNIST & fine-tuned on flattened FashionMNIST). • This comes at the cost of ~5% ▲ in trainable parameters. 🧑💻 Check out the notebooks:   1️⃣ Without Δ𝑏: https://lnkd.in/dt5Bf5VM 2️⃣ With Δ𝑏: https://lnkd.in/dg4dq422 Is the accuracy gain worth the extra parameters? 🤔   Feel free to try it yourself and let me know what you think!? 📚 Catch-up on previous parts: • Part 1: 🚀 Implementing LoRA (Low-Rank Adaptation) from Scratch in PyTorch! 🚀 https://lnkd.in/dEvGTZwi #MachineLearning #PyTorch #LoRA #FineTuning #AI #DeepLearning #LLMs #ML #Experimentation

From visual inspection to using weighted distances, Rukshan Pramoditha walks us through six different methods for choosing the right number of neighbors when applying the k-NN algorithm.

The One Graph Concept That Stumped Me. . . . Have you ever encountered a concept that just wouldn't click? For me, it was finding an articulation point in graph theory. I spent an entire day trying to understand it, and even now, it feels like a brand-new challenge every time I revisit it! 😅 Here are my top 10 must-know graph algorithms: 1. Depth-First Search (DFS) 2. Breadth-First Search (BFS) 3. Dijkstra's Algorithm 4. Bellman-Ford Algorithm 5. Floyd-Warshall Algorithm 6. Kruskal's Algorithm 7. Prim's Algorithm 8. Kosaraju's Algorithm 9. Kahn's Algorithm 10. Graph Coloring Algorithm These algorithms are essential for tasks like finding the shortest paths, detecting cycles, and more. Each has its unique charm and complexity. Which graph concept or algorithm has been your toughest nut to crack? Let's discuss and learn from each other! 👇 #Graphs #Algorithms

Today, I tackled Problem No. 180: M-Coloring Problem in my DSA challenge. 🎨 Problem Overview:   The task was to determine if we can color an undirected graph using at most 'm' colors such that no two adjacent vertices share the same color. Approach:   To solve this problem, I implemented a backtracking algorithm that works as follows: 1. Recursive Coloring: Start from the first vertex and try to assign one of the 'm' colors. If successful, recursively attempt to color the next vertex. 2. Validation: Before assigning a color to a vertex, ensure that it doesn't conflict with the colors of adjacent vertices. 3. Backtrack: If assigning a color leads to a dead end, backtrack and try the next color. Key Takeaways:   1. Backtracking is a powerful technique for solving constraint satisfaction problems like graph coloring. 2. Understanding the adjacency matrix representation of graphs is crucial for implementing effective algorithms. 3. This problem illustrates how simple graph problems can have deeper implications in fields like scheduling and resource allocation. Personal Reflection:   This challenge reinforced my understanding of graph theory and algorithm design. I found it intriguing how coloring problems can relate to real-world scenarios. If anyone has alternative strategies or insights on this topic, I'd love to discuss! #DataStructures #Algorithms #GraphTheory #Backtracking #DSAChallenge

New on KDnuggets by Cornellius Y. Handling large text inputs with traditional transformers can be limiting, but this article breaks down how to use Longformer to extend that capacity. If you're working with lengthy text data, this step-by-step guide on fine-tuning Longformer with Hugging Face Transformers offers a clear path to improving classification and text generation tasks — click to learn how. https://lnkd.in/ghwWb2hY