Siddhant Kumar

Bhaskar Lalwani and I (Aniruddha Mukherjee) had worked to create kabaddiPy recently. kabaddiPy is an open-source repository for Kabaddi data - a first step towards making this fast-growing, dynamic sport more accessible for fans, analysts, and researchers. Kabaddi is the second most-watched sport in India after cricket, yet its data is not as widely available. We hope to be the trailblazers and change this! Whether you're into sports analytics 📊, passionate about Kabaddi 🤼‍♂️, or simply love data, we invite you to: ⭐ *Star the repo* 🔧 Contribute to the data 💡 Explore insights (by building predictive ML models / more!) Open an issue and let's chat! Let’s work together to build something great for India 🇮🇳 and the Kabaddi community! Link to the repo: https://lnkd.in/gGCnY_B4 Link to docs (for an overview): https://lnkd.in/g-Khu5_J btw, we're a part of GSSoC Extended and Hacktober-Fest as well! #opensource #kabaddi #sportsanalytics #dataanalysis #ProKabaddi

79

1 Comment

A recent discovery in the LLM research space discovered that a good chunk of transformer blocks are being used simply as linear layers by the LLMs The intriguing discovery has open up new potentials of model compute optimizations and model performance enhancements. I have tried my level best to put a medium article (due to content size limitation in linkedin) to explain the discovery and potentials findings in a easily understandable manner The article is free for everyone to read, give it a shot https://lnkd.in/gGsASB3z #AI #DeepLearning

31

Training LLMs using reinforcement learning using human feedback. I was slacking on this topic for 6 months and finally was able to go through it, and wow I am amazed at what technologies are being worked on in production-ready systems these days. Generally in RL of LLMs, we have a reward model (let's say a good/bad text classifier) that gives a reward (prediction 1 for good, -1 for bad) and an RL algorithm (based on multiple loss functions) updates the weights of the LLM model to achieve better results. This is an iterative process between the reward model, the LLM, and the RL algorithm. Overall, my take is the most important aspect of training LLM using RLHM is obtaining that human feedback and training a reward model.

28

Thanks to Siddhi Shroff I am debugging complex pytorch systems much more efficiently in VScode. A simple config parameter that boost productivity "debug.inlineValues":"on" For those working with pytorch tensors of 3+ dims - this makes the shapes permutation so much clearer. https://lnkd.in/g7bkE6-7.

32

1 Comment

Brain Teaser #10: Suppose we are fine-tuning a large language model using Low-Rank Adaptation (LoRA). We possess two low-rank update matrices, δWA​ and δWB​​, both associated with the rank "r". Considering our objective to preserve the model's current knowledge while incorporating new data from diverse features, what "mathematical rationale" should guide the selection of one of the update matrices (δWA​ or δWB) for low-rank adaptation? Hint: Consider the specific characteristics or properties of the matrices that can help in developing mathematical reasoning. Here is my response: The norm ratio, calculated as the ratio of the nuclear norm to the Frobenius norm of the update matrix, is an essential metric for assessing how updates impact a model. It provides insights into the model's stability and adaptability. Here’s a brief overview: https://lnkd.in/ekB_q9XX #MachineLearning #DeepLearning #NaturalLanguageProcessing #LoRA #FineTuning #AI #DataScience #ModelOptimization #NeuralNetworks #ArtificialIntelligence #TechInnovation #DataDriven #KnowledgeIntegration #Research #LLMs

30

4 Comments

Can we parallelise model training like transformers yet scale linearly to long sequence length just like RNN? 🤔 Yup, its possible with SSMs, infact they can do inference at computation cost of O(1). 🤯 First of all, how does it even compare with the most advanced Transformer capabilities like, - Self-Attention Mechanisms - Multi-Head Attention - Quadratic Scaling, etc. All of these have been outperformed on the new benchmarks across various domains such as language, audio, and genomics by Mamba. 🏆 Your model might be excelling with the above methods, but does it handle million-length sequences efficiently? Or process long sequences without breaking a sweat? Mamba efficiently integrates a selective state space model. What it effectively does for sequence processing is: ⚡ - Dynamically selects relevant information while discarding irrelevant data. - Operates with linear time complexity, ensuring fast and efficient processing. - Utilizes a hardware-aware parallel algorithm that maximizes GPU efficiency. - Incorporates simplified end-to-end neural network architecture without attention or MLP blocks. And during sequence modeling, - The input is processed through a selective mechanism, allowing the model to focus on the most relevant parts. - The state space parameters (such as Δ, B, and C) are dynamically adjusted based on the input, enhancing content-aware processing. - Efficient recurrent mode computation ensures faster training and inference with lower memory usage. But this approach doesn't just stop at sequence processing; it extends to real-world applications across different domains. For example: - Language modeling with Mamba outperforms Transformers of similar and even larger sizes in both pretraining and downstream evaluations. - Audio modeling with Mamba provides better quality and efficiency, making strides in speech recognition and generation. - Genomic sequence analysis with Mamba achieves state-of-the-art performance, handling long DNA sequences with ease. Mamba is not just an architectural improvement; it's a revolution in sequence modeling. I'll link the research paper down in comments, and you can explore its capabilities further. For more such detailed explanations, follow me. See ya! . . . #Mamba #SequenceModeling #AI #NeuralNetworks #Efficiency #StateSpaceModel #LanguageModeling #AudioModeling #Genomics #DeepLearning

24

❝ How do you taste your Wine? With the right Glass, served from the right Bottle. How do you utilize your LLM? With the right Task, served from the right Compute environment. ❞ If the quote above caught your interest, check out my blog post that was featured on the Toyota Connected India Medium channel last week: Leveraging LLMs in Cloud - 🔗 https://lnkd.in/gfHsdevH --- I hope this blog adds a bit of amusement for the seasoned AI/ML engineers 🧠👨‍💻 with the carefully crafted images and perhaps even the silly wine analogies 😉. Whether you're a cloud engineer ☁️ 👨‍💻, a product owner of cloud-based apps, or someone new to Generative AI in the cloud, this blog offers a detailed, beginner-friendly look into the selection and deployment of LLMs in cloud environments. --- A heartfelt thanks 🙏 to my colleagues at Toyota Connected India for taking the time to review, shape this piece and even share a creative writeup like below in the original post 🫡 👇 🤔 Curious to dive in? Click the link below to explore the blog on Medium! 🌐 🔗 https://lnkd.in/gfHsdevH

18

Bridging Causal Inference and AI Interpretability: Insights from Patchscopes In exploring the Patchscopes framework, I was intrigued by its resemblance to the do-operation in causal inference. Patchscopes performs interventions on a model's internal representations, akin to setting a controlled condition to observe outcomes, much like a do-operation tests causal relationships by manipulating variables. This approach not only enhances understanding of deep neural networks but also bridges concepts between model interpretability and causal inference, offering a robust tool for uncovering how models process information. This insight could significantly enrich our strategies for analyzing complex AI systems. https://lnkd.in/eM6g2mhg

4

From direct query to increasing sentence context window and complementing it with auto merging increase the performance and quality of the RAG inference. The suit from TruLens are really good to grasp the LLM output. The RAG triad is here. https://lnkd.in/ga63kg7Z

22

3 Comments

C++ is safe if you program with security in mind. Buffers have to be checked and input must be sanitized, but with great power comes great responsibility :)

5

Another great talk from the PyData London 2024 - Backtesting and error metrics for modern time series forecasting by Kishan Manani Backtesting is the time series forecasting equivalent to the machine learning cross-validation approach for training and testing models. https://lnkd.in/g-DWgqmB #timeseries #forecasting #machinelearning #python

46

5 Comments

Over the last 1 year, I've dived deep into the codebases of major projects like TensorFlow, MongoDB, Chromium, OpenSSL, LLVM, GCC, Kafka, Qt, CMake, and currently the Linux kernel. Transitioning from data structures and Development using Javascript based Tech Stacks to exploring open source C++ projects felt like stepping into an entirely new world. What I discovered were common practices and patterns that define production-level C++ code across these projects. Curious to know what they are? I've compiled my insights into a PDF—perfect for anyone looking to explore the Open Source Projects and contribute to the same. #C++ #OpenSource #SoftwareDevelopment #Programming

86

8 Comments

Create your own custom LLMs using open-source models to translate from "English to any language". Sharing my blog post and a notebook which shows how to develop your model to translate from "English to my mother language Punjabi". Blog link: https://lnkd.in/gPyknSde Github: https://lnkd.in/gbMq846b Fun fact, the output shows model has learned some "Hindi" in the output as well. Maybe because, I also often mix them together, and model is confused too. Open-source LLM support for local Indian languages is limited. This helps new learners to develop custom solutions for their own use-cases. This tutorial uses a simpler toy (English-Punjabi) dataset and sets up the training environment in Google Colab, runs within the free TPU memory limits. The tutorial is straightforward, making it easy for anyone to get started with language models using open-source tools. To generalize, vast datasets are needed. This notebooks gives a starting point. I hope this guide encourages more people to experiment with LLMs and contribute to local language support. Let's make learning and technology accessible to everyone. Check out the blog and start your journey. #AI #MachineLearning #NaturalLanguageProcessing #LanguageModels #OpenSource #GoogleColab #PunjabiTranslation #CommunityLearning

46

9 Comments

F1-Score vs Accuracy: Why F1-Score Should Be Your Focus. In machine learning, it's tempting to focus on ACCURACY—the percentage of correctly predicted instances. But what if I told you that accuracy isn't always the best indicator of your model's performance? In cases where the data is imbalanced (e.g., fraud detection, disease diagnosis), accuracy can be misleading. Imagine a scenario where only 5% of transactions are fraudulent—if your model predicts "non-fraudulent" 100% of the time, you’ll get 95% accuracy but miss all the critical fraud cases! When it comes to the F1-score: - Precision tells you how many of your positive predictions were correct. - Recall tells you how many of the actual positives you correctly identified. The F1-score balances both precision and recall, giving you a better sense of how well your model performs—especially when false positives and false negatives matter. If you're working on models where missing a positive instance could have serious consequences (like healthcare, security, or finance), the F1-score should be your go-to metric. Focus on the right metrics to make smarter models and better decisions. #MachineLearning #DataScience #AI #F1Score #ModelEvaluation #TechInsights #DataDriven #BrainK

10

🤯 Can a 7B model match performance to a 1.8T model like GPT-4 on vertical topics? The answer is yes 🏆 Case Study: NuminaMath 7B TIR The Numina team fine-tuned NuminaMath 7B TIR using DeepSeekMath-Base 7B, winning the 1st Progress Prize by solving 29/50 International Math Olympiad level problems. Here's their innovative approach: 1️⃣ Fine-tuning Process: • Stage 1: Utilized a large math problem dataset with Chain of Thought (CoT) template • Stage 2: Implemented Tool Integrated Reasoning (TIR) technique -Decomposed math problems into sequences of rationales, Python code, and their outputs - Above are created by GPT-4 • Both stages employed full fine-tuning, updating all model weights 2️⃣ Inference Strategy: • Developed novel Self-Consistency and Tool Integrated Reasoning (SC-TIR) algorithm • Generated 48 candidates (N=48) with a depth of 4 (M=4) for each problem • Created a matrix of reasoning at different stages • Used majority voting to determine the best path as the final outcome 3️⃣ Cost Optimization: • Quantized model to 8-bit precision which reduced compute resources and improved inference speed 🔍 Key Insights for Effective Fine-tuning: • Data Quality: Critical for problem-solving in specific verticals • Relevance: Data must closely align with target problem domain • Code Translation: Non-language data (e.g., math problems) translated into code for improved LLM comprehension • Dataset Templates: Clear demonstration of problem, reasoning steps, and final outcome in code format • Self-Consistency: Multiple attempts ensure reliable results. GPT4o is around 30 times more expensive than 4o-mini. If you can get similar result from 4o-mini by using 10x more token, then you save 2/3 of the cost. 📊 Impressive Results: NuminaMath 7B TIR achieved 68.2 on Mathbenchmark, compared to GPT-4's 76. Considering GPT-4 has at least 1.6T parameters (est. 1.8T per NVIDIA), this result is astonishing. 🤔 Some thoughts • Data Bottleneck: Data could be a huge bottle neck for model improvement ( Alexandr Wang mentioned during interview Harry Stebbings, my favorite podcast 20VC) • Enterprise Data Goldmine: 90% of valuable knowledge resides in enterprise data, unavailable publicly • Competitive Edge: Businesses with domain-specific data have a significant moat in LLM era. 🚀 Potential Opportunities : • Anonymization Techniques: Develop methods for unstructured enterprise data anonymization which make them avaliable for modal training. • Domain-Specific Models: Would be possible to create AI models solving industry-specific problems by using anonymization data. • Value Creation: Huge potential for businesses leveraging their unique data assets What are your thoughts on the future of specialized AI models? How might your industry benefit from domain-specific AI powered by proprietary data? Where do you see the regulation challenge ? #AI #LLM

7

1 Comment

While looking for something cool to post here (after ages!), I stumbled upon this crisp talk I delivered at #ngindia2022 that got rated as one of the best talks in the event. Check out the video to see why! https://lnkd.in/gSGXzfMU Even if you don't know JS, it's worth a watch for I realised that when taught the right way, even the most complex topics can stick with us forever! 😄 #javascript #webdevelopment #software #ngindia #publicspeaking #learning

60

1 Comment

No King Rules Forever: Mamba-2 Is Coming for your Transformers 🐍 Multimodal Large Language Models (MLLMs) are all the rage right now, but they come with a significant computational overhead due to the quadratic complexity of the Transformer architecture. ML-Mamba is a new MLLM that leverages the state-of-the-art Mamba-2 model for inference. Mamba-2 is known for its linear complexity and fast processing of long sequences, making it an ideal candidate for multimodal tasks. The researchers also explored various methods for integrating 2D visual selective scanning mechanisms into the multimodal learning process. Their findings? ML-Mamba achieves performance comparable to state-of-the-art methods like TinyLaVA and MobileVLM v2, while being significantly faster at inference time. Perhaps most notably, ML-Mamba matches the performance of LLaVA with 40% fewer parameters. To learn more about ML-Mamba and other AI highlights, check out this week's LLM Watch: https://lnkd.in/dRr2jxEp

146

2 Comments

LLMs or ChatGPT cannot reason in a rigorous manner. Consider certain patterns that occur frequently in the worlds' text data, programming solutions, maths solutions etc. A B C, B G J K, C T B Then mostly and without specifically instructing to do so, LLMs and Chat GPT can come up with the answer that A (G J K) (T B) is a plausible answer. Here A B C, B G J K, C T B can be loosely seen as text phrases, programming code, mathematical operators/concepts etc. This ability of LLMs and ChatGPT is enough to provide intelligent customer care Q&A or writing emails, stories, or simple to mid complexity programs in SQL, Python, C etc., but it cannot provide strict guarantees of its correctness and this becomes more apparent when it is asked difficult maths problems as below. LLMs, or ChatGPT have no algorithmic way of reasoning about them and that's why it makes errors which have been again loosely called as "hallucinations". Consider the following maths problem asked in the India's JEE Mains exams for the senior secondary students who aspire to get an admission into IITs, NITs and other Govt. engineering colleges in India. This is a 3 points problem to be solved in approximately in 2-3 mins :) If the domain of the function arcsin((3x-22)/(2x-19)) + log( (3x^2 -8x +5)/(x^2 -3x -10) ) is [a,b], then what is the value of 3a+10b? And in the picture is the ChatGPT solution which seems fluent but is absolutely wrong for the domains of both the functions 😊 . For the first function, we have -1 <=(3x-22)/(2x-19) <= 1. The left hand inequality's solution is x in (-inf, 41/5) U (19/2,+inf). While ChatGPT solves the left inequality by declaring x >=11/3. Similar it makes serious errors in the factorization of the quadratic numerator in the second function. It writes fluently but without any reasoning and is absolutely wrong. #LLMs, #ChatGPT #ai

11

1 Comment