Piyush Singla

Looking for recommendations for good hindi-english (hinglish) code-mixed speech to text models/corpuses. Anyone building anything with speech in the Indian context probably realized normal ASR systems fall flat, losing >50% context because of their inability to handle hinglish conversation (which is extremely common for a huge chunk of modern India). See the title of the first video when looking for shark tank. Once you start noticing it, its hard to unnotice. If speech to text doesn't work, every downstream task thas cascading failures. I'm not looking to build anything commercially so hopefully #research corpuses will also be okay. Literally only want to progress speech for India and this is a major blocker. This is one of those areas where other countries can't innovate for us and hand us a solution unfortunately. #speech #asr #codemix #hinglish #stt

3

1 Comment

Check out this highly recommended course for those interested in optimizing their models: - Introduction to Quantization - Symmetric and Asymmetric Quantization Modes - Hands-on Implementation from Scratch using PyTorch - covering Per Tensor, Per Channel (Axis), and Per Group techniques - Weights Packing: Exploring 2 or 4 Bits Quantization Methods https://lnkd.in/gMAuWXHm

43

Many of ML practitioners might have heard the term "Extending context length" for LLMs. In case if you want to dive deeper into this topic along with its code implementation, give this blog a read. I have described about Position Interpolation, NTK aware and Dynamic NTK method, which are widely used for extending context length.

68

Perhaps the most critical thing in a Retrieval Augmented Generation (RAG) system is the retrieval or searching strategy. Poor retrieval will lead to irrelevant documents going to the LLM leading to irrelevant generated answers or worse hallucinations. Is cosine similarity everything? Do check out below illustration which talks about four of the most common strategies for playing around with retrieval strategies in RAG systems: 1. Similarity-based retrieval: This is based on embedding similarity, you can play around with different metrics like cosine, maximal margin relevance etc. 2. Similarity with Threshold retrieval: Useful where we cap the similarity threshold and can help in removing irrelevant document but can drop recall 3. Multi-query retrieval: Make the LLM generate variations of your query and get documents based on these multi-versions of your initial query, often useful to get more recall or coverage of documents 4. Reranker retrieval: Use a model specialised in ranking documents based on search relevance Often you can combine some of these techniques and even use ensemble retrievers. Do explore them and go beyond just cosine similarity!

17

🚀 Big News from the NVIDIA Summit! Meesho has officially confirmed that they will open-source its advanced machine learning (ML) platform, making the same powerful tools that drive one of India's leading eCommerce platforms available to developers and data scientists worldwide. As Debdoot Mukherjee, Head of AI at Meesho, puts it: "This move will empower developers and data scientists everywhere to leverage the power of AI more effectively." This release includes both Meesho’s high-performance model orchestrator and their state-of-the-art feature store. What are your thoughts on the impact this open-source platform could have on the AI community? #MachineLearning #AI #OpenSource #meesho #NVIDIA

101

Thanks to Siddhi Shroff I am debugging complex pytorch systems much more efficiently in VScode. A simple config parameter that boost productivity "debug.inlineValues":"on" For those working with pytorch tensors of 3+ dims - this makes the shapes permutation so much clearer. https://lnkd.in/g7bkE6-7.

32

1 Comment

Bhaskar Lalwani and I (Aniruddha Mukherjee) had worked to create kabaddiPy recently. kabaddiPy is an open-source repository for Kabaddi data - a first step towards making this fast-growing, dynamic sport more accessible for fans, analysts, and researchers. Kabaddi is the second most-watched sport in India after cricket, yet its data is not as widely available. We hope to be the trailblazers and change this! Whether you're into sports analytics 📊, passionate about Kabaddi 🤼‍♂️, or simply love data, we invite you to: ⭐ *Star the repo* 🔧 Contribute to the data 💡 Explore insights (by building predictive ML models / more!) Open an issue and let's chat! Let’s work together to build something great for India 🇮🇳 and the Kabaddi community! Link to the repo: https://lnkd.in/gGCnY_B4 Link to docs (for an overview): https://lnkd.in/g-Khu5_J btw, we're a part of GSSoC Extended and Hacktober-Fest as well! #opensource #kabaddi #sportsanalytics #dataanalysis #ProKabaddi

79

1 Comment

LLMs or ChatGPT cannot reason in a rigorous manner. Consider certain patterns that occur frequently in the worlds' text data, programming solutions, maths solutions etc. A B C, B G J K, C T B Then mostly and without specifically instructing to do so, LLMs and Chat GPT can come up with the answer that A (G J K) (T B) is a plausible answer. Here A B C, B G J K, C T B can be loosely seen as text phrases, programming code, mathematical operators/concepts etc. This ability of LLMs and ChatGPT is enough to provide intelligent customer care Q&A or writing emails, stories, or simple to mid complexity programs in SQL, Python, C etc., but it cannot provide strict guarantees of its correctness and this becomes more apparent when it is asked difficult maths problems as below. LLMs, or ChatGPT have no algorithmic way of reasoning about them and that's why it makes errors which have been again loosely called as "hallucinations". Consider the following maths problem asked in the India's JEE Mains exams for the senior secondary students who aspire to get an admission into IITs, NITs and other Govt. engineering colleges in India. This is a 3 points problem to be solved in approximately in 2-3 mins :) If the domain of the function arcsin((3x-22)/(2x-19)) + log( (3x^2 -8x +5)/(x^2 -3x -10) ) is [a,b], then what is the value of 3a+10b? And in the picture is the ChatGPT solution which seems fluent but is absolutely wrong for the domains of both the functions 😊 . For the first function, we have -1 <=(3x-22)/(2x-19) <= 1. The left hand inequality's solution is x in (-inf, 41/5) U (19/2,+inf). While ChatGPT solves the left inequality by declaring x >=11/3. Similar it makes serious errors in the factorization of the quadratic numerator in the second function. It writes fluently but without any reasoning and is absolutely wrong. #LLMs, #ChatGPT #ai

11

1 Comment

Training LLMs using reinforcement learning using human feedback. I was slacking on this topic for 6 months and finally was able to go through it, and wow I am amazed at what technologies are being worked on in production-ready systems these days. Generally in RL of LLMs, we have a reward model (let's say a good/bad text classifier) that gives a reward (prediction 1 for good, -1 for bad) and an RL algorithm (based on multiple loss functions) updates the weights of the LLM model to achieve better results. This is an iterative process between the reward model, the LLM, and the RL algorithm. Overall, my take is the most important aspect of training LLM using RLHM is obtaining that human feedback and training a reward model.

28

As models grow larger with the rise of LLMs, choosing the right precision type has become important. Sharing some insights over FP16 vs BF16:   🔷  FP16 and BF16 both consume same memory.  🔷  FP16 has better precision but lower dynamic range. In fp16, largest number is 65,504. BF16 can go upto 3.39e+38 !  This is because fp16 has higher # mantissa bits and bf16 has higher #exponent bits. 🔷  For model training, you will have to experiment and understand whether your network is sensitive to range or precision. If a model requires more dynamic range, it will converge better with bf16, which has same range as fp32. However, some models will converge better with higher precision (fp16) 🔷  When finetuning an fp16 model pre-trained in bf16, overflow issues may arise as some of the bf16 model weights might be larger than 65k.   Image source: nvidia   #precisiontype #fp16 #bf16 #modeltraining #LLMs #machinelearning #ai #datascience #torch

30

Have you wondered about the striking synergies between Large Language Models (LLMs) and Automated Machine Learning(Auto ML) Models? A few years ago, AutoML models made a lot of buzz in the AI domain. Even bigger buzz has been made by LLMs in recent times. There are striking synergies between LLMs and AutoML models. The performance of both LLMs and AutoML models improves with the quantity and quality of data. Both aim to automate aspects of machine learning. LLMs automate text-related tasks, while AutoML models automate the entire machine-learning pipeline. While LLMs are suitable for applications requiring deep language understanding and generation, AutoML models are ideal for users who need efficient and scalable ML solutions across different domains. As the use of LLM applications continues to grow, the next decade is poised to witness a significant shift towards the automation and productization of these models rather than their stand-alone deployment. This evolution in technology is a fascinating prospect, and if you share this interest, I invite you to stay tuned and follow along for more. #artificialintelligence #generativeai #LLMs #AutoML #leadership #futurisits #reviewwithsrujana

92

4 Comments

Something I've written on how a T20 batting innings can be assessed. This framework can be extended to assessment of individual players as well, which I will do in the susequent posts. For now, please feel free to comment and if you like the content, please subscribe. #cricket #cricketanalytics #data #dashboards #datavisualization #sportsanalytics https://lnkd.in/gb7T-UzJ

28

6 Comments

Ever wondered how random numbers work in NumPy and why setting a random seed is important? 🌟 Check out my latest blog to learn more about true vs. pseudo-randomness and the role of random seeds. #Randomness #PseudoRandomness #NumPy #PythonProgramming #Algorithms #Computing #Cryptography #ScientificComputing

22

8 Comments

The company's search boss, Prabhakar Raghavan, told employees at an all-hands meeting in March that Google plans to build teams closer to users in key markets, including India and Brazil, where labor is cheaper than in the U.S. Seems like SkillsFuture is leveling up in India and Brazil; also means the Commoditization of typical "Core" roles could be performed beyond the US, with much lower cost. Now, my question, what can our next-generation Singaporean fresh graduates do, than those countries mentioned yet to? Also goes to many staff with the business-as-usual SG&A roles, how to keep them from moving into Thailand and Malaysia?

52

23 Comments

Introducing Saaras 1.0, state of the art multilingual speech-to-text translate model, supporting 10 Indian languages. Supports automatic language identification, and contextual biasing using prompts. Sarvam AI

16

Text chunks in retrieval may risk losing context. The below article introduces contextualizing each chunk using an LLM. The added cost and latency required for contextualizing is partially addressed through a lightweight LLM (haiku) and prompt caching. Along with other known techniques (BM25, reranking, etc.), a significant reduction in retrieval failure is achieved. Overall, a good article to read. https://lnkd.in/g_5fgAN8

10

Long-context LLMs are outperforming RAGs, so will RAGs be dead? 🤔 In this post, you will learn: 1. Comparison of RAG and LC LLMs 2. Performance - Cost Trade-off 3. Self Route: A new approach to bridge the gap What are your views on this research? Let me know in the comments! --------------------------------------------------------------------------- I hope this post helped you! Help people in your network by liking and reposting it ❤️ It inspires me to do more for our community. Follow Bhavishya Pandit to stay updated on GenAI! 🔥 #ml #llm #ai #genai #datascience #artificialintelligence #opensource #ibm #generativeai #largelanguagemodels #huggingface #openai #google #rag #machinelearning #python #langchain #microsoft #genai

515

49 Comments

Turbo Sparse: Achieving LLM SOTA Performance with Minimal Activated Parameters Exploiting activation sparsity helps accelerate inference process of large language models (LLMs) without compromising performance. However, activation sparsity is determined by activation functions, and commonly used ones like SwiGLU and GeGLU exhibit limited sparsity. To address these challenges, this paper proposed a novel dReLU function, which is designed to improve LLM activation sparsity, along with a high-quality training data mixture ratio to facilitate effective sparsification. 𝗞𝗲𝘆 𝗰𝗼𝗻𝘁𝗿𝗶𝗯𝘂𝘁𝗶𝗼𝗻𝘀: - Efficient dReLU activation function: proposed method utilizes fewer than 150B tokens, representing less than 1% of typical pretraining tokens (commonly 15T tokens). - Sparse activated models: released sparsely-activated TurboSparse-Mistral7B and TurboSparse-Mixtral-47B models, which shows better performance compared to original versions - Practical inference speedup: Evaluation shows that with released models, 2-5× speedup can be achieved, with up to 10 tokens/s even without a GPU on TurboSparse-Mixtral-47B. 𝗔𝗻𝗮𝗹𝘆𝘀𝗶𝘀 i) Limitations of existing ReLUfication - sparsity evaluation of ReLULlama-7B and original Llama-2-7B, shows that existing ReLUfication methods can only improve the sparsity from 40% to 67% - This can be attributed to the ReLU activation function applied after gate weight, which masks all negative activations to zero ii) dReLU - newly proposed activation function, where ReLU is applied after both the up- and gate-projection - for an experiment comparing models using dReLU and SwiGLU, results shows that dReLU structure exhibit similar convergence compared to those using SwiGLU -further evaluation results for various sparsity levels demonstrate dReLU activation function enables high sparsity without significant degradation in performance, maintaining competitive performance even at 90% sparsity - In contrast, the SwiGLU-based model experiences a severe increase in perplexity as sparsity reaches 80%. 𝗘𝘅𝗽𝗲𝗿𝗶𝗺𝗲𝗻𝘁𝗮𝗹 𝗥𝗲𝘀𝘂𝗹𝘁𝘀 - TurboSparse-Mistral-7B outperforms Gemma-2B by far, while only activating 3B parameters. - TurboSparse-Mixtral-47B outperforms the original Mixtral-47B with only 4.5B parameters activated - for activations with a value of zero, TurboSparse-Mistral-7B, on average, has 90% of the neurons inactive in each layer, and TurboSparse-Mixtral-47B with 85% on average for each expert FFN 𝗜𝗻𝗳𝗲𝗿𝗲𝗻𝗰𝗲 𝗦𝗽𝗲𝗲𝗱𝘂𝗽 𝗘𝘃𝗮𝗹𝘂𝗮𝘁𝗶𝗼𝗻 - On Pure CPU Inference, ReLUfied models achieve 2.08-2.28× speedup over original model. - On Hybrid GPU-CPU Inference, demonstrate significant speedups ranging from 2.52 to 4.64× compared to the baseline llama.cpp - For Deploying LLMs on mobile phones, achieves a 22.2× speedup using TurboSparse-Mixtral-47B inference compared to llama.cpp with original Mixtral-47B 𝗣𝗮𝗽𝗲𝗿: https://lnkd.in/eeHCWhUt 𝗠𝗼𝗱𝗲𝗹𝘀: https://lnkd.in/etHpZUju

24

1 Comment

F1-Score vs Accuracy: Why F1-Score Should Be Your Focus. In machine learning, it's tempting to focus on ACCURACY—the percentage of correctly predicted instances. But what if I told you that accuracy isn't always the best indicator of your model's performance? In cases where the data is imbalanced (e.g., fraud detection, disease diagnosis), accuracy can be misleading. Imagine a scenario where only 5% of transactions are fraudulent—if your model predicts "non-fraudulent" 100% of the time, you’ll get 95% accuracy but miss all the critical fraud cases! When it comes to the F1-score: - Precision tells you how many of your positive predictions were correct. - Recall tells you how many of the actual positives you correctly identified. The F1-score balances both precision and recall, giving you a better sense of how well your model performs—especially when false positives and false negatives matter. If you're working on models where missing a positive instance could have serious consequences (like healthcare, security, or finance), the F1-score should be your go-to metric. Focus on the right metrics to make smarter models and better decisions. #MachineLearning #DataScience #AI #F1Score #ModelEvaluation #TechInsights #DataDriven #BrainK

10