Mohit Rajpoot

As models grow larger with the rise of LLMs, choosing the right precision type has become important. Sharing some insights over FP16 vs BF16:   🔷  FP16 and BF16 both consume same memory.  🔷  FP16 has better precision but lower dynamic range. In fp16, largest number is 65,504. BF16 can go upto 3.39e+38 !  This is because fp16 has higher # mantissa bits and bf16 has higher #exponent bits. 🔷  For model training, you will have to experiment and understand whether your network is sensitive to range or precision. If a model requires more dynamic range, it will converge better with bf16, which has same range as fp32. However, some models will converge better with higher precision (fp16) 🔷  When finetuning an fp16 model pre-trained in bf16, overflow issues may arise as some of the bf16 model weights might be larger than 65k.   Image source: nvidia   #precisiontype #fp16 #bf16 #modeltraining #LLMs #machinelearning #ai #datascience #torch

٣٠

Question : Why does quoting just the accuracy numbers do not make sense for an AI model❓ Its a common practice among data science practitioners to quote accuracy of an #AI model. While it is easier to understand accuracy numbers, this practice can be misleading and fail to capture the nuances of real-world scenarios, leading to misaligned expectations among business stakeholders. 👉 Accuracy alone does not provide a comprehensive picture of a model's capabilities. It fails to account for crucial factors such as the size and distribution of the training, testing, and validation datasets, the presence of class imbalances, and potential biases. Relying solely on accuracy can mask underlying issues and lead to false confidence in the model's performance. 👉 To truly understand a model's effectiveness, it is essential to consider a broader range of metrics. Precision, recall and F1 score offer valuable insights into a model's ability to correctly identify positive and negative instances, handle imbalanced data, and discriminate between classes. 👉 However, even these technical metrics may not resonate with business stakeholders who are primarily concerned with the practical implications of the model's performance. It is crucial to translate these metrics into business-relevant terms that directly align with the product's objectives and success criteria. For example, in a fraud detection system, precision could be framed as the percentage of flagged transactions that are indeed fraudulent, while recall could represent the proportion of actual fraudulent transactions correctly identified. By presenting these metrics in a business context, stakeholders can better understand the trade-offs and make informed decisions about the model's deployment and ongoing monitoring. 💡 Quick Tip: Dont quote accuracy numbers without mentioning the size of training and testing dataset. Consider a diverse set of metrics tailored to your specific use case and business objectives. Communicate these metrics in a language that resonates with stakeholders, fostering a shared understanding and aligning expectations. #artificialintelligence #productmanagement #datascience #productmanager

٦

Question : Why does an AI product manager need to monitor #AI model❓ Once AI model is developed and deployed, the job doesnt end there. Model encodes pattern of the data it has seen during training phase. There may be several patterns or edge cases which are not part of training data. If model encounters any such patterns in testing or production phase, it will produce results which wont make sense without giving any warning about it. 👉 Data drift is a common challenge faced by AI-powered products. Over time, the distribution of input data can shift, leading to a mismatch between the model's training data and the real-world data it encounters. This can result in degraded model performance, unexpected outputs, and even complete model failure. 👉 The consequences of data drift can be severe, ranging from suboptimal user experiences to regulatory compliance issues. Product managers must be vigilant in detecting and addressing these challenges to ensure the continued success of their AI-powered offerings. 👉 Sudden changes in metrics can indicate input data drift. Additionally, monitoring the distribution of model outputs can reveal output drift, which may signal a need to retrain or fine-tune the model. Detecting data drift early is crucial. Product managers can leverage techniques such as A/B testing, anomaly detection, and continuous monitoring to identify and address these issues. Once detected, the product team can take appropriate actions, such as retraining the model, adjusting the data pipeline, or even considering a model architecture change. 💡 Quick Tip: Integrate model monitoring into your product roadmap and allocate dedicated resources to ensure the long-term success of your AI-powered offerings. Establish a comprehensive model monitoring dashboard to gain real-time visibility into the health and performance of your AI models. #artificialintelligence #productmanagement #datascience #productmanager

٥

📹 🤖 Not sure what to watch? Video recommendation to the rescue! #Recsperts Episode No. 24 on Video #recommendersystems with Amey Dharwadker from Meta is out now Amey Dharwadker is a Machine Learning Engineering Manager at Facebook and he is leading the Video Recommendations Quality Ranking team. In this episode, Amey sheds light on the intricate challenges of delivering #personalized video feeds at scale. With a vast and ever-growing corpus of billions of videos—millions of which are added each month—the cold start problem looms large. Our conversation covers content understanding, user interaction data and feedback aggregation, #exploration, and #evaluation techniques. A key highlight of the conversation is the importance of #realtime #personalization in fast-paced environments, such as short-form video platforms, where user preferences change quickly. Amey also emphasizes the value of cross-domain data in enriching user profiles and improving #recommendations. We also touch on the past and upcoming VideoRecSys workshop and what constitutes a great recommendation #team. ✅ Subscribe on your favorite podcast player ⭐ Leave a rating or review ✍ Share your thoughts on Video Recommender Systems Listen to this new episode of #Recsperts - Recommender Systems Experts! 🎧 Website: https://lnkd.in/eHNe2QJs 🎧 Spotify: https://lnkd.in/eApYQg_n 🎧 Apple: https://lnkd.in/evzdWCXA

٣٢

Awesome companies #hiring Data Scientists 1. Middesk (We make software to build trust between businesses) Hiring: Data Scientist 2. Reframe (App to help people quit or cut back on alcohol use) Hiring: Data Scientist 3. Veryfi (We Liberate End Users from Manual Data Entry.) Hiring: Data Scientist - Fraud Detection 4. Aer Compliance (Compound compliance: everything you need to keep your firm safe in a single place.) Hiring: Senior Data Scientist, Crypto 5. Netflix Hiring: Data Scientist 6. National Basketball Association (NBA) Hiring: Data Scientist, Game Analytics & Strategy 7. Twitch Hiring: Data Scientist 8. Spotify Hiring: Data Scientist II - Trust & Safety 9. SynergisticIT (The Best Programmers in the Bay Area.....Period!) Hiring: Junior Data Scientist 10. Coinbase (We're building an open financial system for the world.) Hiring: Data Scientist Follow me for more posts like this. If you're hiring and want to post on MissionHires for free, please DM me.

١٧

Question : Responsible AI deployment - what is that and why should an AI product manager bother about it❓ Ethical development of AI products is not just about algorithmic transparency or handling sensitive user data. One critical aspect of responsible AI deployment is the need for ongoing monitoring and evaluation. 👉 Product managers should establish robust mechanisms to continuously assess the performance and impact of their AI models, ensuring they remain aligned with intended use cases and do not exhibit unintended or harmful behaviors over time. 👉 Product managers must consider the potential for AI systems to amplify existing societal biases or create new forms of discrimination. Proactive measures, such as conducting bias assessments and implementing fairness-aware algorithms, are essential to mitigate these risks and promote inclusive and equitable AI solutions. 👉 Transparency and accountability are also crucial pillars of responsible AI deployment. Product managers should strive to provide clear and accessible information about their AI models, including their capabilities, limitations, and decision-making processes. 💡 Quick Tip: Track your model performance not only from technical point of view but also from ethical, social and regulatory implications of your AI product. #ArtificialIntelligence #productmanagement #productmanager #DataScience 

٧

This is a major development for the open-source AI ecosystem. Apple has gone fully open-source! They have just released DCLM, a 7B open-source LLM, including weights, training code, and dataset. They plan to open-source all their models for deployment across their billion devices. Apple's embrace of open-source marks a significant shift. Here’s a summary: → Apple has released and open-sourced their DCLM 7B model, outperforming Mistral/Qwen2/Gemma on various benchmarks. → It is primarily trained on English data with a 2048 context window, and licensed under the Apple Sample Code License. → The release includes an exceptional explanation of the data curation process, offering valuable insights into LLM training. → DCLM is trained on 2.5T tokens using the DataComp-LM (DCLM) framework, with datasets from Common Crawl. → DataComp for Language Models (DCLM) is a testbed for controlled dataset experiments aimed at improving language models. → The DCLM Framework is designed for future experiments with data curation strategies like deduplication, filtering, and data mixing. Why is this significant? Apple's AI strategy now includes open-sourcing all their models. This shift, alongside their recent on-device AI announcements at WWDC, indicates a move toward smaller models and open AI. The future of AI is looking increasingly open. Resources: Model: https://lnkd.in/grgNegcN Repository: https://lnkd.in/gzcTsxqY Dataset: https://lnkd.in/gqY_U2x5

٤٦

F1-Score vs Accuracy: Why F1-Score Should Be Your Focus. In machine learning, it's tempting to focus on ACCURACY—the percentage of correctly predicted instances. But what if I told you that accuracy isn't always the best indicator of your model's performance? In cases where the data is imbalanced (e.g., fraud detection, disease diagnosis), accuracy can be misleading. Imagine a scenario where only 5% of transactions are fraudulent—if your model predicts "non-fraudulent" 100% of the time, you’ll get 95% accuracy but miss all the critical fraud cases! When it comes to the F1-score: - Precision tells you how many of your positive predictions were correct. - Recall tells you how many of the actual positives you correctly identified. The F1-score balances both precision and recall, giving you a better sense of how well your model performs—especially when false positives and false negatives matter. If you're working on models where missing a positive instance could have serious consequences (like healthcare, security, or finance), the F1-score should be your go-to metric. Focus on the right metrics to make smarter models and better decisions. #MachineLearning #DataScience #AI #F1Score #ModelEvaluation #TechInsights #DataDriven #BrainK

١٠

LLMs ecosystem is brimming with innovation and churning developer platforms and consumer products at amazing pace. As an AI enthusiast and data science practitioner, here are some of my favorite products and frameworks: Cursor AI - If a single product has convinced me about the power of LLMs, it has to be Cursor AI. Think of it as Copilot on steroids. It can enable you to ask question on your entire codebase, write whole code modules using prompts, write test cases for your code base. People have been sharing demos where they built E2E consumer app and deployed it within few hours with Cursor AI. I have dabbled a bit but I am going to subscribe now and indulge. Notebook LM - I find this free tool from Google ultra useful. Somehow it hasn't got the recognition it deserves. You can upload large PDFs and ask questions out of it. I have tried on PDFs with 100+ pages and answers are spot on with citations. You can convert the responses to audio too. Anthropic - Claude from Anthropic was one of the first LLM products I subscribed to. I am fascinated with its coding abilities and love its artifact feature where it generates a code or graphic illustration to explain something. It taught me the rules of American Football much better than many youtube videos. Anyscale - The problems of latency and cost optimization of LLMs are close to my heart. Anyscale has created frameworks to perform distributed training and inference on cluster of GPUs and faster inference using vLLMs. Unsloth AI - As the name suggests, this framework makes full training (from scratch) and supervised and instruction fine tuning of LLMs much faster by writing GPU kernels for faster calculations. CrewAI - AI Agents are the new rage in the town. I find them quite compelling too. There are other frameworks like Autogen from Microsoft for this but I like CrewAI for its minimalistic signature to create and execute agents and comprehensive integrations unstructured.io - Their motto is to make any document RAG-ready. They solve for the optimal chunking problem in the RAG pipelines. Wonderful example of solving a very specific but critical problem. TrueFoundry - TrueFoundry is great at recommending the best GPU(s) for hosting open source LLMs. It promises cloud cost optimization on training and inference with several optimizations like vLLMs, Flash Attention and others. I also quite like the RAG-as-a-service which can be converted to APIs automatically. Greptile - Greptile promises to automate the PR workflow for code check-ins. It provides comments on PR quality and its readiness to merge to codebase. It also generates test cases based on PR Pinecone - My go to vector database. I like facility of storage/latency optimized pods and namespaces to create multi-tenant RAG applications. Significant mention - Langchain, Perplexity

٩٢

٤ تعليق

Perhaps the most critical thing in a Retrieval Augmented Generation (RAG) system is the retrieval or searching strategy. Poor retrieval will lead to irrelevant documents going to the LLM leading to irrelevant generated answers or worse hallucinations. Is cosine similarity everything? Do check out below illustration which talks about four of the most common strategies for playing around with retrieval strategies in RAG systems: 1. Similarity-based retrieval: This is based on embedding similarity, you can play around with different metrics like cosine, maximal margin relevance etc. 2. Similarity with Threshold retrieval: Useful where we cap the similarity threshold and can help in removing irrelevant document but can drop recall 3. Multi-query retrieval: Make the LLM generate variations of your query and get documents based on these multi-versions of your initial query, often useful to get more recall or coverage of documents 4. Reranker retrieval: Use a model specialised in ranking documents based on search relevance Often you can combine some of these techniques and even use ensemble retrievers. Do explore them and go beyond just cosine similarity!

١٧

𝑻𝒊𝒑𝒔 𝒇𝒐𝒓 𝒏𝒆𝒘 𝑫𝒂𝒕𝒂 𝑺𝒄𝒊𝒆𝒏𝒄𝒆 𝑴𝒂𝒏𝒂𝒈𝒆𝒓𝒔 I became a manager in data science early in my career. It has been a journey fraught with failures and some successes. Here are some of my learnings along the way - 📈 𝑹𝒐𝒍𝒆, 𝒓𝒆𝒔𝒑𝒐𝒏𝒔𝒊𝒃𝒊𝒍𝒊𝒕𝒚 𝒂𝒏𝒅 𝒑𝒓𝒊𝒐𝒓𝒊𝒕𝒊𝒆𝒔 Early on, my vision was small which narrowed the value I was able to create. I had two focuses - 1) ship out high quality work, 2) coach & promote my team. After >10 years in leadership roles, here is how I NOW think of the responsibilities of a data science manager in the hierarchy of importance - 🔹 Build authentic relationships with stakeholders and team members 🔹 Identify and align on highest impact work the team can enable and unlock 🔹 Set up systems, processes, and people to execute on these projects 🔹 Manage projects and priorities 🔹 Coach your direct reports on execution Depending on the seniority of the role, the distribution across these may vary but the core buckets remain the same. 🏆 𝑩𝒆 𝒊𝒏 𝒕𝒉𝒆 𝑨𝒓𝒆𝒏𝒂 Don’t be carried away by the business of meetings that make you feel important but don’t create real value. Spend at least 1/3rd of your time towards hands-on work. Don’t get rusty on the technical chops. Be an SME in several areas in the workflow - tech stack/tools, data models, methodologies, algorithms and specific tables. This will build trust with your reports while reducing dependency, increasing team efficiency and productivity. 🔄 𝑻𝒂𝒊𝒍𝒐𝒓 𝑷𝒓𝒐𝒄𝒆𝒔𝒔𝒆𝒔 & 𝑷𝒓𝒐𝒅𝒖𝒄𝒕𝒔 Data science lacks standard processes, unlike software engineering, due to its evolving role, inherent variability in methods depending on project goals. Creating processes and systems that will help with the ground reality of the ecosystem and culture at your company, is important. 🎲 𝑼𝒏𝒅𝒆𝒓-𝑷𝒓𝒐𝒎𝒊𝒔𝒆 𝒂𝒏𝒅 𝑶𝒗𝒆𝒓-𝑫𝒆𝒍𝒊𝒗𝒆𝒓 In the unpredictable world of predictive modeling 😉, where factors such as data anomalies, algorithm biases, and unexpected market shifts can disrupt the accuracy of forecasts, it pays to temper expectations and try to exceed them. #datascience #analytics #businessanalytics #leadership #techleadership #careers #techmanagers

٢٨

١ تعليق واحد

GPT4o for voice typing ------------------------------------- So far, accuracy with Sellinam, Google voice typing for #tamil is not 100%. After spelled, it requires edit. Tried #GPT4o to output the spoken words in Tamil. It is accurate and doesn't require any hard edit. கேட்பதும் நீ மறுப்பதும் நான் கெஞ்சுவதும் நீ இளகுவதும்... Disclaimer - Haven't tried complex sentences yet. Haven't used any other LLM for voice typing prior.

٩

Long-context LLMs are outperforming RAGs, so will RAGs be dead? 🤔 In this post, you will learn: 1. Comparison of RAG and LC LLMs 2. Performance - Cost Trade-off 3. Self Route: A new approach to bridge the gap What are your views on this research? Let me know in the comments! --------------------------------------------------------------------------- I hope this post helped you! Help people in your network by liking and reposting it ❤️ It inspires me to do more for our community. Follow Bhavishya Pandit to stay updated on GenAI! 🔥 #ml #llm #ai #genai #datascience #artificialintelligence #opensource #ibm #generativeai #largelanguagemodels #huggingface #openai #google #rag #machinelearning #python #langchain #microsoft #genai

٥١٧

٤٩ تعليق

There is a gloom simmering in the data science industry. Confetti is flowing in the wind of generative AI. But at the same time an entire batch of freshers are anxious about their placements. Hiring has been extremely low across the board. Irrespective of tier-1, 2 or 3 campuses. These days whenever any student enquires about our courses, they mainly try to understand whether we provide "placement assistance" or not. Meaning if we will be able to find them a job or not. This is a great indicator to understand the overall mood of the batch. They're sadly, desperate for an opportunity. Looking at the industry from this vantage point breaks you a little bit. So from last two weeks I've been doing research on the topic. Talking to several industry veterans, hundreds of students, reading. And the conclusion is simple and worrying. AI adoption in everyday life is the future. We're moving towards an age where "internet of things" will be a reality pretty soon. Most of the work that can be automated will definitely be automated. As we'll be increasingly a more capitalistic society, most of the necessities will be controlled by companies. And as we all very well know, automation is an effective cost cutting measure. So many people will loose jobs, a lot of people will have to learn new skills to adjust, payouts will follow a leptokurtic distribution. Overall it'll be a gradually more competitive and skill-based economy. Harder to thrive in. Even though it might sound like doom. Life always finds a way. A different way for sure. But definitely a way. Which frankly I don't know yet. I'm planning to read more about this topic. it's a ocean of knowledge to consume. I'm in the water but still near the bay. What you'll see soon is something beautiful, that has come of these research. Till then please Stay Tuned. #lastbench #resumereview #coverletter #interviewpreparation #interviewtips #placement #placementassistqnce #datascience #analytics #machinelearning #careercoach

١٣٤

٢ تعليق

India's AI Opportunity: Building Applications on Top of Large Language Models 🇮🇳🚀 In an exclusive interview with The The Economic Times, Andrew Ng, a prominent figure in the AI world, shared his insights on India's potential in the AI landscape. Here are the key takeaways: • India's biggest AI opportunity lies in building applications on top of large language models (LLMs) and foundation models. • While training LLMs is capital-intensive, the real value lies in the application layer, which is more accessible to startups and enterprises. • India should focus on sectors where its economy is already strong, such as IT services, manufacturing, agriculture, and finance, to maintain its advantages through AI applications. • The generative AI moment has leveled the playing field for startups and enterprises, with India standing at the verge of opportunities. • Ng's AI Fund invests in startups building AI applications, with an average of one investment per month. They have also invested in an Indian healthcare AI startup, Jivi Health. • Ng believes that with the rise of AI, learning to code will become a significant personal productivity booster, similar to language literacy. #AI #MachineLearning #DeepLearning #IndiaTech #Startups #TechTrends

١

Why Multimodal RAG is Important? 🔹It provides more accurate and comprehensive information. Multimodal RAG models combine the power of information retrieval and generative language models to produce answers that are not just accurate but also contextually enriched. This is because they can access and process a wider range of data sources, including text, images, and videos. 🔹It is more adaptable to different domains and tasks. Multimodal RAG models can be easily customized to provide information on a wide range of topics by leveraging different knowledge sources. This makes them ideal for use in a variety of applications, such as customer service, education, and research. 🔹It improves the overall user experience. Multimodal RAG models can provide more informative, reliable, and relevant responses, which can lead to a better user experience. This is because they can take into account the user's context and preferences, and they can provide explanations and citations for their answers. Check out this super interesting new course from Deeplearning AI in partnership with Weaviate. https://lnkd.in/d3H6tFsf

٣٠

١ تعليق واحد

Question : Sampling Techniques : What are they❓ As we saw recently in exit polls on Indian elections, choosing samples from a population can have a significant impact on predictions. #AI enthusiasts need to have a solid understanding of statistical sampling techniques. Statistical sampling is essential for AI product managers because it helps them: 1️⃣ Understand the Target Population: Accurate sampling ensures that the data used to train and evaluate AI models is representative of the target population, reducing the risk of biased or skewed results. 2️⃣ Optimize Data Collection: Sampling techniques can help AI product managers determine the appropriate sample size and sampling method, ensuring efficient and cost-effective data collection. 3️⃣ Validate Model Performance: Sampling is crucial for testing and validating AI models, as it allows for the creation of representative test sets that accurately reflect the real-world scenarios the model will encounter. AI enthusiasts should be aware of the different types of sampling methods, including: 🔺 Probabilistic Sampling Techniques: methods where randomisation is used instead of deliberate choice. 👉 Simple Random Sampling: Each member of the population has an equal chance of being selected. 👉Stratified Sampling: The population is divided into subgroups (strata), and samples are drawn from each stratum. 👉Cluster Sampling: The population is divided into clusters, and a random sample of clusters is selected. 👉Systematic Sampling: Every nth member of the population is selected, starting from a random point. 🔺 Non-Probabilistic Sampling Techniques: methods where some selection criteria exists 👉Convenience Sampling: Selecting participants based on their availability and accessibility. 👉Quota Sampling: Selecting participants to match predetermined proportions in the population. 👉Purposive Sampling: Selecting participants based on specific characteristics or criteria. 👉Snowball Sampling: Selecting participants through referrals from initial participants. The choice of sampling method depends on the specific goals of the AI project, the available resources, and the characteristics of the target population. Choosing the wrong sampling technique can lead to biased results, skewed data, and ultimately, the development of AI models that fail to perform well in real-world scenarios. 💡 Quick Tip: When selecting a sampling technique, consider the diversity and representativeness of the target population, the level of precision required, and the resources available for data collection.  #AI #ArtificialIntelligence #productmanagement #productmanager #DataScience 

٧

Those who still think ChatGPT cannot code like pro Today I asked ChatGPT to write a medium complex task that would take 2 audio files and merge specific segments of audio Got step by step instructions on which python library to install and code as well I got a error and asked ChatGPT to fix the error It pointed the error to exact location of the open source package and asked to change the function in open source package with a modified one. Note: In this case it is not only using the library but also pointing error within the library and with updates to it I got a different error and again asked ChatGPT to help Again responded back asking me to use a new library and got new code Ran it and worked like a charm All this in around 10 min without going to Google, Stackoverflow and github issues for that package On top of it this is just the beginning... #chatgpt

١٬٧٢٩

١٩٣ تعليق

Another important point to consider when interpreting PR AUC values is that they are typically lower than ROC AUC values. This difference becomes clear when we understand the baseline metrics for each curve: Baseline ROC AUC: For a random model, the baseline ROC AUC is always 0.5. This reflects the fact that a random model produces a diagonal ROC curve (y = x), where the TPR equals the FPR across all thresholds, indicating no discriminatory power. Baseline PR AUC: For a random model, the baseline PR AUC equals the fraction of positive examples in the dataset. It is a good exercise to think about why the baseline PR AUC equals the fraction of positive examples in the dataset. So, for example, a model with ROC AUC = 90% and PR AUC = 70% can still be excellent for imbalanced datasets; the lower PR AUC just reflects the challenge of predicting the minority class accurately.

٤