Nandini R

Fanning out queries across multiple shards makes it expensive - this is a common understanding but it is not entirely true ⚡ While going through the internals of Rockset, a real-time analytics database, I found they are fanning out queries across nodes, but still offer very low latency response times. My dissection video on how they execute the query in their distributed setup while being able to scale horizontally, along with some performance optimization nuances is dropping tomorrow. hit the notify button - youtu.be/HWeTVy5i9vE By the way, this will be the 3rd video in the Rockset internals series and I dissected and researched through 18 of their blog posts and 2 whitepapers. Exploring the details was fun, and I am sure you will love it too. Here are the first two videos of the series 1. Architecture deep-dive - youtu.be/I4eEEAgoCig 2. Converged Indexes - youtu.be/xGaUJTHuehQ ⚡ I keep writing and sharing my practical experience and learnings every day, so if you resonate then follow along. I keep it no fluff. youtube.com/c/ArpitBhayani #AsliEngineering #SystemDesign #DatabaseInternals

170

2 Comments

With Sarath Shekkizhar, we are exploring the geometric structures behind LLM reasoning capabilities. So far, our findings suggest: 1. Higher density in self-attention graphs implies higher intrinsic dimension. 2. Higher intrinsic dimension induces more expressive power/reasoning with MLPs. 3. Longer context and more attention heads can boost intrinsic dimension. This geometric lens offers new insights into LLM performance and potential improvement strategies. Early days, but we believe it is a promising direction! Stay tuned for more as we dive deeper into the geometry of AI reasoning. Read our pre-print: https://lnkd.in/gcekSWYD #AI #MachineLearning #LLM

25

On Device frameworks are the way to go on a tragectory of personalized AI agents and the below post by Vaibhav Srivastav shows the great progess made in the field which only increases the enthusiasm!! 🎉👏🏻👏🏻

3

Looking for recommendations for good hindi-english (hinglish) code-mixed speech to text models/corpuses. Anyone building anything with speech in the Indian context probably realized normal ASR systems fall flat, losing >50% context because of their inability to handle hinglish conversation (which is extremely common for a huge chunk of modern India). See the title of the first video when looking for shark tank. Once you start noticing it, its hard to unnotice. If speech to text doesn't work, every downstream task thas cascading failures. I'm not looking to build anything commercially so hopefully #research corpuses will also be okay. Literally only want to progress speech for India and this is a major blocker. This is one of those areas where other countries can't innovate for us and hand us a solution unfortunately. #speech #asr #codemix #hinglish #stt

3

1 Comment

Many of ML practitioners might have heard the term "Extending context length" for LLMs. In case if you want to dive deeper into this topic along with its code implementation, give this blog a read. I have described about Position Interpolation, NTK aware and Dynamic NTK method, which are widely used for extending context length.

68

🤔 Interesting perspective from Infosys Chairman Nandan Nilekani: "Let Silicon Valley build LLMs; India should lead in AI use cases." At Meta's Build with AI summit, Nilekani suggested that India should focus on practical AI applications rather than competing in the LLM space. But is this the right approach? Key points to consider: • Silicon Valley giants are heavily invested in LLM development • India could focus on: - Synthetic data generation - Small language models - Real-world applications - Building robust data infrastructure The vision is for India to become the global "use case capital" for AI, emphasizing frugal innovation and practical solutions. What do you think? 💭 Should India stick to developing practical AI applications? 💭 Or should we also invest in fundamental AI research and LLM development? 💭 Could focusing solely on use cases limit our AI capabilities in the long run? Let's discuss! Share your thoughts in the comments below. 👇 #AI #DigitalIndia #LLM

29

Can we parallelise model training like transformers yet scale linearly to long sequence length just like RNN? 🤔 Yup, its possible with SSMs, infact they can do inference at computation cost of O(1). 🤯 First of all, how does it even compare with the most advanced Transformer capabilities like, - Self-Attention Mechanisms - Multi-Head Attention - Quadratic Scaling, etc. All of these have been outperformed on the new benchmarks across various domains such as language, audio, and genomics by Mamba. 🏆 Your model might be excelling with the above methods, but does it handle million-length sequences efficiently? Or process long sequences without breaking a sweat? Mamba efficiently integrates a selective state space model. What it effectively does for sequence processing is: ⚡ - Dynamically selects relevant information while discarding irrelevant data. - Operates with linear time complexity, ensuring fast and efficient processing. - Utilizes a hardware-aware parallel algorithm that maximizes GPU efficiency. - Incorporates simplified end-to-end neural network architecture without attention or MLP blocks. And during sequence modeling, - The input is processed through a selective mechanism, allowing the model to focus on the most relevant parts. - The state space parameters (such as Δ, B, and C) are dynamically adjusted based on the input, enhancing content-aware processing. - Efficient recurrent mode computation ensures faster training and inference with lower memory usage. But this approach doesn't just stop at sequence processing; it extends to real-world applications across different domains. For example: - Language modeling with Mamba outperforms Transformers of similar and even larger sizes in both pretraining and downstream evaluations. - Audio modeling with Mamba provides better quality and efficiency, making strides in speech recognition and generation. - Genomic sequence analysis with Mamba achieves state-of-the-art performance, handling long DNA sequences with ease. Mamba is not just an architectural improvement; it's a revolution in sequence modeling. I'll link the research paper down in comments, and you can explore its capabilities further. For more such detailed explanations, follow me. See ya! . . . #Mamba #SequenceModeling #AI #NeuralNetworks #Efficiency #StateSpaceModel #LanguageModeling #AudioModeling #Genomics #DeepLearning

24

📍 💡 Need some visual inspiration? #Recommendations to foster new ideas and explore the world. #Recsperts Episode No. 22 with Prabhat Agarwal and Aayush Mudgal from Pinterest is out now! Prabhat is a Senior ML Engineer and Aayush is a Staff ML Engineer. Prabhat works on #recommendations and #search systems at Pinterest, leading representation learning efforts. Aayush is responsible for #ads #ranking and privacy-aware conversion modeling. In our interview, Prabhat takes us on the journey of #personalizing the Pinterest Homefeed. We discuss how to capture users' short- and long-term preferences through multiple #embeddings and the role of candidate generators for content diversification. Aayush dives into the specifics of ads ranking at Pinterest and helps us to better understand how multifaceted ads can be. He also shares some insights about #exploration and corresponding randomization in the context of ads and how user behavior is very different between different kinds of ads. Both highlight the role of #counterfactual evaluation and its impact for faster #experimentation. ✅ Subscribe on your favorite podcast player ⭐ Leave a rating or review ✍ Share your thoughts on Pinterest Homefeed Personalization and Ads Ranking Listen to this new episode of #Recsperts - Recommender Systems Experts! 🎧 Website: https://lnkd.in/gVP5Ar_B 🎧 Spotify: https://lnkd.in/gPyguxY3 🎧 Apple: https://lnkd.in/gDCJjFzw

48

5 Comments

I've recently been replicating some of our released work on LLM with MLX. I enjoy MLX for a variety of reasons. Minimalist is one of them. MLX's API is a subset of numpy and much less than PyTorch's. There was a moment when I thought the API was insufficient to support beam search and tree attention. However, as I moved along, I realized that this simple API is just as powerful as the ones on stage. Rethinking how to rewrite our PyTorch programs in MLX often resulted in simpler, more readable code. For example, the following two lines generate attention bias in PyTorch (https://lnkd.in/gszdymfE):   attn_bias = torch.zeros(causal_mask.shape, dtype=dtype, device=causal_mask.device)   return attn_bias.masked_fill_(causal_mask.logical_not(), LOG_0) MLX lacks masked_fill_. So I rewrote the above as a single line:   return mx.logical_not(causal_mask) * LOG_0 Yes. It takes time to rethink and rewrite code. However, if this work helps to keep MLX's API simple, it eases the learning process for others. I appreciate a small set of primitives. We would benefit from higher layers of APIs built on top of it, rather than a large flat set of primitives.

82

2 Comments

🚀 Making LLMs work for you - A Guide to Become 10x Programmer/MLE/DS I am sure many of you use LLMs for your day-to-day work. Mostly, I have seen people use LLMs while writing code (Copilot), or as an alternative to Copilot to catch bugs, but they have become much more useful than that, provided you know how to use them well. Here are a few ways I have been using LLMs in my daily work that are helping me increase my productivity: 📚 1. Quick Paper Assessment with ELI5 Summaries - Paper abstracts are called abstracts for a reason. Asking an LLM for an ELI5 summary makes it much easier to know if I want to read a paper. 💻 2. Implementing Research Papers in Code - Try giving an LLM like Claude/ChatGPT a research paper and ask to code it. You would be surprised. Tip: Claude performs much better for this task. 🔄 3. Upgrading Code Structure and Readability - Do you want to add typing hints when writing your Python code, or do you need comments in your code, or do you have a big function that you want to be converted to classes in your code? LLMs can do it all, and you would be amazed at the code-conversion power they have. ⚡ 4. Optimizing Code Performance - The thing with LLMs is that if you tell them what to do, they can do it well. For example, if you give them a non-optimized piece of code that runs correctly, they will be able to write it in a much-optimized way. For example, I just used an LLM to convert a matrix multiplication in PyTorch to sparse MM as I was getting an OOM memory error because one of the matrices became too large. I am not even joking, but that would have taken at least 1-2 hours for me to come up with. 📝 5. Automated Documentation - Got bored of writing documentation for what you did but have the code? LLMs also work the other way around. Want to write a summary of what you did in a week, you can use an LLM for that. Saves so much time that gets wasted in formatting. 🧪 6. Streamlined Test Framework Creation - Let LLMs handle the boilerplate for Unit tests & Integration tests while you focus on core logic. 🌐 7. Language-Agnostic Development - You can paste a function in Python and ask for its equivalent in Rust, complete with language-specific best practices and error handling. This dramatically reduces the learning curve when working with new languages and helps maintain productivity during language transitions. 🤝 Apart from all of the above, LLMs work well as mentors as well as interns. LLMs are great when you just need someone to bounce ideas with. I sometimes just talk to an LLM as I would to a colleague talking about the problems I have and how I can solve them. And they come up with the right ideas or send me in the right direction a lot of times. They also act as a personal assistant for free, who can deal with writing messages, creating charts, and doing all sorts of uninteresting work for you. 💭 What's your favorite way to use LLMs? Let me know in the comments! #MachineLearning #AI #Programming

47

Bhaskar Lalwani and I (Aniruddha Mukherjee) had worked to create kabaddiPy recently. kabaddiPy is an open-source repository for Kabaddi data - a first step towards making this fast-growing, dynamic sport more accessible for fans, analysts, and researchers. Kabaddi is the second most-watched sport in India after cricket, yet its data is not as widely available. We hope to be the trailblazers and change this! Whether you're into sports analytics 📊, passionate about Kabaddi 🤼‍♂️, or simply love data, we invite you to: ⭐ *Star the repo* 🔧 Contribute to the data 💡 Explore insights (by building predictive ML models / more!) Open an issue and let's chat! Let’s work together to build something great for India 🇮🇳 and the Kabaddi community! Link to the repo: https://lnkd.in/gGCnY_B4 Link to docs (for an overview): https://lnkd.in/g-Khu5_J btw, we're a part of GSSoC Extended and Hacktober-Fest as well! #opensource #kabaddi #sportsanalytics #dataanalysis #ProKabaddi

79

1 Comment

Last November the news was on fire, with speculation that an OpenAI model called Q* was getting close to AGI. Today Adam Selipsky announced their new product Amazon Q, coincidence? I think not. This used to be a product called CodeWhisperer. But it looks like it's merged into a product suite under the name Q. The website for CodeWhisperer had an announcement listed: "Amazon CodeWhisperer is now Amazon Q Developer, which is generally available. All of the functionality that CodeWhisperer provides is part of Amazon Q Developer – in-line code suggestions, security scanning, open-source license tracking, and more. This website will remain live until mid May. For the latest information and to learn more about Amazon Q Developer, visit the website." Any how, it looks like the play is to connect to internal data sources and employees to make generative AI apps based by their own apps. They note in their FAQs "When you sign up for Amazon Q Developer Pro or Amazon Q Business plans, we don't use content to improve underlying models for others. Your data is your differentiator." But it doesn't look like there are separate instances of the model but rather a super-model (yeah I made that pun). So if you trust that your data won't get integrated than you should be fine. Just like the trainwreck you couldn't look away from on the news. 🙄 P.S. 👍 If you found these insights useful, please like this post to support more content like this! 🔁 Feel free to share with your network. P.P.S. Subscribe to my newsletter, The Artificially Intelligent Enterprise, for weekly insights into how to succeed with AI in Business - https://lnkd.in/eqJNArYx

211

17 Comments

🎓 Is a Masters in USA in Computer Sciene/Data Science still worth it? There is a lot of speculation around this question, with clickbait news coming up with random facts and suggestions. The economy and job market has been tough in US and across the world in 2024 and there is uncertainties around visa and work permit. Lots of student are yet to land an internship or job offer with surmounting pressure of visa validity. I wanted to share some ground truths based on personal experience helping family members and friends currently navigating this journey. 📚 Education's True Value The worth of a Masters degree transcends job prospects - it's about deep subject matter expertise and intellectual growth. 🌟 Quality Matters US universities continue to offer world-class research facilities and educational excellence - choose institutions aligned with your specific interests and career goals. 💼 Career Perspective While Big Tech opportunities are attractive, don't make them your sole focus. Many tech giants now have robust operations in India. Evaluate if your goals can be achieved without a Masters in US. 🔍 Beyond Academics Your degree is just one part of the equation. Focus on: Building professional networks Participating in industry events Contributing to research Engaging in hackathons Exploring potential PhD paths 🚀 Pre-Masters Preparation Leverage online resources (MOOCs, YouTube) to build foundational knowledge before your program begins. This gives you a competitive edge for internships and job placements. 💰 Financial Planning Be realistic about the investment. While US opportunities are valuable, maintain a global perspective on career possibilities, including emerging markets. If you're passionate about your field and have done thorough research, don't let market uncertainties deter you. Success comes from strategic planning and staying adaptable. #HigherEducation #MastersDegree #STEM #InternationalEducation #CareerGrowth #USEducation #TechCareers #GlobalOpportunities #StudentLife #CareerAdvice #IndianStudentsAbroad #StudyAbroad #Education2024 #TechEducation #CareerDevelopment

14

Those who still think ChatGPT cannot code like pro Today I asked ChatGPT to write a medium complex task that would take 2 audio files and merge specific segments of audio Got step by step instructions on which python library to install and code as well I got a error and asked ChatGPT to fix the error It pointed the error to exact location of the open source package and asked to change the function in open source package with a modified one. Note: In this case it is not only using the library but also pointing error within the library and with updates to it I got a different error and again asked ChatGPT to help Again responded back asking me to use a new library and got new code Ran it and worked like a charm All this in around 10 min without going to Google, Stackoverflow and github issues for that package On top of it this is just the beginning... #chatgpt

1,729

193 Comments

A recent discovery in the LLM research space discovered that a good chunk of transformer blocks are being used simply as linear layers by the LLMs The intriguing discovery has open up new potentials of model compute optimizations and model performance enhancements. I have tried my level best to put a medium article (due to content size limitation in linkedin) to explain the discovery and potentials findings in a easily understandable manner The article is free for everyone to read, give it a shot https://lnkd.in/gGsASB3z #AI #DeepLearning

31

It is well understood that when generating text, Large Language Models (LLMs) predict multiple possible next words, each associated with a probability. An effective decoding strategy is essential for selecting the next token from the probability distribution, ensuring coherent text generation. For an insightful discussion on the various decoding strategies employed in LLMs, explore the video "LLMs: Hands-on Decoding Strategies", which delves into key techniques like greedy search, beam search, top-k sampling, and nucleus (Top-p) sampling. LLMs: Hands-on Decoding Strategies --- https://lnkd.in/gVk-5Rim #LLMs #AI #MachineLearning #NLP #DecodingStrategies #ArtificialIntelligence #LanguageModels #TechTutorial #AICommunity #Innovation #DeepLearning

29