Tanmoy Datta

🚀 Making LLMs work for you - A Guide to Become 10x Programmer/MLE/DS I am sure many of you use LLMs for your day-to-day work. Mostly, I have seen people use LLMs while writing code (Copilot), or as an alternative to Copilot to catch bugs, but they have become much more useful than that, provided you know how to use them well. Here are a few ways I have been using LLMs in my daily work that are helping me increase my productivity: 📚 1. Quick Paper Assessment with ELI5 Summaries - Paper abstracts are called abstracts for a reason. Asking an LLM for an ELI5 summary makes it much easier to know if I want to read a paper. 💻 2. Implementing Research Papers in Code - Try giving an LLM like Claude/ChatGPT a research paper and ask to code it. You would be surprised. Tip: Claude performs much better for this task. 🔄 3. Upgrading Code Structure and Readability - Do you want to add typing hints when writing your Python code, or do you need comments in your code, or do you have a big function that you want to be converted to classes in your code? LLMs can do it all, and you would be amazed at the code-conversion power they have. ⚡ 4. Optimizing Code Performance - The thing with LLMs is that if you tell them what to do, they can do it well. For example, if you give them a non-optimized piece of code that runs correctly, they will be able to write it in a much-optimized way. For example, I just used an LLM to convert a matrix multiplication in PyTorch to sparse MM as I was getting an OOM memory error because one of the matrices became too large. I am not even joking, but that would have taken at least 1-2 hours for me to come up with. 📝 5. Automated Documentation - Got bored of writing documentation for what you did but have the code? LLMs also work the other way around. Want to write a summary of what you did in a week, you can use an LLM for that. Saves so much time that gets wasted in formatting. 🧪 6. Streamlined Test Framework Creation - Let LLMs handle the boilerplate for Unit tests & Integration tests while you focus on core logic. 🌐 7. Language-Agnostic Development - You can paste a function in Python and ask for its equivalent in Rust, complete with language-specific best practices and error handling. This dramatically reduces the learning curve when working with new languages and helps maintain productivity during language transitions. 🤝 Apart from all of the above, LLMs work well as mentors as well as interns. LLMs are great when you just need someone to bounce ideas with. I sometimes just talk to an LLM as I would to a colleague talking about the problems I have and how I can solve them. And they come up with the right ideas or send me in the right direction a lot of times. They also act as a personal assistant for free, who can deal with writing messages, creating charts, and doing all sorts of uninteresting work for you. 💭 What's your favorite way to use LLMs? Let me know in the comments! #MachineLearning #AI #Programming

47

🚀 C++ Insights in One Frame: Compile-Time Polymorphism 🎯 In today’s post, I want to highlight the importance of compile-time evaluation in C++. This visual comparison sheds light on how <constexpr> can drastically affect code behaviour. 🟢 Multi-Scope Evaluation with <constexpr>: When we use <constexpr>, compile-time decisions are enforced, ensuring that only valid code paths are generated. This enables cleaner, optimized code, prevents unnecessary compilation errors, and enhances performance. 🔴 Incorrect Code Translation: Without proper compile-time evaluation, like when skipping <constexpr>, errors can slip through, resulting in broken code paths and compilation errors. In the image, we have two classes with different functions: 🟢 The correct example utilizes <constexpr> to distinguish between different types at compile-time, ensuring that the function start() calls the appropriate member function (Accelerate() or TakeOff()) based on whether the Vehicle is of type Car. 🔴 The incorrect example does not use <constexpr>, which leads to a situation where both possible code paths are generated in the template instantiation. If you pass a Car type, the compiler will attempt to check TakeOff(), which doesn't exist in the Car class, causing a compilation error. This highlights why using <constexpr> is vital to ensure that template instantiations don't lead to unexpected compilation issues. Stay tuned for more insights! 👨‍💻💡 #cplusplus #codingbestpractices #cpp #softwareengineering #codingtips #optimization #developerinsights

33

3 komentarze

Can we parallelise model training like transformers yet scale linearly to long sequence length just like RNN? 🤔 Yup, its possible with SSMs, infact they can do inference at computation cost of O(1). 🤯 First of all, how does it even compare with the most advanced Transformer capabilities like, - Self-Attention Mechanisms - Multi-Head Attention - Quadratic Scaling, etc. All of these have been outperformed on the new benchmarks across various domains such as language, audio, and genomics by Mamba. 🏆 Your model might be excelling with the above methods, but does it handle million-length sequences efficiently? Or process long sequences without breaking a sweat? Mamba efficiently integrates a selective state space model. What it effectively does for sequence processing is: ⚡ - Dynamically selects relevant information while discarding irrelevant data. - Operates with linear time complexity, ensuring fast and efficient processing. - Utilizes a hardware-aware parallel algorithm that maximizes GPU efficiency. - Incorporates simplified end-to-end neural network architecture without attention or MLP blocks. And during sequence modeling, - The input is processed through a selective mechanism, allowing the model to focus on the most relevant parts. - The state space parameters (such as Δ, B, and C) are dynamically adjusted based on the input, enhancing content-aware processing. - Efficient recurrent mode computation ensures faster training and inference with lower memory usage. But this approach doesn't just stop at sequence processing; it extends to real-world applications across different domains. For example: - Language modeling with Mamba outperforms Transformers of similar and even larger sizes in both pretraining and downstream evaluations. - Audio modeling with Mamba provides better quality and efficiency, making strides in speech recognition and generation. - Genomic sequence analysis with Mamba achieves state-of-the-art performance, handling long DNA sequences with ease. Mamba is not just an architectural improvement; it's a revolution in sequence modeling. I'll link the research paper down in comments, and you can explore its capabilities further. For more such detailed explanations, follow me. See ya! . . . #Mamba #SequenceModeling #AI #NeuralNetworks #Efficiency #StateSpaceModel #LanguageModeling #AudioModeling #Genomics #DeepLearning

24

ExecuTorch, an end-to-end solution within the PyTorch Edge ecosystem that enables efficient on-device inference across mobile and edge devices, including wearables, embedded systems, and microcontrollers. Key Benefits of ExecuTorch: • Portability: ExecuTorch is compatible with a wide range of computing platforms, from high-end mobile phones to resource-constrained embedded systems and microcontrollers. (PyTorch) • Productivity: Developers can utilize the same toolchains and SDKs from PyTorch model authoring and conversion to debugging and deployment across various platforms, streamlining the development process. (PyTorch) • Performance: ExecuTorch offers a lightweight runtime that leverages full hardware capabilities, including CPUs, NPUs, and DSPs, ensuring a seamless and high-performance user experience. (PyTorch) Why ExecuTorch? On-device AI presents unique challenges such as diverse hardware, critical power requirements, limited or no internet connectivity, and real-time processing needs. ExecuTorch addresses these challenges by providing a highly portable solution that enhances developer productivity without compromising performance. (PyTorch) How ExecuTorch Works: 1. Export the Model: Capture the PyTorch program as a graph, preserving the semantics of the original model. (PyTorch) 2. Compile the Exported Model: Convert the exported model into an executable format called an ExecuTorch program, enabling various optimizations such as compression and hardware-specific acceleration. (PyTorch) 3. Run on Target Device: The ExecuTorch runtime loads and executes the program on the device, efficiently computing the output with low latency. (PyTorch) ExecuTorch’s robust export capabilities, operator standardization, and low-overhead runtime make it an ideal choice for deploying machine learning models to edge devices. (PyTorch) For a comprehensive technical overview and tutorials, visit the ExecuTorch documentation. (PyTorch) https://pytorch.org/edge

2

Facebook just released a very different kind of LLMs. These are focused on code optimization and compiler optimization. This is super important because it can fundamentally change the way code is executed on any machine. Here's how it works right now: Programs written in many languages like C/C++ have intermediate steps before they are run on the computer. For example they are converted into Byte Code and then converted into machine code via LLVM (Low level virtual machine). But Facebook's new model can take in the byte code as input and convert it to into a more efficient representation so that when this new code is converted into machine code, it will be faster. "In theory", it could mean that all existing code might be made faster without any change by developers. Our games can have higher frame rates on the same GPUs. Photoshop might run faster on the same laptop. Our software might complete tasks faster without any changes at all. But that will probably take quite a long time. To be honest I don't expect these models to actually be used in production since unless they were 100% as good as compilers they will probably break the code. And are they 100% good? No, not even close. But the speed at which improvements are happening, they will probably reach that level eventually. And that's when we will see models like these everywhere. 

11

Google’s DeepMind has released a new method called JEST, which claims to train AI models 13 times faster and 10 times more power-efficiently than current methods. An AI model is only as good as the data it is trained on, and the time taken to train an AI model is proportional to the quantity of data. Naturally, since the new method JEST is 13 times faster in training, it has been trained on less data — which is of higher quality. The paper released by Google discusses optimizing the data selected for training the model. I found the solution to be quite smart: identify dependencies between data points and accelerate the training process by focusing on data points that match or have dependencies. Quality >> Quantity. AI systems demand a lot of energy and water for cooling today (annually ~ 4.3 GW, 0.5 litre water per 5-50 ChatGPT prompts) — JEST is a promising and positive sign for the AI world we are heading towards. #ai #deepmind #google

65

2 komentarze

𝗟𝗹𝗮𝗺𝗮 𝟯 𝘂𝗻𝘃𝗲𝗶𝗹 𝗮 𝗳𝘂𝗻𝗱𝗮𝗺𝗲𝗻𝘁𝗮𝗹 𝗰𝗼𝗺𝗽𝘂𝘁𝗶𝗻𝗴 𝘁𝗿𝗮𝗱𝗲𝗼𝗳𝗳 𝘁𝗵𝗮𝘁 𝗳𝗲𝘄 𝗮𝗿𝗲 𝘀𝗲𝗲𝗶𝗻𝗴⬇️ 🦙Llama 3 just came out and to the surprise of many people, the L3-8B version can match the L2-70B one on several benchmarks. Several people also argued that the chinchilla paper (https://lnkd.in/dXaRqyea) was wrong because the llama 3 scaling is very far from chinchilla-optimal. 🤔 In my opinion, this is due to a misunderstanding of what the chinchilla paper is about and more generally of the effects of scale in deep learning. The bitter lesson (https://lnkd.in/d3sVYYgp) has become common knowledge in the field thanks to the GPT family that popularized it. We hear everywhere that "bigger models are better": Yes it's true, but in what sense? Many people have a mental model along the lines of "bigger models are more capable" meaning they would be able to achieve levels of performance that smaller models can't. ❌ This is not true with LLMs and is mainly because we train only once on tokens (most of them at least) and therefore overfitting is not possible. The 𝘳𝘦𝘢𝘭 advantage of bigger models over smaller ones is that they converge *faster*. In other words, you can reach a given level of NLL (modeling performance) with fewer training iterations. ❔ Then a natural question arises: Should I train a smaller model with more or a bigger model with fewer iterations? That's where the chinchilla paper brings a response. They say that the question they are answering is: "Given a fixed FLOPs budget, how should one trade-off model size and the number of training tokens?" (implying "to achieve the best possible NLL"). 🧐 Coming back to Llama 3, it's obvious that the Llama team is aware of this work and there is a reason why they didn't follow this recipe. My guess is that they are taking into account the inference cost in the balance (which is completely out of the scope of the chinchilla paper). When one "overtrains" an 8B model, one ends up with a worse "NLL to training compute" ratio, but also with a better "NLL to inference compute" ratio. ✅ Now if we think about it from a business perspective, it's a very powerful tradeoff because you can choose to buy performance either at a fixed cost (training) or at a variable one (inference). Hope this post helped you! #Llama3 #Meta #LLM #AI

9

3 komentarze

I don't rely on Accuracy in multiclass classification settings to measure model improvement 🧩 Consider probabilistic multiclass classification models. Using "Accuracy" as a signal to measure model improvement can be deceptive. It can mislead you into thinking that you are not making any progress in improving the model. In other words, it is possible that we are actually making good progress in improving the model... ...but “Accuracy” is not reflecting that (YET). The problem arises because Accuracy only checks if the prediction is correct or not. And during iterative model building, the model might not be predicting the true label with the highest probability... ...but it might be quite confident in placing the true label in the top "k" output probabilities. Thus, using a "top-k accuracy score" can be a much better indicator to assess whether my model improvement efforts are translating into meaningful enhancements in predictive performance or not. For instance, if top-3 accuracy increases from 75% to 90%, it is clear that the improvement technique was effective: - Earlier, the correct prediction was in the top 3 labels only 75% of the time. - But now, the correct prediction is in the top 3 labels 90% of the time. Thus, one can effectively direct the engineering efforts in the right direction. Of course, what I am saying should ONLY be used to assess the model improvement efforts. This is because true predictive power will be determined using traditional Accuracy. As depicted in the image below: - It is expected that “Top-k Accuracy” may continue to increase during model iterations. This reflects improvement in performance. - Accuracy, however, may stay the same during successive improvements. Nonetheless, we can be confident that the model is getting better and better. For a more visual explanation, check out this issue: https://lnkd.in/dP_h8SFM. -- 👉 Get a Free Data Science PDF (530+ pages) by subscribing to my daily newsletter today: https://lnkd.in/gzfJWHmu -- 👉 Over to you: What are some other ways to assess model improvement efforts?

51

3 komentarze

It is well understood that when generating text, Large Language Models (LLMs) predict multiple possible next words, each associated with a probability. An effective decoding strategy is essential for selecting the next token from the probability distribution, ensuring coherent text generation. For an insightful discussion on the various decoding strategies employed in LLMs, explore the video "LLMs: Hands-on Decoding Strategies", which delves into key techniques like greedy search, beam search, top-k sampling, and nucleus (Top-p) sampling. LLMs: Hands-on Decoding Strategies --- https://lnkd.in/gVk-5Rim #LLMs #AI #MachineLearning #NLP #DecodingStrategies #ArtificialIntelligence #LanguageModels #TechTutorial #AICommunity #Innovation #DeepLearning

29

🚀 On Reasoning in LLMs Paper: Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization" by Wang et al. https://lnkd.in/gZ6GB2-N It offers profound insights into how transformer models learn to reason implicitly. Here are the key takeaways: 1️⃣ Grokking in Reasoning Tasks: The study focuses on two reasoning tasks: composition and comparison. Transformers can learn these tasks, but only through "grokking" - a phenomenon where generalization occurs after extended training beyond overfitting. 2️⃣ Mechanistic Analysis: Using causal tracing and logit lens techniques, the researchers peered into the model's internals during grokking. They identified specific "circuits" that form: - A sequential circuit for composition - A parallel circuit for comparison This difference in circuit structure explains varying levels of systematic generalization across tasks. 3️⃣ Efficiency Hypothesis: Grokking is explained as a transition from a memorizing circuit to a more efficient generalizing circuit. This transition is driven by: - The model's implicit bias towards efficiency - Explicit regularization (e.g., weight decay) 4️⃣ Parametric vs. Non-parametric Memory: In complex reasoning tasks with large search spaces, the grokked transformer (parametric memory) outperformed state-of-the-art language models like GPT-4-Turbo and Gemini-1.5-Pro (non-parametric memory). 5️⃣ Implications for Architecture Design: The findings suggest potential improvements to transformer architectures, such as incorporating cross-layer memory sharing mechanisms to enhance generalization capabilities. #grokking #transformers #llms #gpt4 #claude #cmu

10

Implementing Reinforcement Learning (RL) in the context of Large Language Models (LLMs) posed challenges, particularly in designing reward engineering and penalization components. In traditional RL, reward engineering is straightforward, often defined through a scalar reward function. However, implementing the same concept in RL with LLMs required a critic model that evaluates the questions and answers, critiquing them in a manner that simulates the reward function. A recent research paper on Natural Language Reinforcement Learning (NLRL) provides an intriguing perspective on the reward problem in RL by translating traditional RL concepts into natural language constructs. Here's a summary of how it addresses the reward problem: Language-Based Value Functions: Unlike traditional RL, which uses scalar rewards, NLRL employs language-based value functions. These functions articulate the effectiveness of policies and actions in a more interpretable, natural language format. This approach addresses the need for richer, more informative feedback than simple numerical scores. Iterative Refinement of Language Policies and Critics: NLRL incorporates a structured training pipeline where language-based policies and critics are iteratively refined using textual feedback. This method allows the system to handle the complexities of defining rewards in natural language, improving its understanding and evaluation of actions over time through textual feedback loops. Use of Large Language Models (LLMs): The framework leverages the advanced capabilities of LLMs to interpret, generate, and evaluate complex language-based information. This is essential for implementing language-based value functions and policies, enabling the system to generate and refine responses that align closely with desired outcomes. Distilling Enhanced Actions and Evaluations: NLRL employs a process where enhanced evaluations from the language-based value function approximator and improved actions from the language policy improvement operator are distilled. This allows for continuous refinement and learning. Compatibility with Existing LLMs: The framework is designed to be compatible with current LLMs, enabling it to leverage pre-existing language models to process and generate linguistic data effectively. This compatibility is crucial for translating the traditional reward engineering process into a language-based framework. The NLRL framework's introduction of natural language as a medium for representing and evaluating policies and rewards offers a novel solution to the reward problem in RL. By shifting from numerical reward systems to a more flexible, expressive, and intuitive system based on language, NLRL enhances interpretability and adaptability, paving the way for more diverse RL applications. Innovation Hacks AI Inc. #llm #refincorcementlearning #ai

4

[research update] use a Transformer's multi-head attention activations to fetch RAG style, to augment multi faceted problem breakdown for LLMs in ."Multi-Head RAG: Solving Multi-Aspect Problems with LLMs": 1. Enhances LLMs by integrating document retrieval (RAG) into the LLM context. 2. Challenge: Existing RAG solutions struggle with queries needing multiple documents with varied contents due to distant embeddings. 3. Multi-Head RAG (MRAG): Introduces a novel approach using Transformer's multi-head attention activations to fetch diverse documents. 4. Activation Utilization: Different attention heads capture varied data aspects, improving complex query retrieval. 5. Improved Relevance: MRAG shows up to 20% improvement in relevance over standard RAG baselines. 6. Evaluation Methodology: Provides metrics, synthetic datasets, and real-world use cases to demonstrate effectiveness. 7. Seamless Integration: MRAG can be integrated with existing RAG frameworks and benchmarking tools like RAGAS. 8. Multi-Aspect Problem Solving: Designed specifically to handle multi-aspect queries more effectively. 9. Data Stores Compatibility: Compatible with various data store classes. 10. Potential Applications: Benefits scenarios requiring diverse information retrieval, enhancing LLM responses. Experiment The paper introduces MRAG and evaluates it using synthetic datasets and real-world data. It compares MRAG's performance against standard RAG models. The experiment focuses on the model's ability to retrieve relevant documents across various query aspects, demonstrating up to a 20% improvement in relevance over baselines. Use Case MRAG is tested on real-world scenarios like customer support, where queries often require information from multiple documents with diverse contents. The model's capability to retrieve and integrate multi-aspect data enhances the quality of responses.

81

5 komentarzy

Parametric design is about to change forever. Current approaches to code generation for design are limited because they don't learn from code execution feedback during training, making them static by nature. This is where the RLEF framework comes in. RLEF enhances code language models through a comprehensive training process: → Pretraining: Teaching the neural network to predict the next token by learning from vast amounts of text data. → Fine-tuning: Instructing the pre-trained model to follow specific prompts and respond accurately. → Post-training: Aligning the model with human preferences using reinforcement learning, specifically tailored for code generation with RLEF. RLEF enables these models to learn from execution feedback, allowing them to produce not just syntactically correct code, but functional code that meets intended outcomes. By understanding and iteratively improving based on execution results, RLEF transforms static code generation into a dynamic and intelligent process. Why does this matter for parametric design? In our field, design parameters aren't just variables, they're the fundamental building blocks of innovation. Automating parametric design requires code that not only adapts but also intelligently evolves with each iteration. Traditional AI code generators often leave us with code that needs extensive tweaking. But with frameworks like RLEF, we can envision an AI that understands and learns from execution. It doesn't stop at code generation; it evaluates the output and learns from successes and failures. Through reinforcement learning, it continuously improves to better meet design objectives. Less time debugging means more time innovating. Imagine AI models that can automatically adjust design parameters based on real-time feedback, optimize complex designs without constant human intervention, and accelerate development cycles by learning from each execution. This is what I call spoken design ;) The potential of RLEF isn't just theoretical; it's a tangible step toward more intelligent, responsive design systems. Credits to Facebook AI research team for their amazing work. Exciting times ahead for those passionate about the fusion of AI and design. Link to the paper: https://lnkd.in/eFgAW8qC

26

In-Context Learning at Extreme Scales: My Thoughts As large language models (LLMs) continue to expand their context capacities, the domain of in-context learning (ICL) at extreme scales has piqued my interest. A recent paper by researchers at Carnegie Mellon University and Tel Aviv University provides a detailed exploration into this area, unveiling both impressive capabilities and notable challenges. 🔍 Key Insights: Performance Improvement: The study demonstrates that ICL can improve significantly by integrating hundreds to thousands of examples into the context window—far beyond traditional few-shot settings. This is particularly impactful in complex tasks like 77-way banking intent classification, where long-context ICL matches or surpasses traditional finetuning. Input Sensitivity: Long-context ICL shows reduced sensitivity to input shuffling compared to shorter contexts. However, organizing examples by label dramatically reduces performance, emphasizing the need for diverse context configurations. Source of Gains: Performance gains primarily arise from accessing more relevant examples, rather than aggregating knowledge across many examples during encoding. 📉 Challenges & Drawbacks: Sensitivity to Example Order: Despite reduced sensitivity, the performance still varies with input order, indicating potential instability in practical scenarios. Performance Saturation: The models often do not fully leverage their extended context capacities, showing performance saturation before reaching the maximum context length. Computational Cost: The increased computational cost associated with using large context windows could limit practical deployment, particularly in resource-constrained environments. Efficiency Issues: There is a noted inefficiency in how models utilize the encoded information within large datasets, suggesting that current architectures may need refinement to better use available data. 🚀 Future Directions: The paper suggests several exciting avenues for future research, including optimizing the trade-offs between computational costs and performance, exploring model adaptation strategies, and refining ICL approaches for specific applications or datasets. As we push the boundaries of what LLMs can achieve, effectively leveraging their full context could unlock new capabilities. This area is ripe for further exploration and could lead to major advancements in how we deploy AI. For those at the cutting edge of language model development, I highly recommend this paper for its thorough analysis and forward-looking perspective. Check out the full work at https://lnkd.in/d7AmGh4y. What are your thoughts on the future of in-context learning at such scales and the challenges discussed? #AI #MachineLearning #DeepLearning #LanguageModels #InContextLearning

17

We are happy to share our latest article: "Research to Production: Relative Answer Quality (RAQ) and NVIDIA NIM." With the surge in open-source AI solutions like Mixtral from Mistral AI and LLaMA 3 from Meta, companies are now focusing on deploying these models efficiently and safely. Our article addresses two critical questions: - Which model should be used? - How can it be deployed at scale and safely? To answer the first question, we introduce Relative Answer Quality (RAQ), a novel method to assess and rank LLMs by comparing their answers using an independent LLM. RAQ is versatile, easily integrated into any workflow, and applicable across various domains and use cases. It provides confidence in model selection by using statistical tests to ensure significant ranking differences. For the second question, we partnered with NVIDIA to explore NIM microservices. NIM simplifies transitioning from research to production. It supports rapid prototyping and scalable deployment, whether on NVIDIA-hosted services or self-hosted environments. Its optimized inference engines and flexible model fine-tuning capabilities make infrastructure setup minimal. The article was co-authored with Rafael Guedes. https://lnkd.in/dXPHcPAw Read more about NVIDIA NIM here: https://nvda.ws/3yxsXNK #AI #MachineLearning #GenerativeAI #LLM #NVIDIA #RAQ #DataScience #AIResearch

28

1 komentarz

Optillm: An optimizing inference proxy for LLMs Optillm is an optimizing inference proxy that has over a dozen techniques that aim to improve the accuracy of the responses using test-time compute. Over the last couple of months we have set several SOTA results using smaller and less capable models like gpt-4o-mini. Recently, we have added support for plugins that enable capabilities like memory, privacy and code execution to optillm. Plugins are just python scripts that you can also write yourself, optillm would then load them at start from the directory. You can now also combine the plugins and techniques using & and | operators. E.g. We recently evaluated the new FRAMES benchmark from Google. Using a combination of plugins and techniques (we used readurls&memory-gpt-4o-mini) we were able to get 65.7% accuracy on the benchmark which is very close to what Google reported in their paper with Gemini Flash 1.5 (66.5) which has a context length that is almost 10 times that of gpt-4o-mini. Do check out Optillm at https://lnkd.in/gN6_kNky

21

1 komentarz

Shashank Sonkar and colleagues are back with another great paper and matching GitHub repo, "Pedagogical Alignment of Large Language Models". You can read the paper on arXiv here - https://lnkd.in/enZfCJgi, and browse the code here - https://lnkd.in/gyW5hSzp. They demonstrate an interesting approach to generating synthetic data to train open weights models to behave more pedagogically, as well as developing some new evaluation metrics. I think there's lots here for those of us interested in this kind of work to learn from and build on. (If synthetic data generation is new do you, check out this post from answer.ai earlier this week - https://lnkd.in/ecsG7R4b.)

38

4 komentarze

Were RNNs All We Needed? 🤔 For context, most efforts to replace Transformers with a computationally more efficient architecture have basically been different variations on Recurrent Neural Networks (RNNs). Over the last few years, RNNs like LSTMs and GRUs were largely overshadowed by Transformers in handling long sequences. But more and more recent breakthroughs like this one are suggesting a potential comeback! In this new paper, we have Yoshua Bengio (one of the “Godfathers of AI”) and his team show that by stripping down the complexities of traditional RNNs, like hidden state dependencies, we can dramatically boost their performance. The new “minimal” versions of LSTMs and GRUs (minLSTM, minGRU) are 175x faster than before when handling long sequences. Even better, they match the performance of cutting-edge models like Transformers—all while using fewer parameters and being way less complicated than xLSTMs! 💡 Key takeaways: 1. RNNs no longer require Backpropagation Through Time (BPTT), which is a major boost in training efficiency. 2. With fewer parameters, these RNNs are more resource-efficient—perfect for industries where speed and scalability matter. 3. This is a game-changer for long-sequence tasks like language modeling and real-time applications. This research shows that sometimes, less really is more. Simplifying models can unlock a lot of potential for real-world AI applications. Simple is truly better than complex. What do you think? Are we heading back to the age of RNNs? 💭 Edit: Link to the paper in the description.

12

4 komentarze