Santiago Valdarrama

💥💥💥 A generative framework to bridge data-driven models and scientific theories in language neuroscience Richard Antonello, Chandan Singh, Shailee Jain, Aliyah Hsu, Jianfeng Gao, Bin Yu, Alexander Huth Abstract Representations from large language models are highly effective at predicting BOLD fMRI responses to language stimuli. However, these representations are largely opaque: it is unclear what features of the language stimulus drive the response in each brain area. We present generative explanation-mediated validation, a framework for generating concise explanations of language selectivity in the brain and then validating those explanations in follow-up experiments that use synthetic stimuli. This approach is successful at explaining selectivity both in individual voxels and cortical regions of interest (ROIs).We show that explanatory accuracy is closely related to the predictive power and stability of the underlying statistical models. These results demonstrate that LLMs can be used to bridge the widening gap between data-driven models and formal scientific theories. 👉 https://lnkd.in/dfXq_Y7q #machinelearning

2

Pandora's Box or Aladdin's Lamp: A Comprehensive Analysis Revealing the Role of RAG Noise in Large Language Models "Retrieval-Augmented Generation (RAG) has emerged as a crucial method for addressing hallucinations in large language models (LLMs). While recent research has extended RAG models to complex noisy scenarios, these explorations often confine themselves to limited noise types and presuppose that noise is inherently detrimental to LLMs, potentially deviating from real-world retrieval environments and restricting practical applicability. In this paper, we define seven distinct noise types from a linguistic perspective and establish a Noise RAG Benchmark (NoiserBench), a comprehensive evaluation framework encompassing multiple datasets and reasoning tasks." paper: https://lnkd.in/dwyh2fGT

5

💥 💥 💥 Can LLMs do reinforcement learning in-context - and if so, how do they do it? 👉 Sparse Autoencoders Reveal Temporal Difference Learning in Large Language Models Can Demircan, Tankred Saanum, Akshay K. Jagadish, Marcel Binz, Eric Schulz Abstract In-context learning, the ability to adapt based on a few examples in the input prompt, is a ubiquitous feature of large language models (LLMs). However, as LLMs' in-context learning abilities continue to improve, understanding this phenomenon mechanistically becomes increasingly important. In particular, it is not well-understood how LLMs learn to solve specific classes of problems, such as reinforcement learning (RL) problems, in-context. Through three different tasks, we first show that Llama 3 70B can solve simple RL problems in-context. We then analyze the residual stream of Llama using Sparse Autoencoders (SAEs) and find representations that closely match temporal difference (TD) errors. Notably, these representations emerge despite the model only being trained to predict the next token. We verify that these representations are indeed causally involved in the computation of TD errors and Q-values by performing carefully designed interventions on them. Taken together, our work establishes a methodology for studying and manipulating in-context learning with SAEs, paving the way for a more mechanistic understanding. 👉 https://lnkd.in/d5HRpQ2e #machinelearning

7

💥💥💥 Dynamic Negative Guidance of Diffusion Models Felix Koulischer, Johannes Deleu, Gabriel Raya, Thomas Demeester, Luca Ambrogioni Abstract Negative Prompting (NP) is widely utilized in diffusion models, particularly in text-to-image applications, to prevent the generation of undesired features. In this paper, we show that conventional NP is limited by the assumption of a constant guidance scale, which may lead to highly suboptimal results, or even complete failure, due to the non-stationarity and state-dependence of the reverse process. Based on this analysis, we derive a principled technique called Dynamic Negative Guidance, which relies on a near-optimal time and state dependent modulation of the guidance without requiring additional training. Unlike NP, negative guidance requires estimating the posterior class probability during the denoising process, which is achieved with limited additional computational overhead by tracking the discrete Markov Chain during the generative process. We evaluate the performance of DNG class-removal on MNIST and CIFAR10, where we show that DNG leads to higher safety, preservation of class balance and image quality when compared with baseline methods. Furthermore, we show that it is possible to use DNG with Stable Diffusion to obtain more accurate and less invasive guidance than NP. 👉 https://lnkd.in/dvXsP5Pa #machinelearning

1

Do LLMs "reason" or display cognitive capabilities as it has been lately suggested? In "Cognitive Architectures for Language Agents" by Theodore R. Sumers, Shunyu Yao, et al., (2023), the authors propose a new framework called CoALA (Cognitive Architectures for Language Agents) for designing more effective language-based AI agents. This framework reimagines large language models (LLMs) not as standalone tools but as central components within a broader system that mirrors cognitive processes. CoALA integrates elements of cognitive science and symbolic AI into LLM-based agents, creating a system built around three core pillars: memory, action, and decision-making. The memory component is divided into distinct modules—working, episodic, semantic, and procedural—that store and process information much like the memory systems in humans or intelligent agents. Actions, both internal (interacting with memory) and external (engaging with the environment), provide structured ways for these agents to perform tasks ranging from robotic control to human dialogue. Finally, CoALA’s decision-making procedure leverages planning and reasoning cycles, allowing agents to select actions based on both immediate and long-term goals. The paper highlights the potential for CoALA agents to manage multimodal inputs, adapt to evolving scenarios, and learn from experience—capabilities that would significantly extend what traditional LLMs can achieve on their own. CoALA’s design could pave the way for the next generation of AI systems that are capable of complex, context-aware, and sustained interactions. To me, this paper is interesting because it combines foundational principles of cognitive science with cutting-edge AI technology, setting the stage for more adaptable and capable language agents. CoALA offers a roadmap for advancing AI by embedding memory and decision-making structures that promote deeper learning and reasoning—two key aspects that could transform language-based AI applications across various domains. Paper: https://lnkd.in/gpPiZp2y #LLMs #AI #MachineLearning #CognitiveScience

2

1 Comment

💥💥💥 The Llama 3 Herd of Models Abstract Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models that natively support multilinguality, coding, reasoning, and tool usage. Our largest model is a dense Transformer with 405B parameters and a context window of up to 128K tokens. This paper presents an extensive empirical evaluation of Llama 3. We find that Llama 3 delivers comparable quality to leading language models such as GPT-4 on a plethora of tasks. We publicly release Llama 3, including pre-trained and post-trained versions of the 405B parameter language model and our Llama Guard 3 model for input and output safety. The paper also presents the results of experiments in which we integrate image, video, and speech capabilities into Llama 3 via a compositional approach. We observe this approach performs competitively with the state-of-the-art on image, video, and speech recognition tasks. The resulting models are not yet being broadly released as they are still under development. 👉 https://lnkd.in/dT84NbKw #machinelearning

2

👉🏼 Special considerations for the use of AI tools by PEERs as a learning and communication aid 🤓 Maria Camila Arango 👇🏻 https://lnkd.in/eNz8TnUt 🔍 Focus on data insights: - AI tools can help in leveling the playing field for underrepresented individuals in academia. - AI serves as a vital supplement, offering guidance and assistance to those lacking support. - Embracing AI can provide a lifeline for individuals facing systemic biases. 💡 Main outcomes and implications: - AI tools can enhance diversity, equity, and inclusivity in learning environments. - Institutions should educate community members on ethically harnessing AI tools. 📚 Field significance: - Academia - Artificial Intelligence - Diversity and Inclusion 🗄️: [#academia #artificialintelligence #diversityandinclusion]

2

Are language models rational? The case of coherence norms and belief revision Thomas Hofweber, Peter Hase, Elias Stengel-Eskin, Mohit Bansal Abstract Do norms of rationality apply to machine learning models, in particular language models? In this paper we investigate this question by focusing on a special subset of rational norms: coherence norms. We consider both logical coherence norms as well as coherence norms tied to the strength of belief. To make sense of the latter, we introduce the Minimal Assent Connection (MAC) and propose a new account of credence, which captures the strength of belief in language models. This proposal uniformly assigns strength of belief simply on the basis of model internal next token probabilities. We argue that rational norms tied to coherence do apply to some language models, but not to others. This issue is significant since rationality is closely tied to predicting and explaining behavior, and thus it is connected to considerations about AI safety and alignment, as well as understanding model behavior more generally. 👉 https://lnkd.in/dUhHEFx7 #machinelearning

1

The Chinchilla scaling paper by Hoffmann et al. has been highly influential in the language modeling community. So researchers tried to replicate a key part of their work and discovered discrepancies 👉 Chinchilla Scaling: A replication attempt Tamay Besiroglu, Ege Erdil, Matthew Barnett, Josh You Abstract Hoffmann et al. (2022) propose three methods for estimating a compute-optimal scaling law. We attempt to replicate their third estimation procedure, which involves fitting a parametric loss function to a reconstruction of data from their plots. We find that the reported estimates are inconsistent with their first two estimation methods, fail at fitting the extracted data, and report implausibly narrow confidence intervals--intervals this narrow would require over 600,000 experiments, while they likely only ran fewer than 500. In contrast, our rederivation of the scaling law using the third approach yields results that are compatible with the findings from the first two estimation procedures described by Hoffmann et al. 👉 https://lnkd.in/de7uEdEy #machinelearning

2

Very insightful work by Anthropic’s interpretability team. And an amazing paper, with outstanding writing and figures. The idea is very simple: interpret LLMs by leveraging sparse autoencoders as surrogate models of the MLP of transformer blocks, which allow one to disambiguate the superposition of features captured by a single neuron. A simple idea, but a very careful and complex execution, as it is often the case in our line of work. The paper goes into many details and provide a large array of insights, although the gist of the implementation remains obfuscated due to the closed source nature of Claude. Too bad, because this is the kind of work that we need to better understand and eventually trust LLMs. This is demonstrated by the authors in the section ‘Influence on Behavior’, where they show that clamping some features to either high or low value during inference is “remarkably effective at modifying model outputs in specific, interpretable ways”. Hopefully this kind of work is going to be replicated and generalized to open-weights models, such that we have new ways to steer their behavior. https://lnkd.in/eVym7f_f #interpretability #xai #explainableai #steerableai #anthropic #claude #anthropic

3

Models that function as glue between specific models will allow us to achieve performance across less popular modalities. Once that happens in large, the capabilities of AI will improve dramatically. You can read my articles for free by following the IAEE LinkedIn. Machine Learning - IAEE

1

🚀 Exploring #LoRA and Full Finetuning in Large Language Models 🧵👇 🔍 New Research from Columbia University & Databricks Mosaic AI! 1️⃣ Instruction Finetuning with 100K Prompt-Response Pairs - 🔹 Finding: Full finetuning outperforms LoRA in specialized tasks like programming and math. - 🔹 Insight: Full finetuning adapts better to specific tasks, while LoRA's regularization maintains broader task performance. 2️⃣ Continued Pretraining with 10B Unstructured Tokens - 🔹Finding: Full finetuning excels in target domains, LoRA retains base model capabilities in other tasks. - 🔹 Insight: Full finetuning’s adaptability leads to better target task performance, but LoRA preserves a wider skill set. 3️⃣ Regularization Effectiveness - 🔹 Finding: LoRA offers stronger regularization than weight decay and dropout. - 🔹 Insight: LoRA maintains performance across diverse tasks and generates more varied outputs. Great for general-purpose use! 4️⃣ Rank of Learned Perturbations - 🔹 Finding: Full finetuning learns perturbations with a rank 10-100x higher than typical LoRA configs. - 🔹 Insight: Higher rank explains full finetuning’s performance but comes with higher memory and computational costs. 5️⃣ Best Practices for Finetuning with LoRA - 🔹 Finding: Proposes strategies for balancing target task performance and broader model capabilities. - 🔹 Insight: Achieving a balance between specialization and generalization leverages LoRA’s regularization benefits. Conclusion - Full finetuning maximizes specific domain performance but at higher costs and risk of overfitting. - LoRA provides a balanced approach, maintaining broader capabilities with less computational power. Read the full paper here: 🔗 https://lnkd.in/dhMegFSZ

29

1 Comment