Qi Liu

Exciting Advances in Vision Language Models! I’m thrilled to share insights from a recent paper titled "Token-Level Detective Reward Model for Large Vision Language Models" by Deqing Fu, Tong Xiao, Rui Wang, Wang Zhu, Pengchuan Zhang, Guan Pang, Robin Jia, and Lawrence Chen. This innovative research addresses key challenges in reward models for multimodal large language models. Key Highlights: - Traditional reward models assign binary feedback to entire texts, limiting their effectiveness in providing nuanced feedback. - In the context of multimodal language models that process both images and text, existing models risk developing biases, leading to a disconnect from image content. - The proposed Token-Level Detective Reward Model (TLDR) offers fine-grained annotations for each text token, enhancing model accuracy and grounding. Methodology: - The authors introduce a perturbation-based approach to generate synthetic hard negatives, allowing for the creation of token-level labels. - TLDR models demonstrate their utility in helping off-the-shelf models self-correct and in evaluating hallucinations in generated content. Impact: - TLDR models can accelerate human annotation processes by 3x, broadening the acquisition of high-quality vision-language data. This work showcases the potential of refined feedback mechanisms in improving the performance and reliability of multimodal models. Congratulations to the team for pushing the boundaries of AI research! #AI #MachineLearning #VisionLanguageModels #Research #Innovation #Meta Aakanksha Tiwari 🪙

18

🚀 Language Model Adaptation: ReFT  Stanford University and Pr(Ai)²R Group have published an exciting new study titled "ReFT: Representation Finetuning for Language Models,".This work, authored by Zhengxuan Wu, Aryaman Arora, Zheng Wang, Atticus Geiger, Dan Jurafsky, Christopher D. Manning, and Christopher Potts, introduces a paradigm-shifting method in the field of parameter-efficient finetuning (PEFT).  🔬 What is ReFT? Representation Finetuning (ReFT) focuses on direct interventions in hidden representations of frozen language models rather than updating weights. This novel approach leverages the semantic richness of internal representations to achieve downstream task adaptation with significantly fewer parameters.  💡 Key Technical Contributions: 1. LoReFT (Low-Rank Linear Subspace ReFT):     - A highly efficient method within the ReFT family.     - Operates on low-dimensional linear subspaces of hidden representations using low-rank projection matrices.     - Achieves 15×–65× parameter efficiency compared to LoRA while maintaining or improving task performance.  2. DiReFT (Direct Intervention ReFT):   - An ablated version of LoReFT with relaxed constraints, improving computational efficiency with a slight tradeoff in performance.  3. Mathematical Foundations:   - Utilizes Distributed Interchange Interventions (DII) to manipulate task-specific subspaces.     - Formulates interventions as linear transformations, represented as Phi(h) = h + R^T(W h + b - R h), where R, W, and b are trainable parameters.  4. Efficiency Across Tasks and Models:   - Demonstrated state-of-the-art results on multiple benchmarks, including commonsense reasoning, arithmetic reasoning, instruction-following, and GLUE tasks.     - Applicable to models of various sizes, from RoBERTa-base (125M) to LLaMA-2 (13B), and shows improved scalability with model size.  5. Interpretable and Modular:   - ReFT interventions are modular and interpretable, aligning with insights from causal abstraction studies and neural representation research.  📊 Benchmark Achievements: - Commonsense Reasoning: Outperformed existing PEFTs by achieving the highest accuracy with minimal parameter updates.   - Instruction Following: Matched or exceeded performance of full finetuning while using orders of magnitude fewer parameters.   - Natural Language Understanding (GLUE): Competitive results even with small-scale models like RoBERTa. 🔗 Dive Deeper: - Full Paper: https://lnkd.in/d3hEeVYj - Tools and Examples: https://lnkd.in/dKemcftA 

36

New in the AFP: The Category of Sets An axiomatization of category SET on top of HOL, following the presentation given by Halvorson, including Cantor's diagonalization theorem and mathematical induction. https://lnkd.in/g6WTQ-MX

13

📣 Kiroong Choe is presenting our new IEEE TVCG paper, Enhancing Data Literacy On-demand: LLMs as Guides for Novices in Chart Interpretation, at the "Look, Learn, Language Models" session this Friday: https://lnkd.in/eygdA_jt 📝 Author List: Kiroong Choe, Chaerin Lee, Soohyun Lee, Jiwon Song, Aeri Cho, Nam Wook Kim, Jinwook Seo 🎓 Kiroong (https://kiroong.github.io/), a Ph.D. student at Seoul National University advised by Prof. Jinwook Seo, spearheaded this work by leading a group of other students. I enjoy meeting with him because he always thinks critically about my input and offers different perspectives when needed, which in turn improves my own thinking. 🙂 📊 We compared three different methods of learning assistance for interpreting complex visualizations for novices: 1) using web search, 2) LLMs with text-based support, and 3) LLMs with both text and visual support. We developed a "visual prompting" system to support the third condition. 💡 An interesting finding is that, despite participants preferring the LLM-based approach, when we counted the number of insights, the web search performed the best, even though it was more time-consuming and challenging to use. 🤔 This might suggest that cognitive difficulty leads to more engagement and thus more learning. This would be an important consideration for future LLM-based learning systems. 📄 Please check out the full paper manuscript: https://lnkd.in/etfn6yGg

18

At EMNLP, Amazon researchers showed how to “detoxify” an LLM's output by using an auxiliary model to estimate the distribution closest to the LLM's current distribution that satisfies toxicity constraints and penalizing the gap between the two. #EMNLP2024 #LLM #LargeLanguageModels

30

A Survey of Generative Search and Recommendation in the Era of Large Language Models Yongqi Li, Xinyu Lin, Wenjie Wang, Fuli Feng, Liang Pang, Wenjie Li, Liqiang Nie, Xiangnan He, Tat-Seng Chua Abstract With the information explosion on the Web, search and recommendation are foundational infrastructures to satisfying users' information needs. As the two sides of the same coin, both revolve around the same core research problem, matching queries with documents or users with items. In the recent few decades, search and recommendation have experienced synchronous technological paradigm shifts, including machine learning-based and deep learning-based paradigms. Recently, the superintelligent generative large language models have sparked a new paradigm in search and recommendation, i.e., generative search (retrieval) and recommendation, which aims to address the matching problem in a generative manner. In this paper, we provide a comprehensive survey of the emerging paradigm in information systems and summarize the developments in generative search and recommendation from a unified perspective. Rather than simply categorizing existing works, we abstract a unified framework for the generative paradigm and break down the existing works into different stages within this framework to highlight the strengths and weaknesses. And then, we distinguish generative search and recommendation with their unique challenges, identify open problems and future directions, and envision the next information-seeking paradigm. 👉 https://lnkd.in/dPCmCG-t #machinelearning

1

Even though LLM is commonly considered as a powerful data augmentation tool, naively prompting LLM with arbitrary augmentation instructions cannot really achieve very good improvements as you expected. In this work, we explore how to empower LLM to automatically generate and select suitable data augmentation methods for specific downstream tasks and show that our Self-LLMDA can well generalize to different tasks.

58

2 Comments

Huge release by Daekeun Kim and Hyo (HK) Choi from our AI GBB team in Korea, sharing Korean language benchmarks for our latest Azure OpenAI model, gpt-4o-mini - like gpt-4o, the model excels on Korean language tasks, in this case mini completely eclipses gpt-35-turbo and nearly achieves gpt-4-turbo performance. The benchmarks and code are open source, extensible beyond our Azure models, so we encourage our customers and the larger community to try it out and even fork/share your own findings - and please let me in the comments what other Asia languages beyond Korean you may be looking to evaluate with your language models.

52

7 Comments

In his latest post from #TheBatch, Andrew Ng offers his argument on why #California's proposed law SB-1047 is likely to stifle #AI builders and impose unnecessary guardrails on #AI innovation. The law established a “hazardous capability” designation that makes *builders* of AI models liable if someone uses their models to do harm. Andrew argues that laws should not regulate the technology (the tool itself) but its application (how it is implemented).   It's a thoughtful read --> DeepLearning.AI #AIgovernance #AIregulation https://lnkd.in/eBD-9rja

3

Congratulations to Qianlin Liang on successfully defending his Ph.D thesis titled "Resource Management for Edge AI." Qianlin's thesis looks at model-driven resource management for edge AI workloads. #edgeAI #cloudcomputing

93

7 Comments

Brand new paper on September 5th, 2024 on Mamba and Physics-informed machine learning. State-space models are accurate and efficient neural operators for dynamical systems Zheyuan Hu, Nazanin Ahmadi Daryakenari, Qianli Shen, Kenji Kawaguchi, George Karniadakis arXiv: https://lnkd.in/gnVbq5rm Physics-informed machine learning (PIML) has emerged as a promising alternative to classical methods for predicting dynamical systems, offering faster and more generalizable solutions. However, existing models, including recurrent neural networks (RNNs), transformers, and neural operators, face challenges such as long-time integration, long-range dependencies, chaotic dynamics, and extrapolation, to name a few. To this end, this paper introduces state-space models implemented in Mamba for accurate and efficient dynamical system operator learning. Mamba addresses the limitations of existing architectures by dynamically capturing long-range dependencies and enhancing computational efficiency through reparameterization techniques. To extensively test Mamba and compare against another 11 baselines, we introduce several strict extrapolation testbeds that go beyond the standard interpolation benchmarks. We demonstrate Mamba's superior performance in both interpolation and challenging extrapolation tasks. Mamba consistently ranks among the top models while maintaining the lowest computational cost and exceptional extrapolation capabilities. Moreover, we demonstrate the good performance of Mamba for a real-world application in quantitative systems pharmacology for assessing the efficacy of drugs in tumor growth under limited data scenarios. Taken together, our findings highlight Mamba's potential as a powerful tool for advancing scientific machine learning in dynamical systems modeling.

98

Our new paper on BART for Regression Discontinuity Designs with Rafael Campello de Alcantara, Meijia Wang, and Hedibert F. Lopes is recently up on ArXiV. https://lnkd.in/gV6gZdsG On the one hand, RDDs provided limited inferences because the estimated causal effects are unlikely to generalize away from the cutoff. On the other hand, they crop up more and more in settings where business decisions are triggered by "business logic" using some well-defined metric. This work modifies the all-star Bayesian machine learning method BART to estimate treatment effects in RDDs. By using BART, conditional average treatment effects (heterogeneous/subgroup effects) can be explored in a principled data-driven manner. We develop a painstaking prior elicitation procedure too, if you're worried about the impact of priors. I'll post about it again when it's accepted for publication and again when user-friendly software is available, but for now we'd welcome any feedback. I think it's a nice piece of work.

116

For #InContextLearning, how do we attribute an #LLM prediction to each task demonstration? 🚀 The work of Zijian Zhou, Xiaoqiang Lin, Xinyi Xu, Alok Prakash, Daniela Rus, and Bryan Kian Hsiang Low in #ICML2024 Workshop on In-Context Learning introduces DETAIL: Task DEmonstration Attribution for Interpretable In-context Learning! Our DETAIL method: ✅ Optimizes demonstration selection & ordering in an interpretable way; ✅ Detects corrupted demonstrations; ✅ Computable in REAL-TIME! #Interpretability #PromptEngineering 📝 Paper: https://lnkd.in/gUWmP_y5 🕹️ Github: https://lnkd.in/gSjkfciJ

14

💡 It is not a secret that when it comes to some domain-specific expertise, models trained with targeted fine-tuning of the smaller models are superior to #LLMs in both performance characteristics and cost. 🎯 📝My colleague Jennifer Jie Xu and I have recently published a study, titled “Entity recognition from colloquial texts” (DSS, vol. 179)  in which we evaluated fine-tuning strategies for recognizing symptom descriptions in colloquial texts. The paper describes distinguishing characteristics of colloquial symptom entities found in patient forums, and presents design choices for effective fine-tuning of symptom-labeling models. 🔑 Key Findings: 🔖 Colloquial symptom mentions differ from those in clinician-written texts by using a greater number of words, frequent use of non-medical terms, and figurative language. 🔖 Adding colloquial text to the training data improves model’s ability to correctly identify longer symptom descriptions. 🔖  Models fine-tuned on a mixture of formal medical and colloquial texts perform best overall, when used on both colloquial and formal texts. Interestingly, they also are on par or better for formal texts than models trained on formal texts alone.  🔖 Fine-tuned Roberta and Deberta models worked on par or better than pretrained biomedical models fine-tuned in the same way . 🎯The study of symptom identification in colloquial texts was motivated by the need to improve patient-doctor communication. ⁉ Question: Do you know other domains, in which colloquial texts may contain similarly important information?🤔 #MachineLearning #ModelFineTuning #LLM #NaturalLanguageProcessing #HealthcareIT #DigitalHealth

13

One of the hidden gems I found in github! Check it out!

4

Graph Machine Learning in the Era of Large Language Models (Hong Kong Polytechnic University, April 2024) Paper: https://lnkd.in/gTxVyM_4 Abstract: "Graphs play an important role in representing complex relationships in various domains like social networks, knowledge graphs, and molecular discovery. With the advent of deep learning, Graph Neural Networks (GNNs) have emerged as a cornerstone in Graph Machine Learning (Graph ML), facilitating the representation and processing of graph structures. Recently, LLMs have demonstrated unprecedented capabilities in language tasks and are widely adopted in a variety of applications such as computer vision and recommender systems. This remarkable success has also attracted interest in applying LLMs to the graph domain. Increasing efforts have been made to explore the potential of LLMs in advancing Graph ML's generalization, transferability, and few-shot learning ability. Meanwhile, graphs, especially knowledge graphs, are rich in reliable factual knowledge, which can be utilized to enhance the reasoning capabilities of LLMs and potentially alleviate their limitations such as hallucinations and the lack of explainability. Given the rapid progress of this research direction, a systematic review summarizing the latest advancements for Graph ML in the era of LLMs is necessary to provide an in-depth understanding to researchers and practitioners. Therefore, in this survey, we first review the recent developments in Graph ML. We then explore how LLMs can be utilized to enhance the quality of graph features, alleviate the reliance on labeled data, and address challenges such as graph heterogeneity and out-of-distribution (OOD) generalization. Afterward, we delve into how graphs can enhance LLMs, highlighting their abilities to enhance LLM pre-training and inference. Furthermore, we investigate various applications and discuss the potential future directions in this promising field."

6

Good opportunity

1

It's my great honor to share the excellent work from Nan Jiang, Chengxiao Wang, Kevin Liu, Xiangzhe Xu, Lin Tan, Xiangyu Zhang and Petr Babkin: Nova: Generative Language Models for Assembly Code with Hierarchical Attention and Contrastive Learning. We have also released fine-tuned models for binary code decompilation. Highlights: 1. Nova is built with hierarchical attention specially designed for binary and contrastive learning. 2. Nova is pre-trained on 3B binary and source code tokens. 3. Preprint: https://lnkd.in/gyNgCSvx 4. Models: https://lnkd.in/gJDprVNE; https://lnkd.in/g364Fn5k; https://lnkd.in/gk2MDxqu; https://lnkd.in/g4VmYvCu

6

2 Comments

Great work by my PhD student Zeping Yu 3/3 papers accepted at #EMNLP24. NaCTeM The National Centre for Text Mining Department of Computer Science, The University of Manchester Research funded by #AIST Japan, AI Research Centre, NEDO 1. Neuron-Level Knowledge Attribution in Large Language Models Abstract Identifying important neurons for final predictions is essential for understanding the mechanisms of large language models. Due to computational constraints, current attribution techniques struggle to operate at neuron level. In this paper, we propose a static method for pinpointing significant neurons. Compared to seven other methods, our approach demonstrates superior performance across three metrics. Additionally, since most static methods typically only identify "value neurons" directly contributing to the final prediction, we propose a method for identifying "query neurons" which activate these "value neurons". Finally, we apply our methods to analyze six types of knowledge across both attention and feed-forward network (FFN) layers. Our method and analysis are helpful for understanding the mechanisms of knowledge storage and set the stage for future research in knowledge editing. The code is available on https://lnkd.in/dSP6Qemv [github.com]. 2. Interpreting Arithmetic Mechanism in Large Language Models through Comparative Neuron Analysis We find arithmetic ability resides within a limited number of attention heads, with each head specializing in distinct operations. To delve into the reason, we introduce the Comparative Neuron Analysis (CNA) method, which identifies an internal logic chain consisting of four distinct stages from input to prediction: feature enhancing with shallow FFN neurons, feature transferring by shallow attention layers, feature predicting by arithmetic heads, and prediction enhancing among deep FFN neurons. Moreover, we identify the human-interpretable FFN neurons within both feature-enhancing and feature-predicting stages. These findings lead us to investigate the mechanism of LoRA, revealing that it enhances prediction probabilities by amplifying the coefficient scores of FFN neurons related to predictions. Finally, we apply our method in model pruning for arithmetic tasks and model editing for reducing gender bias. Code is on https://lnkd.in/daedpttN [github.com].

52