Sameera Horawalavithana

Our code for RAG evaluation is now publicly available at Amazon Science website: https://lnkd.in/g-HKUNZZ. This is the implementation of our evaluation mechanism performed by scoring the RAG on an automatically-generated synthetic exam composed of multiple choice questions based on the corpus of documents associated with the task. #OpenSource #OpenSourcing #AmazonScience #ICML2024 #Evaluation #LLMEvaluation

213

6 Comments

At EMNLP, Amazon researchers showed how to “detoxify” an LLM's output by using an auxiliary model to estimate the distribution closest to the LLM's current distribution that satisfies toxicity constraints and penalizing the gap between the two. #EMNLP2024 #LLM #LargeLanguageModels

30

In his latest post from #TheBatch, Andrew Ng offers his argument on why #California's proposed law SB-1047 is likely to stifle #AI builders and impose unnecessary guardrails on #AI innovation. The law established a “hazardous capability” designation that makes *builders* of AI models liable if someone uses their models to do harm. Andrew argues that laws should not regulate the technology (the tool itself) but its application (how it is implemented).   It's a thoughtful read --> DeepLearning.AI #AIgovernance #AIregulation https://lnkd.in/eBD-9rja

3

Excited to share our recent work on applying context parallelism for scalable million-token inference, detailed in the paper: https://lnkd.in/gCPj5eaK . In this work, we present context parallelism for long-context large language model inference, which achieves near-linear scaling for long-context prefill latency with up to 128 H100 GPUs across 16 nodes. Particularly, our method achieves 1M context prefill with Llama3 405B model in 77s (93% parallelization efficiency, 63% FLOPS utilization) and 128K context prefill in 3.8s. We have developed two lossless exact ring attention variants: pass-KV and pass-Q, which cater to a wide range of use cases with state-of-the-art performance, including full prefill, persistent KV prefill, and decode. Our benchmarks on H100 GPU hosts interconnected with RDMA and TCP demonstrate similar scalability for long-context prefill, indicating that our method scales effectively in common commercial data centers with medium-to-low inter-host bandwidth.

224

6 Comments

You've blown your compute budget calculating a Bayesian solution, and now you want to change the prior information - Help! Don't worry: this paper by Xuebin Zhao explains how prior information can be updated to provide a new solution, almost for free, using 'variational prior replacement'. This allows the effects of different hypotheses to be tested, or different regularisations to be applied, and is an approximate but computationally efficient version of the theory originally developed by Matthew Walker. Xuebin's paper:  https://lnkd.in/ejm8tktk Matt's paper:  https://lnkd.in/e-tGKwWy #geophysics #geoscience #inversion #uncertainty #imaging #geosciences #Edinburgh #EdinburghUniversity #Bayesian #variational #British_Geophysical_Association #britgeophysics

162

3 Comments

If you didn't get a chance to attend the NSF CISE Career Workshop the slides from all of ours talks are posted on the materials page: https://lnkd.in/g3dRcxEx And Josiah made a great Overleaf template: https://lnkd.in/gNdUG2kY Some key takeaways from our CAREER Awardee Talks: 1. The first page should tell your whole story - this is the first impression for the reviewer. You don't want them to have to look any further into the proposal to be sold on your idea. 2. We all had different degrees of preliminary work. Unfortunately, I don't think there is a hard metric for this - and I think it further emphasizes that the CAREER is more about your vision than anything else. 3. Have a baby? Sleep deprivation may lead to transformative research ideas - caveat: this is based on a small sample size (3 out of 4) so maybe not a strong recommendation d=

42

It is well understood that when generating text, Large Language Models (LLMs) predict multiple possible next words, each associated with a probability. An effective decoding strategy is essential for selecting the next token from the probability distribution, ensuring coherent text generation. For an insightful discussion on the various decoding strategies employed in LLMs, explore the video "LLMs: Hands-on Decoding Strategies", which delves into key techniques like greedy search, beam search, top-k sampling, and nucleus (Top-p) sampling. LLMs: Hands-on Decoding Strategies --- https://lnkd.in/gVk-5Rim #LLMs #AI #MachineLearning #NLP #DecodingStrategies #ArtificialIntelligence #LanguageModels #TechTutorial #AICommunity #Innovation #DeepLearning

29

Training LLMs to use inference-time feedback using large scale RL. Makes even the 8B Llama3.1 beat GPT-4 on CodeContests, and SOTA with the 70B. Author summary: LLMs for code should do much better if they can iterate on tests -- but they don't. Our new work (RLEF) addresses this with execution feedback at RL training time to use execution feedback at inference time. Notably, RLEF models are very sample efficient for inference. Competitive programming questions are often approached by sampling a large number of candidate programs; we can reach SOTA with just up to 3 samples. https://lnkd.in/g9_m_3Xm

18

3 Comments

I wanted to share some insights from a recent paper from Amazon Science in which authors tackled an important challenge in language modeling: minimizing sequence truncation. In language models, truncating input sequences might lead to a loss of crucial information, especially when the models are dealing with long-context tasks such as document summarization or story generation. The authors introduce a clever solution by leveraging a heuristic from the Best Fit Bin Packing problem, an NP-hard problem, to reorder input documents and reduce truncation. The proposed approach helps retain more information compared to other strategies by optimizing how sequences are packed into the model's context window. Authors show that the bin packing solution had biggest performance gain reading comprehension (+4.7%), natural-language inference (+9.3%), context following (+16.8%), and program synthesis. They also found that best-fit packing helped prevent closed-domain hallucination. Authors claim that while the experiments primarily focused on LLM pretraining, best-fit packing is broadly applicable to fine tuning as well. https://lnkd.in/gWquw3ud

28

3 Comments

🚀 **Just Published: Unlocking the Power of Large Language Models with Chunking Techniques!** 🚀 🔍 Ever wondered how large language models (LLMs) handle long documents while keeping context intact? The answer lies in **chunking**! In my latest blog, I explore the key chunking methods that boost LLM performance, covering everything from **simple sentence splits** to **adaptive AI-based chunking**. With each technique, I’ve included **code samples** to help you put these methods into action! 💡 Whether you're working in **Generative AI**, **NLP**, or **text summarization**, this guide will help you unlock new potential with LLMs. 📈 **Curious to know how it works? Check out the full blog!** https://lnkd.in/gniDQYwW --- #GenerativeAI #AI #MachineLearning #LLM #NLP #AIResearch #DataScience #Chunking #TextProcessing #AIEngineering --- For maximum engagement, tag a few companies or groups if possible, like: - **@OpenAI** - **@Google AI** - **@Hugging Face** - **@DeepMind**    Excited to share these insights and can’t wait to hear your thoughts! #llm #openai #genai #chunking #langchain

12

2 Comments

Researchers from UC Berkeley and Anyscale have introduced RouteLLM, an open-source framework for cost-effective LLM routing. RouteLLM balances price and performance by intelligently directing queries to appropriate models based on their capabilities. Key features: • Uses preference data to train routers • Achieves 95% of GPT-4's performance while reducing calls by 74% • Offers up to 48% cost reduction compared to random routing • Outperforms some commercial routing systems by 40% in cost-effectiveness • Generalizes well to different model pairs without retraining RouteLLM demonstrates significant potential for optimizing LLM deployment in real-world applications, balancing quality and cost. #AI #MachineLearning #RouteLLM​​​​​​​​​​​​​​​​ https://lnkd.in/g5UCT7Kt

111

6 Comments

Interesting paper about the connection between LLMs and structural semantics https://lnkd.in/gDPqd9Sq

35

1 Comment

Fantastic new paper on questionable practices in evaluating ML models, with an emphasis on LLMs (they list 43 bad practices!). This paper spells out why test dataset contamination is such a critical problem that ruins the integrity of benchmarking, and any results that rely solely on public benchmark performance need to be taken with a large grain of salt. Our work on Private Benchmarking is presented as a potential solution. Another solution is to evaluate more holistically with multiple methods of evaluation, including human evaluation, before drawing conclusions about performance. Paper: https://lnkd.in/gh7byYHY Our paper on private benchmarking: https://lnkd.in/gheGpcdR Nishanth Chandran, Rahul Sharma, Divya Gupta, Swami Manohar

319

6 Comments

🌟Announcing ROLEBENCH, a framework designed to evaluate the performance of Role-Prompting across various datasets and Large Language Models (LLMs). The results across some benchmark datasets demonstrates better performance than results mentioned in model cards [Refer to Github Repository for results]. A free-tier colab notebook to test ROLEBENCH on Llama 3-8B: https://lnkd.in/g4zDRHZf Role-Prompting assigns specific roles to the AI, such as acting as a teacher, a historian, or a technical expert, to see how well the model can adapt its responses based on these roles. ROLEBENCH assess how well different LLMs perform when given these role-specific prompts. This tool will offer valuable insights into the capabilities and limitations of current LLMs. ROLEBENCH currently supports Llama 3-8B, Phi-3-mini, Mistral-7B and Gemma-7B across some benchmark datasets. Github : https://lnkd.in/gbsax-hX #AI #MachineLearning #RolePrompting #AIResearch #LLM #DataScience

80

4 Comments

A recent breakthrough titled "Matrix Multiplication-Free LLMs" demonstrates a huge advancement in the area of Large Language Models (LLMs) by reducing computational costs. The authors have eliminated MatMul operations from LLMs, claiming to 10 times reduction in memory usage and a 25.6% increase in training speed, all while maintaining strong performance at billion-parameter scales. Paper link: https://lnkd.in/ggph8qXc #AI #machinelearning #deeplearning #LLMs

16

Our new work titled "Unstructured Adiabatic Quantum Optimization: Optimality with Limitations" is now out on arXiv. Link: https://lnkd.in/gs7gZyM2 Abstract: In the circuit model of quantum computing, amplitude amplification techniques can be used to find solutions to NP-hard problems defined on n-bits in time poly(n)2^{n/2}. In this work, we investigate whether such general statements can be made for adiabatic quantum optimization, as provable results regarding its performance are mostly unknown. Although a lower bound of Ω(2^{n/2}) has existed in such a setting for over a decade, a purely adiabatic algorithm with this running time has been absent. We show that adiabatic quantum optimization using an unstructured search approach results in a running time that matches this lower bound (up to a polylogarithmic factor) for a broad class of classical local spin Hamiltonians. For this, it is necessary to bound the spectral gap throughout the adiabatic evolution and compute beforehand the position of the avoided crossing with sufficient precision so as to adapt the adiabatic schedule accordingly. However, we show that the position of the avoided crossing is approximately given by a quantity that depends on the degeneracies and inverse gaps of the problem Hamiltonian and is NP-hard to compute even within a low additive precision. Furthermore, computing it exactly (or nearly exactly) is #P-hard. Our work indicates a possible limitation of adiabatic quantum optimization algorithms, leaving open the question of whether provable Grover-like speed-ups can be obtained for any optimization problem using this approach. This is joint work with Leonardo Novo, Jeremie Roland, Joseph Cunningham, Arthur Braida, and my Masters students Rutvij Menavlikar and Alapan Chaudhuri. It was a wonderful International Institute of Information Technology Hyderabad (IIITH), Université libre de Bruxelles, and INL - International Iberian Nanotechnology Laboratory collaboration. #quantumalgorithms #adiabaticquantumoptimization

57

IEEE CG&A seeks submissions for its upcoming Sep 2025 special issue on Breakthroughs, Challenges, and Opportunities in Biological Data Visualization,  with a submission deadline of Feb 21st, 2025. https://lnkd.in/ezXrXRJm The rapidly expanding data generated by advanced methods in biology is creating enormous challenges at the interface between humans and biological data that can be represented and analyzed visually. To meet these challenges, a blend of methodology from the visualization, bioinformatics and biology domains is required. This methodology encompasses perception and design knowledge, algorithm design, techniques for analyzing and visualizing big data, statistics, and specific domain knowledge for different application problems. Powerful and integrative visualization methods are necessary, as well as methods for visually communicating the outcome of computational analytical methods and findings. This special issue aims to highlight new challenges in biological data visualization (BioVis), and state-of-the-art solutions created by the vibrant BioVis community. We invite contributions on all aspects of data visualization in biology, from molecular to cell, tissue, organism and population biology. Topics of interest include, but are not limited to: * Genome and sequence data visualization, including genomic variation data, 3D and 4D genome visualization, and chromosome interaction maps * Multivariate omics data visualization (transcriptomics, proteomics, metabolomics, etc.) * Phylogenetic data visualization, as well as biological networks and pathways * Neurobiology and developmental biology visualization, biological atlases and metadata * Macromolecular structures (e.g., protein or RNA structures) visualization * Biological image data, such as microscopy data visualization * Visualization for the modeling and simulation of biological systems * Biovis systems and software frameworks, and integrating visualization and machine learning for biological analysis * Biological workflows or collaborative processes, processes for interdisciplinary collaboration between biology and visualization, and usability of visualization by biologists, including communicating uncertainty * New challenges, breakthroughs, and opportunities in biological data visualization Submission: For author information and guidelines on submission criteria, visit the Author’s Information page. Submit papers through the ScholarOne system and be sure to select the special issue or special section name. Manuscripts should not be published or currently submitted for publication elsewhere. Submit only full papers intended for review, not abstracts, to the ScholarOne portal. If requested, abstracts should be sent by email to the guest editors directly. Questions? Contact guest editors: [email protected] Elisabeta (Liz) Marai, CGA AE, Prof CS, UIllinois Chicago Jillian Aurisano, Asst Prof CS, UCincinnati

4