Jesse Engel

Thanks Hamdi Amroun, PhD - this is a great paper to share. In the real world business context we often need AI to reason over large pieces of text, and often you need/want multi-document summarization (MDS). 'When testing five LLMs with benchmarks using news and conversation datasets, they found up to 75% of content in MDS summaries was hallucinated, with notable increases towards summaries' end. Alarmingly, even for non-existent topics, models like GPT-3.5-turbo and GPT-4 generated fabricated content 79% and 44% of the time, respectively. Analysis of 700+ generated insights showed that hallucinations often arose from failures to follow instructions or overly generic content.' I agree with Hamdi's sentiments that despite improvements in post-processing techniques, robust solutions are urgently needed to reduce hallucinations, as inaccurate summaries can lead to business risks (notably misinformation and misrepresentation). I would add that there is still a lot of utility in LLMs/Multimodal models but we need to be vigilant - this is not about AI malevolence or intentional deception. LLMs do not possess consciousness or intent; they generate content based on patterns in data they were trained on.

6

1 Comment

Lin Guan and Yifan Zhou have led a very interesting study investigating the effectiveness of Vision Language Models in critiquing robot behaviors in terms of the desirability of their "style." If you are wondering how our work on LLM's inability to verify correctness squares with this study on using them as style critics, see my earlier post https://lnkd.in/gBYAr5qq ! This is the reason why being a style critic is one of the roles LLMs can play in LLM-Modulo framework: https://lnkd.in/gyAyKx4E Check out the details of the work @ https://lnkd.in/gfVZ4EYW ( Lin Guan also has a nice twitter thread on the work here: https://lnkd.in/gnieU-qM )

89

2 Comments

Small models becoming smarter! Phi-3 reinforces the importance of data quality.

24

"Why did Santa's AI-driven sleigh keep running into trees?" So much 🔥 from Julie Wang and John Allard in today’s OpenAI live stream! They covered Reinforcement Fine-Tuning, medical research, and shared some truly punny jokes. Moments like these remind me why I love my team: brilliant people solving fascinating problems—and having fun along the way! https://lnkd.in/gJnNiTXc If you'd like to participate in the Research Program by sharing data, check out https://lnkd.in/gZYehx9T or ping me. Reinforcement Fine-Tuning is a new technique for teaching the new models how to reason. It's kinda like how AlphaGo taught a model how to play by "grading" games, then learning what "good" moves are by comparing moves in winning vs losing games. This opens up a whole new paradigm for models, where you're not just teaching them the answer, but teaching them how to arrive at the right answer. What is Reinforcement Fine-Tuning (RFT)? It’s a groundbreaking technique for teaching models how to reason. Think of it like AlphaGo: the model learns by being “graded” on games and understanding what makes moves in winning games better than losing ones. With RFT, prompts are like the “games” and chain-of-thought reasoning is like the “moves.” By evaluating the "winning" reasoning steps, the model learns which ones lead to better answers. This means we’re not just teaching models the answers—we’re teaching them how to think through problems and consistently arrive at the right solutions. My team’s mission is simple: give you the world’s best model for your domain. If you’re finding it hard to get API responses to work for your business—whether it’s a performance issue, cost concern, or accuracy challenge—let me know. We have tools to improve outcomes, and I love working with customers to see how we can do even more. Oh, and Santa’s sleigh? Julie gave us the answer: “Because he hadn’t ‘pine’-tuned yet!” 🥁

22

1 Comment

📜 A new blog post explaining the fundamentals of RL for LLMs—the perfect place to intuitively understand PPO and RLHF from first principles. 👇👇 Pre-trained LLMs are often compared to Lovecraft's Shoggoth, a monstrous, shape-shifting beast. PPO is the algorithm that brought it under control. Take a look! 🔗 https://lnkd.in/eXqDvrUV

29

1 Comment

I enjoyed this blog post on agents. Some historical context is always great. It argues that we have much better reasoning engines with LLMs, and by building engineering around them, we can also make agentic workflows more reliable. https://lnkd.in/dBFBS9zM

38

2 Comments

After reading Anthropic and OpenAI success stories with sparse autoencoders for interpretability of text, I was curious about its application to images... After a quick search, I found out that Hugo Fry wrote a blogpost in LessWrong: https://lnkd.in/dA33fWqE which also provides some playground! Of course part of the success is due to the ImageNet structure itself (as many papers out there) but still is an interesting exercise. It enables lots of experiments, and I already can think about practical usage for improving image search in marketplaces. #interpretability #image #CLIP

7

3 Comments

Large Language Models (LLMs) as super lossy compression. I have been thinking a lot about demystifying what LLMs do because lots of y'all's are very mystified. In the linked article, I try to reframe them for you, hoping it will help you understand what they good for and what they are not good for. Side note: I am still looking for my next adventure. If communicating complicated concepts so that regular people can grasp them is something you need, you should message me. I write code and stuff too and am usually fun to be around. #WrittenByMe

6

3 Comments

Apart from daily stuff around LLMs,RAG Vector Databases and Gen AI. 😄 Today at 11 AM our time, randomly I have got a link to read through @Stephen Wolfarm’s writing on “Why does the Universe exist? Some Perspectives from our Physics project”. It was a joyous read and stimulated my thought process which was subconsciously bothering/exciting me in my mind for a quite long time. I, generally, never write articles on any social media platforms/Networking websites. But today’s pondering on the said subject insisted me to do so. Stephen is a mathematician, physicist, computer scientist and an AI expert, his writing touched maths, physics, logic as well as AI to some extent. Stephen Wolfarm touched the subjects of metaphysics, philosophies and religious beliefs to explain the core subject of Universe’s existence but didn’t elaborate on this theoretical subject because of his scientific thinking and logic. He has a great mindset which only get satisfied with reasoning backed by the science. My two cents are about not to get solid answers around the topic as I feel nobody can satisfy me in this subject, nor I might meet/communicate with a guy who’s proven superior knowledge on the fundamental question of “Why does the Universe exist”? From my perspective, Initial take about “Universe existence” is not a single question but bunch of queries intertwined with intricacies and complexities. Here, I need to confess my ignorance about not having good knowledge about quantum mechanics and big bang theory on universe formation/expansion/ending, also, I never gone through Stephen hawking’s books on this subject though I have been hearing about them from many of my friends. Questions around the universe are many; starting to ask, is this universe existing now is at nascent stage or half way through or about to end soon? Is it analogous to the destiny of black holes? With little knowledge people say the galaxies have expiry dates, my query is so does universe? One more interesting query by Stephen on the topic, does universe is single? or many in count? Every creature/living being carry fate of death on earth, does it apply to universe too? Vaguely, I feel yes, because galaxies are dying means at higher level universe might die also one or other day. Parallelly, things happening in universe like sun rise, full moon day, earth rotation, motion of all nine planets. May be limit our scope of discussion here up to milky way galaxy, the daily routine of nook and corner of milky way galaxy is happening so correctly; that said, who is governing all these natural things? Don’t tell gravitational force or some mathematical/physics topic. If nature is doing that based on physics principles, who is governing it? God or consciousness? I am reluctant to use that word consciousness because by that I am peeping into philosophy, also god who is linked to religious realms of thought. #llms #nlproc #GenAI

1

Frontiers | Global explanation supervision for Graph Neural Networks The paper discusses various methods for explaining and understanding neural networks. Several authors have proposed techniques to generate explanations for graph neural networks, including GNNExplainer, XGNN, and RelEx. Other researchers have focused on developing methods for attributing the output of a model to specific input features or relationships within the data, such as few-shot learning with per-sample rich supervision. Graph Neural Networks: Foundations, Frontiers, and Applications provides an overview of the current state of graph neural networks. The book covers the basics of GNNs, as well as their applications in various fields. The automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain is discussed in relation to neuroimage analysis. A probabilistic graphical model explanation for graph neural networks, PGM-explainer, is also introduced. Learning deep attribution priors based on prior knowledge is another technique that has been explored by researchers. The Wu-Minn Human Connectome Project provides an overview of the project's objectives and methods. The project aims to investigate human brain development and function across various life stages. Various authors have contributed to this research, including N. Tzourio-Mazoyer, D. Papathanassiou, and B. Landeau. The work presented in these papers showcases the diversity of techniques being developed to explain and understand neural networks, from graph neural networks to probabilistic graphical models. https://ift.tt/iMDS8aU #news #cto #tech

1

Excited to share the work that Chris Glaze Changho Shin Frederic Sala and I did on long context LLM evaluations. While there’s plenty of problems with LLM evals, a recurring one we’ve seen at Snorkel, working with enterprise clients, is having to rely on far too simple/toy benchmarks that do not relate to business problems. It is why we developed SWiM, a framework that lets users test long context model capabilities directly on their own data and tasks. Read the paper here: https://lnkd.in/g3uSuKcW

31

I wanted to share some insights from a recent paper from Amazon Science in which authors tackled an important challenge in language modeling: minimizing sequence truncation. In language models, truncating input sequences might lead to a loss of crucial information, especially when the models are dealing with long-context tasks such as document summarization or story generation. The authors introduce a clever solution by leveraging a heuristic from the Best Fit Bin Packing problem, an NP-hard problem, to reorder input documents and reduce truncation. The proposed approach helps retain more information compared to other strategies by optimizing how sequences are packed into the model's context window. Authors show that the bin packing solution had biggest performance gain reading comprehension (+4.7%), natural-language inference (+9.3%), context following (+16.8%), and program synthesis. They also found that best-fit packing helped prevent closed-domain hallucination. Authors claim that while the experiments primarily focused on LLM pretraining, best-fit packing is broadly applicable to fine tuning as well. https://lnkd.in/gWquw3ud

28

3 Comments

🦗 Relatively quiet day on the quantum front — but there is quantum excitement to be had in Berkeley Lab’s process for identifying and fabricating quantum materials. Plus, if there were QIS standards for secondary schooling, these would be it. ⚆ Researchers at Berkeley Lab have combined rapid computational predictions with precise fabrication techniques to accelerate the discovery of quantum materials; the results are shared in a publicly available database called the Quantum Defect Genome ⚆ IBM has become a partner in the Superconducting Quantum Materials and Systems Center at Fermilab, focusing on advancing superconducting quantum computing technology through tackling challenges in quantum computing, communication, large-scale deployment, and the workforce ⚆ Quandela and Welinq have teamed up to combine their expertise in quantum processors and interconnect technology to create custom quantum links for photonic quantum computers ⚆ attocube systems Inc is providing essential tools for the quantum technology supply chain to improve energy efficiency for scalable quantum computing applications ⚆ The case for integrating quantum informatics into secondary school curricula is made by positioning it within traditional computer science education, aligning it with the Great Principles of Computing, and establishing standards for expected student outcomes ⚆ Don't miss a single qubit -- follow me & subscribe to the free newsletter for your daily quantum computing news & research insights. hashtag #quantumcomputing #quantumtechnology #innovation #quantumscience #quantumphysics

3

If you want to dive deeper into Gen AI, specifically fine-tuning LLMs, check out this great (and free) introductory course by Sharon Zhou, PhD! It provides a nearly 2-hour walkthrough with coding examples using Python. Requires some basic knowledge of Python and machine learning so you don't get lost 😱 😆

7

AI has accelerated materials discovery, akin to advances in protein folding and language models, but often struggles to translate predictions to the physical world due to the lack of experimental data. AI at Meta just announced that they took on a new bold domain. They ran ~700 million AI-accelerated simulations, analyzed 20,000 theoretical materials and synthesized ~600 brand new metal alloys with novel properties for catalysis. This exercise bridges the gap between simulations and reality like nothing else, accelerating new materials development by 10-100x. The resulting material set now becomes part of the Open Catalyst library, and can be used to create novel electrocatalysts for carbon capture and other applications to solve our climate crisis. None of this would have been possible without a frontier tech startup from Europe that Meta and University of Toronto relayed on this work. Because it is VSPARTICLE that have created the machines that make software defined materials at scale. When I first met founder Aaike van Vugt, Tobias Pfeiffer and Tobias Coppejans about three years ago, I was blown away by the technology their team in Delft, Netherlands were developing. VSParticle’s technology can break most of the periodic table down to the nano-level particles, mix and match those, and “print” the result physically out. Turning software-defined “dream” of a material into a physical, real thing is magical, and accelerates the discovery of next-generation materials. This breakthrough technology holds significant potential for addressing global challenges such as the climate crisis. VSParticle’s collaboration with Meta's FAIR research team is a huge step for how this vision from a decade ago becomes real. Meta runs a mean AI shop, but without VSParticle's technology it would have taken them years or decades in wet labs to produce actual samples of their theoretical discoveries. This would have made the rapid training loops for their AI infeasible. Decarbonising society requires technology like VSParticle's as we need to manipulate molecules, not just bits. It’s because of companies like VSParticle that I believe that Europe has what it needs to be the exporter of the tech that will change the climate crisis. We have the universities, the entrepreneurs, investors and the policymakers here willing to create and support these kinds of complex deep technologies that will make a difference globally.    Congratulations from all of us at Plural to the VSParticle team and here’s to the next million new materials found by AI!

131

7 Comments

This paper is inspired! Much of the current AI weather model research is focused on more accurate and faster versions of what we already have. That's important, but the most exciting advances will come from asking "what *new capabilities* can AI models provide?" I've heard the phrase "AI models will allow us to ask new science questions" before. These folks actually did it and found something fascinating. [Side note: the approach they use find a more accurate initial state is the same one we use to assimilate near real time streamflow observations into HydroForecast at Upstream Tech. Cool!]

35

3 Comments

🌶 Those fancy new LLMs beating standard evaluation benchmarks are all just talk until stress-tested in the real world (Ask any generative AI practitioner, and they'll likely agree). 😅 While open LLM leaderboards exist, I think that it's super important for the LLM community to prioritize the development of more robust and standardized benchmarks tailored to the current landscape. A recent initiative that looks very promising is the User Reported Scenarios (URS) benchmark. Here are some insights: ⛳ Many existing benchmarks primarily focus on specific predefined model abilities, such as world knowledge, mathematics, or coding skills. However, they often overlook the intended usage of LLMs by real users in diverse scenarios. ⛳The User Reported Scenarios (URS) benchmark addresses this by collecting 1,846 real-world use cases from 712 participants across 23 countries. These scenarios are categorized based on 7 user intents, reflecting the diverse ways users interact with LLMs. ⛳The user intents in the User Reported Scenarios (URS) benchmark are categorized into seven main categories: 🌟 Factual Question Answering 🌟Professional Problem Solving 🌟Text Assisting 🌟Asking for Advice 🌟Seeking Creativity 🌟Leisure 🌟Usage through API ⛳The evaluation process involves generating reference answers for each test case, which are then checked by humans. The test model generates its own output based on questions. This output, along with the question, corresponding user intent, and reference answer, is provided to a strong evaluator model for scoring. The final scores, determined by a parser, form the benchmark results. Read the paper here: https://lnkd.in/e8eQ5H2P 🚨 I post #genai content daily, follow along for the latest updates! #genai #llms #evaluation

181

2 Comments