Benoit Dupin

Check out our new work on LLM reasoning! A fine-tuning-free approach that can reach the reasoning level of 300B+ closed-resource LLMs using only ~10B open-source LLMs #LLM #Reasoning #M*

13

Thanks Hamdi Amroun, PhD - this is a great paper to share. In the real world business context we often need AI to reason over large pieces of text, and often you need/want multi-document summarization (MDS). 'When testing five LLMs with benchmarks using news and conversation datasets, they found up to 75% of content in MDS summaries was hallucinated, with notable increases towards summaries' end. Alarmingly, even for non-existent topics, models like GPT-3.5-turbo and GPT-4 generated fabricated content 79% and 44% of the time, respectively. Analysis of 700+ generated insights showed that hallucinations often arose from failures to follow instructions or overly generic content.' I agree with Hamdi's sentiments that despite improvements in post-processing techniques, robust solutions are urgently needed to reduce hallucinations, as inaccurate summaries can lead to business risks (notably misinformation and misrepresentation). I would add that there is still a lot of utility in LLMs/Multimodal models but we need to be vigilant - this is not about AI malevolence or intentional deception. LLMs do not possess consciousness or intent; they generate content based on patterns in data they were trained on.

6

1 Comment

I remember when this all started. Faris - invited me to catch up at our favorite spot around Blok M. I had no idea he’d just wrapped up a procedure at Pondok Indah Hospital! (He really knows how to add drama to our catch-ups.) We’d been on separate paths and projects since "the winding down project", but there he was, reminding me of our dream to ride this AI wave together again. It was a plan we’d hatched with Sahal Zain & Gurnoor Dhillon when the world of AI was still mostly theoretical and jargon-heavy. Somehow, the conversation continued with Gurnoor Dhillon while he was waiting for his meeting with someone who would eventually become one of CalvinBall’s key early adopters. It felt like everything was coming together, like a series of seemingly random dots connecting to form a bigger picture. Seeing how far we’ve come as a team at CalvinBall Technologies is truly exciting. What started as theoretical discussions has now evolved into practical Gen AI solutions that are solving everyday problems and hold endless possibilities to shape the industry. 👨‍🚀 👩‍🚀 Here’s to pushing boundaries and achieving new milestones together! Gurnoor Dhillon, Sandeep Ramesh, Faris -, Sahal Zain, Eva Zuliya, Ratu Nadiah Khairunnisa 🚀 #GenAI #AIInnovation #TechJourney #ArtificialIntelligence #TechTransformation #CalvinballTech #Calvinballers

55

2 Comments

Very insightful work by Anthropic’s interpretability team. And an amazing paper, with outstanding writing and figures. The idea is very simple: interpret LLMs by leveraging sparse autoencoders as surrogate models of the MLP of transformer blocks, which allow one to disambiguate the superposition of features captured by a single neuron. A simple idea, but a very careful and complex execution, as it is often the case in our line of work. The paper goes into many details and provide a large array of insights, although the gist of the implementation remains obfuscated due to the closed source nature of Claude. Too bad, because this is the kind of work that we need to better understand and eventually trust LLMs. This is demonstrated by the authors in the section ‘Influence on Behavior’, where they show that clamping some features to either high or low value during inference is “remarkably effective at modifying model outputs in specific, interpretable ways”. Hopefully this kind of work is going to be replicated and generalized to open-weights models, such that we have new ways to steer their behavior. https://lnkd.in/eVym7f_f #interpretability #xai #explainableai #steerableai #anthropic #claude #anthropic

3

If you haven’t tried the Nvidia NIMs, you don’t know what you are missing. The latest NV-CLIP NIM microservice, a cutting-edge multimodal foundation model is now available. Designed to generate image and text embeddings,you can use it for multimodal search, zero shot classification, and more. Try now. https://lnkd.in/gEKqq-fF

44

Lin Guan and Yifan Zhou have led a very interesting study investigating the effectiveness of Vision Language Models in critiquing robot behaviors in terms of the desirability of their "style." If you are wondering how our work on LLM's inability to verify correctness squares with this study on using them as style critics, see my earlier post https://lnkd.in/gBYAr5qq ! This is the reason why being a style critic is one of the roles LLMs can play in LLM-Modulo framework: https://lnkd.in/gyAyKx4E Check out the details of the work @ https://lnkd.in/gfVZ4EYW ( Lin Guan also has a nice twitter thread on the work here: https://lnkd.in/gnieU-qM )

89

2 Comments

Great work by the team producing this collection of assets on how to use NVIDIA NIM on AWS to optimize your inference performance to get state-of-the-art deployments. Running inference endpoints across Amazon's NVIDIA GPUs instances on EC2, Amazon EKS, and Amazon SageMaker has never been faster or more efficient. Here’s why NVIDIA NIM is the best solution for AI developers and enterprises: ✅ Lightning-fast Deployment: Prebuilt containers with optimized generative AI models, deployable with just a few commands, help you get started in minutes. ✅ Unparalleled Performance: Experience up to 2.5x higher throughput, 4x faster time-to-first-token, and 2.2x faster inter-token latency compared to other open-source solutions—because speed and efficiency matter when scaling AI. ✅ Seamless Integration with AWS: NIM is purpose-built to integrate with AWS managed services like EKS, EC2, and SageMaker, providing a streamlined path from development to production, no matter where your AI models run. ✅ Enterprise-Grade Security and Flexibility: Maintain full control of your data, IP, and custom models while scaling AI workloads in a secure and enterprise-ready environment. ✅ Cost-Effective and Scalable: Reduce time-to-market and operational costs, optimizing response times and throughput, all while leveraging the robust infrastructure of AWS. Whether you’re deploying large language models or domain-specific AI, NVIDIA NIM on AWS empowers you to deliver high-performance AI applications, bringing vision to reality at scale. #GenerativeAI #AIinference #NVIDIANIM #AWS #EC2 #EKS #SageMaker #AI https://lnkd.in/e722U8Np Abhishek Sawarkar Brian Pickering Guy Farber Adel El Hallak Jiahong Liu Timothy Ma Saurabh Trikande Vara Bonthu

117

3 Comments

https://lnkd.in/dJW9sZpx SelfExtend introduces a novel approach to expanding the context window of LLMs. It leverages a two-tiered attention mechanism—grouped attention for long-range dependencies and neighbor attention for local interactions—both derived from the model's original self-attention. This method allows seamless context window extension without requiring fine-tuning, and extensive experiments have demonstrated its effectiveness across various benchmarks.

12

1 Comment

Large Language Models (LLMs) are absolute powerhouses when it comes to generating top-notch #GenAI responses. Yet, #LLMs lack that in-depth, domain-specific knowledge that can make all the difference in certain use cases. Additionally, let's not forget the hefty computing resources and expertise required to develop and deploy them. In my latest article, I delve into why Small Language Models (SLMs) are becoming the go-to choice for specific tasks. With their more focused approach, #SLMs are more efficient than LLMs, and have a smaller compute footprint. Thus, they are more suitable to run locally on workstations or on-prem servers. Check out my article 👇 and ‘d love to hear your thoughts about it https://lnkd.in/diRYaNXF #DellTechEMEA #iWork4Dell #AMD, #Intel, #NVIDIA 

143

7 Comments

The 7th workshop on Neural Scaling Laws is about to start: https://lnkd.in/gQSB5k8v Scaling laws, emergency, alignment, transfer learning, with a focused track on multilingual models. Looking forward to kick it off with Irina Rish and discuss all these topics with Yuhai Tu, Paul Bogdan, Hailey Schoelkopf, Darshil Doshi, Rio Yokota, Neha Sengupta, Preslav Nakov, Tatiana Shavrina, PhD, Yishi Xu. Join in person or virtually! #icml2024 #ai

37

Faculty friends: We've recently opened the NVIDIA Academic Grant applications. The "Graphics & Vision" track covers lots of XR and Spatial computing topics: == Augmented Reality and Virtual Reality == > Digital twins > Human-computer interaction for immersive visualization > Immersive analytics > AI for immersive AR / VR > High-end visualization systems (multi-node, large scale visualization research systems) > 3D scene composition and content capture

56

1 Comment

The keynote speech by Trevor Cai from #OpenAI at #HotChips 2024 was interesting. Trevor's presentation reiterated the ongoing hypothesis at OpenAI—that the AI models must continue to grow to achieve superior performance, with no apparent end in sight... He bolstered this argument using a log-log power law plot, which essentially plots the relationship between compute (in FLOPs) and model performance (i.e., loss) on a logarithmic scale. This approach has enabled OpenAI to accurately predict the performance of models like GPT-4 by curve-fitting the smaller models on the plot, thus demonstrating the scalability of this method. Trevor then drew a fascinating parallel to Moore's Law, which famously predicted the doubling of transistors on chips every two years. This prediction held true for decades, fueling massive advancements in semiconductor technology. He suggested that similar underlying principles might sustain the exponential growth of AI models for a significant time, though the mechanisms might evolve. However, Trevor notes that such growth isn't without its challenges. He highlighted several critical issues, including the need for advanced data centers, enhanced reliability, availability, and serviceability (RAS) systems, and more sophisticated power management strategies. The massive scale of synchronized training introduces vulnerabilities like power draw jitters and single-point failures that can cause widespread disruptions. If synchronized training causes so many challenges, I wonder if the research community is focusing on alternative algorithms for LLMs that don't require gradient synchronization after each iteration.? 🤔 💡 Trevor points out that as AI models became more intelligent, the demand for inference has grown significantly. He notes that while training infrastructure can be planned and scaled, inference capacity must be more responsive to actual usage and demands, making planning more challenging and dynamic. The technology and economics of inference will continue to evolve. While the conference is about hot chips, he has no specific recommendations on GPU/accelerator architectures, whether inference could benefit from custom hardware, etc. Overall, it is evident that openAI will continue to grow the models and datasets it trained on for each new release of GPT and will continue to challenge the AI/GPU and networking hardware industry to support this with technological advancements. At some point in the future, more than half the data on the Internet will have originated from the LLM models. At that point, how would the log-log plot look? 🤔 Wouldn’t models trained on their self-generated data collapse, and the losses start to grow? He acknowledges the problem in Q&A and says they are "investigating" techniques to overcome this…. Any thoughts? #AI #LLMs #datacenters #junipernetworks

83

8 Comments

Google has unveiled "Willow," a groundbreaking quantum computing chip that can complete calculations in minutes that would take conventional computers 10 septillion years. This 4cm² chip, developed in Santa Barbara, shows unprecedented error correction capabilities and could revolutionize drug development, AI advancement, and material science research. Key breakthrough: The chip addresses quantum computing's historic challenge of error correction, marking a significant step toward practical quantum applications. According to Dr. Peter Leak from Oxford Quantum Institute, we're likely just 5 years away from seeing real-world quantum computing applications. While this advance promises transformative potential in healthcare and scientific research, it also raises critical considerations for cybersecurity. The ability to break current encryption methods means industries must accelerate their transition to quantum-resistant security protocols. This isn't just a technological milestone - it's a wake-up call for cybersecurity preparedness across all sectors. #QuantumComputing #ArtificialIntelligence #TechInnovation #FutureOfTechnology #GoogleAI #CyberSecurity #Innovation #EmergingTech #QuantumSupremacy #TechNews https://lnkd.in/eYv8xU9m

8

2 Comments

Explore Geneformer, a cutting-edge #AI #deeplearning model, now available through #BioNeMo framework on NVIDIA NGC. Pre-trained on extensive transcriptome data, it uses attention mechanisms to focus on crucial #gene interactions, enhancing #drugdiscovery and #geneticresearch.

12

#LLM sizes will be a significant consideration in the next generation of products that we will be witnessing in the coming years. Router models cannot be the go-to solution eventually. As a founder in this domain, one needs to think beyond constrained boxes and democratize these powerful systems as much as possible for small and medium business. We are planning to set things straight in the coming months. #Angel #angelfunding #VC #investments #VCfunding #AI #AIStartup #Quantization #LLMs #LLM

4

Hallucinations in LLMs are inevitable, no seriously! Josh Fritzsche and I have worked hard at Voze to find a way to tune, prompt and even plead 🙏 with LLMs to prevent them from the occasional 🍄 hallucination. It seemed like the better we have done the harder the edge cases have been to solve and eventually we realized that they tend to frequently happen when LLMs predict or make anticipations on "nothing". I remember at one point nearly going insane with frustration because when LLMs had a hallucination, they were EPIC! 😱 Then we pushed through it and benefited for the effort! How did we solve this common problem? We didn't (Not completely), we got better at identifying the issues as they happened. Once we identified in the data we were able to start looking for patterns. The same catalysts that cause it to happen once if left unchanged caused it over and over again. Our solution is astoundingly simple. Once you know that it's likely to happen..... let it! Accept it will 🍄, but also be preemptive in not focusing on it and use alternative data sources, generators or providers. Change the conditions that cause it but continue to monitor it as it happens. As Dan Caffee is fond of saying to me on a regular bases. "Dusty we need the system to be adaptive and provide mechanisms to learn from it's past." Well he is right! All we needed was data and the people that contribute to it's quality. Huge props 👏 to strong support from Kathy and Janelles audit teams. We were armed with a knowledge and a desire to learn from past mistakes. Armed with confidence that we could correct these problems we could focus on the data to prevent their impact going forward. When doing millions of LLM (Generative AI) calls regularly Josh and I have learned that hallucinations will happen. We also know exactly how we will handle and prevent it going forward! Failure is not doing anything and accepting defeat.

4

2 Comments

I was fortunate to attend Siggraph, a conference on computer graphics and generative AI. Jensen Huang, CEO of NVIDIA, gave a keynote address at the conference. He discussed NVIDIA's journey from a computer graphics company to a leader in accelerated computing and generative AI. Democratizing these tech advancements to general purpose use cases and enabling companies to build capabilities is still a far journey. Some of the thought provoking points that he mentioned are here: - NVIDIA's history: Started in 1993, focused on computer graphics, pioneered GPU technology, and entered the AI market in 2012 with the acquisition of AlexNet. - Accelerated computing: NVIDIA's focus on creating a computing model that could augment general-purpose computing to solve complex problems. - Generative AI: A new way of writing software where machines learn to program themselves, given input and example outputs. - NVIDIA's role: Developing the hardware (GPUs) and software (like CUDA) to power generative AI applications. - Future of AI: More controllable and accurate through techniques like reinforcement learning, guardrails, and retrieval augmented generation (RAG). - Impact on industries: Generative AI will transform various sectors like scientific computing, design, robotics, and customer service. - NVIDIA's strategy: Focus on software development and domain-specific libraries (DSLs) to maintain its lead in accelerated computing. Challenges and opportunities: Addressing ethical concerns around AI, such as potential job displacement and the need for transparency. - NVIDIA's vision: Creating a future where AI assistants augment human capabilities across industries and democratize access to powerful tools. #AI #Trending #GenAI #acceleratedcomputing #GPUs #CUDA

16

1 Comment

Agreed Srini Pagidyala. This is where we need to to understand the limitations of LLM or Generative AI. Accurately representing the real world requires interaction with it. I feel that it requires more brain functions added to the generative part. The generative part is like our brain creating abstract thoughts based on experiences (data). Now some entity should course correct those thoughts to match with the real world. Our human brain does interact with the real world. Point here is that LLM should have the capability to collect relevant world data (from the context it is generating for) and course correct it’s content. Well a far fetch for today’s LLMs

5