Yann LeCun

Lin Guan and Yifan Zhou have led a very interesting study investigating the effectiveness of Vision Language Models in critiquing robot behaviors in terms of the desirability of their "style." If you are wondering how our work on LLM's inability to verify correctness squares with this study on using them as style critics, see my earlier post https://lnkd.in/gBYAr5qq ! This is the reason why being a style critic is one of the roles LLMs can play in LLM-Modulo framework: https://lnkd.in/gyAyKx4E Check out the details of the work @ https://lnkd.in/gfVZ4EYW ( Lin Guan also has a nice twitter thread on the work here: https://lnkd.in/gnieU-qM )

89

2 Comments

Anthropic just dropped Computer Use and Claude 3.5 Sonnet & Haiku. Anthropic has recently announced the release of their 3.5 models, pushing the boundaries of AI capabilities and their applications for efficient computer use. These models not only demonstrate improved language comprehension and reasoning but also enhance usability across a wide range of tasks—from automating processes to driving smarter interactions with technology. The potential here is immense: these advancements could lead to more intuitive, AI-powered tools that seamlessly integrate into our daily workflows, transforming everything from how we manage data to how we innovate with machine learning models. 🚀 As AI continues to evolve, this is a crucial step towards more responsible and powerful AI solutions, making technology even more accessible and aligned with human values. Looking forward to seeing how this influences the future of AI research and real-world applications! This will end up being a major change in ai agents ngl. Hopefully, the control doesnt go to someone with bad mindset. You can see all the example use cases in this X thread: https://lnkd.in/dhb_NfB5

20

Polaris 2.0 from Hippocratic AI is here. Safer than ever. 3T+ parameter constellation architecture with voice latency equal to Polaris 1.0 at 1T+ parameters. Read a summary of the changes from our chief scientist Subhabrata (Subho) Mukherjee's blog: https://lnkd.in/gXweaWe5

278

1 Comment

I'm happy to share our latest research, created with the amazing Deci team at NVIDIA: 𝐏𝐮𝐳𝐳𝐥𝐞: 𝐃𝐢𝐬𝐭𝐢𝐥𝐥𝐚𝐭𝐢𝐨𝐧-𝐁𝐚𝐬𝐞𝐝 𝐍𝐀𝐒 𝐟𝐨𝐫 𝐈𝐧𝐟𝐞𝐫𝐞𝐧𝐜𝐞-𝐎𝐩𝐭𝐢𝐦𝐢𝐳𝐞𝐝 𝐋𝐋𝐌𝐬 Read the full paper here: https://lnkd.in/drukCaAw Introducing Puzzle, a framework to accelerate LLM inference on specific hardware while preserving their capabilities. Using an innovative decomposed neural architecture search (NAS), we optimize models with tens of billions of parameters under various hardware constraints such as throughput and memory limitations. Puzzle leverages blockwise local knowledge distillation and mixed-integer programming to explore a vast architecture space. By creating non-uniform models with adapted computation allocation, we optimize each layer's expressivity, achieving significant efficiency gains without compromising performance. We demonstrate Puzzle with Nemotron-51B, derived from Llama-3.1-70B-Instruct, achieving a 2.17× inference speedup on a single NVIDIA H100 GPU while preserving 98.4% of the original capabilities, using only 45B training tokens versus 15T for the original model. Try a demo of Nemotron-51B here:  https://lnkd.in/du3Kcc6H #LLM #NeuralArchitectureSearch #DeepLearning #NAS #InferenceAcceleration #MachineLearning #NVIDIA

42

1 Comment

This new article discusses our efforts in delivering tensor parallelism, weight-only quantization, vLLM integration, and the RBLN model zoo. It is definitely worth reading! 👉 https://lnkd.in/gRnG8_pR #TensorParallelism #WeightOnlyQuantization #vLLM #ModelZoo

84

Emerging technologies don’t simply materialize fully formed; they require the establishment of supply chains, the development of manufacturing processes, and the evolution of the technology itself. Along this journey, the cost incurred includes learning from the ten unsuccessful paths before arriving at the eleventh one that proves effective…a perceptive read if you are working in the Spatial Computing space

20

"Impressive acceleration in AI capabilities! Anthropic’s new models, including Haiku 3.5, are setting new standards, with 3.5 outperforming Opus at the same cost as the previous Haiku 3. But what's really exciting is the introduction of visual agent capability in the new computer-use beta, released October 23 on the API. Now, developers can direct Claude to interact with a computer just like a human—navigating screens, moving cursors, clicking buttons, and typing text. This opens up huge potential for automation platforms like Zapier and other workflow tools, which could soon leverage AI agents to perform visual tasks across interfaces, expanding the possibilities for intuitive, hands-on automation. The implications for user-friendly automation and interaction across digital platforms are transformative!" see https://lnkd.in/e5HbYsf7

3

1 Comment

In computational linguistics, researchers are focused on enhancing language models' ability to extract precise information from extensive textual data. Recent studies introduce "retrieval heads," specialized attention mechanisms that selectively focus on critical segments of large texts, improving targeted data retrieval. These heads, integrated into transformer-based models like LLaMA and Yi, significantly enhance accuracy and efficiency, particularly in handling long-context scenarios. The methodology involves thorough experiments across various models, including the Needle-in-a-Haystack test, which embeds specific information to measure precision. Results show models with active retrieval heads outperform those without, with accuracy dropping notably when these heads are masked. The empirical evidence underscores the effectiveness of retrieval heads in improving information retrieval precision and reliability. This research deepens our understanding of attention mechanisms in large-scale text processing, paving the way for more efficient and accurate language models. https://lnkd.in/dNNPvMXw https://lnkd.in/ddbHX6EA Innovation Hacks AI #aiinnovation #generativeai

1

Learn a development pattern to systematically improve the accuracy and reliability of LLM applications in our new short course, Improving Accuracy of LLM Applications, built in partnership with Lamini and Meta, and taught by Lamini’s CEO, Sharon Zhou, PhD, and Meta’s Senior Director of Partner Engineering, Amit Sangani. (Disclosure: I am an investor in Lamini.) The path to tuning an LLM application can be complex. In this course, you'll learn a systematic sequence of steps for improving accuracy by reducing hallucinations:  - Create an evaluation dataset to measure model accuracy - Add prompt engineering and self-reflection - Fine-tune your model including "memory-tuning" which is a new method of embedding facts in an LLM Using the Llama 3-8B parameter model, you will: - Build a text-to-SQL agent with a custom schema and simulate situations where it hallucinates - Understand the difference between instruction fine-tuning, which gives pre-trained LLMs instructions to follow, and memory fine-tuning - See how Performance-Efficient Fine-tuning (PEFT) techniques like Low-Rank Adaptation (LoRA) reduce training time by 100x and Mixture of Memory Experts (MoME) reduces it even further I appreciate Meta releasing the Llama's family of open models -- this course gives an example of the unique type of work that developers can do with such models. Please sign up here: https://lnkd.in/gyxHwPkp

3,934

72 Comments

It’s finally here! I’m thrilled to introduce you to Llama 3, our latest generation of models that is setting a new standard for state-of-the art performance and efficiency for openly available LLMs. Key highlights: • 8B and 70B parameter openly available pre-trained and fine-tuned models • Trained on more than 15T tokens, 7x+ larger than Llama 2's dataset! • State-of-the-art performance across industry benchmarks • New capabilities, including enhanced reasoning and coding • 3x more efficient training than Llama 2 • New trust and safety tools with Llama Guard 2, Code Shield, and CyberSec Eval 2. • Training underway on a leading-edge 400B+ parameter model  We’re also releasing a new version of Meta AI powered by Llama 3. We’re making it available in 13 new countries, integrating enhanced search capabilities and adding new ways to experience it on the web at meta.ai. Our latest updates to image generation on Meta AI is state of the art, allowing you to create, animate and share stunning images with a simple text prompt in seconds. We think Meta AI is the world’s most intelligent AI assistant and we can't wait for you to try it and tell us what you think. Read more about it here: https://lnkd.in/eB5yzi8R

2,385

80 Comments

This week, I'm interviewing extraordinary Bradley Voytek — neuroscience prof at UC San Diego, founding member of UCSD's Data Science Institute, and he was Uber's first data scientist. Got questions for him? Questions at the intersection of neuroscience and data science would be particularly interesting, given that Prof. Voytek's research specialty is combining large-scale data science and machine learning to study how brain regions communicate with one another, and how that communication changes with aging and disease. (UCSD is arguably the world's best neuroscience research institute, so this is quite a special opportunity for ya!) More detail on Brad: • Professor in UC San Diego's Department of Cognitive Science, the Halıcıoğlu Data Science Institute, and the Neurosciences Graduate Program. • Alfred P. Sloan Neuroscience Research Fellow. • Kavli Fellow of the National Academies of Sciences. • Joined Uber as their first data scientist, when it was a 10-person startup, helping build their data science strategy and team. • Outreach work has appeared in Scientific American, NPR... and Comic-Con! • Co-authored the amusing book "Do Zombies Dream of Undead Sheep?" The "Super Data Science Podcast with Jon Krohn" is available on all major podcasting platforms and a video version is on YouTube. Brad's episode will likely be #829 and be released on October 22nd. #superdatascience #machinelearning #ai #datascience #neuroscience

86

1 Comment

Are you the curious type that enjoys discovering and learning exciting new sub-areas and applications of Machine Learning? Want to get a deeper understanding of how ML is used for physics and engineering? Heard Jensen Huang repeat excitedly "Digital twin", and you couldn't understand what's the big fuss about? Then this awesome YouTube Play list is for you - Physics Informed Machine Learning - great set of video lectures by Steven Brunton. The lectures are engaging and clear, even for people with fairly basic background in ML and Physics. It's still evolving, but there is already a great set of new introductory lectures, as well as curated deep dive lectures by Steve from previous years. This is a somewhat similar course from the CAMLab in ETH - https://lnkd.in/eDK-u8SS, which I watched last year. It's also excellent and covers the key topics, though I think it's a bit less accessible than Steve's. Try both and let me know 😄 I recommend! 👍👌 https://lnkd.in/eMeKWtdx

37

3 Comments

Discover the benefits of the Cerebras AI Model studio: - Train 1-175 billion parameter models 8x faster than most other publicly available GPU instances - Enable higher-performing models with longer sequence lengths (up to 50,000!) - Train your models from scratch or fine-tune open-source models with your data Learn more about our partnership with Cerebras Systems: https://hubs.la/Q02tw4KY0

9

If you think compute scaling will plateau, think again. We are in the baby states of inference time compute. VLLMs coupled with computer tools, reward learning and RL can leverage orders of magnitude more inference time compute than what we have seen with foundation models. Here’s a concrete training architecture (replace \pi with a large pre-trained vLLM and add it to the reward learner too, \R is the space of all computer tools)

22

1 Comment

In the race to advance AI, we often focus on algorithms and data. But what if the most fundamental bottleneck lies in the design of GPUs? I recently spoke with Sean Lie, co-founder and CTO of Cerebras Systems, about their audacious approach to AI computing. Instead of iterating on existing chip designs, they're reimagining the entire system from the ground up. This end-to-end approach is pushing Cerebras to innovate in unexpected areas. Take server design, a field long thought to be settled. When you're working with a chip the size of a dinner plate—57x larger than NVIDIA’s H100—even servers need an overhaul. It's a reminder that breakthrough innovation in AI will not come from algorithms alone, but from rethinking every layer of the stack, from bare metal to high-level software. Our full conversation here: https://lnkd.in/gWZCi4zE

37

Effortless Integration for Smarter Insights: CloudEagle.ai Meets Bitbucket Integrating CloudEagle.ai with Bitbucket takes just minutes and gives you: ✔ Simplified App Tracking: No more manual reports, get the app usage data you need, when you need it. ✔ Centralized Control: Manage your Bitbucket security from the CloudEagle dashboard. ✔ Proactive Monitoring: Catch unusual login activity before it becomes a problem. Ready for a smarter, more secure development workflow? Book a demo today - https://lnkd.in/gXAhChrt #saas #saasmanagement #toolstuesday Atlassian #jira #toolstuesday

5

Congrats VERSES (OTCQB:VRSSF) for: Introducing "an efficient alternative to deep learning, reinforcement learning and generative AI".... that "achieved 99.8% accuracy on a subset of the MNIST digit recognition task, a common benchmark in machine learning, using only 10,000 training images (90% less data)" Hey Gary Marcus did you see this? #ai #activeinference #spatialweb

1

RAG of 2024 (ColbertV2 Reranking) !!! I've just published an in-depth guide on how to build an efficient local semantic search system using the innovative technologies of ColBERTv2, DSPy, and Qdrant . If you're interested in advancing your search capabilities with the latest AI technology, this article is for you. 🔍 Discover how the deep contextual understanding of ColBERTv2 and the dynamic optimization capabilities of DSPy can transform your data retrieval processes. Plus, learn about integrating Qdrant for scalable and precise search results. 📖 Check out the full article here: https://lnkd.in/gi_mQNH8 #SemanticSearch #AI #MachineLearning #DataScience #ColBERTv2 #DSPy #Qdrant #TechInnovation

20

2 Comments