Ramin Mehran’s Post

Workshop on Large Language Models (LLMs) I’m taking part in a workshop to learn more about Large Language Models (LLMs). This workshop covers how these models work and their applications. This workshop is designed to explore the intricacies of LLMs, including their architecture, applications, and the latest advancements in the field. It provides a comprehensive understanding of how these models work and their impact on various industries.

"This course gives you a synopsis of the 𝐞𝐧𝐜𝐨𝐝𝐞𝐫-𝐝𝐞𝐜𝐨𝐝𝐞𝐫 architecture, which is a powerful and prevalent machine learning architecture for sequence-to-sequence tasks such as machine translation, text summarization, and question answering. "

Gain hands-on experience working with the vision transformer (ViT) by following along François Porcher's clear, step-by-step tutorial on implementing the architecture from scratch.

YOCO - You Only Cache Once - is an interesting and fascinating Transformer architecture alternative to LLM's . As you know Transformers' language modeling approach is extremely expensive and unorthodox. This forces the companies behind Large Language Models (LLMs) to spend billions, with a b, running these models. YOCO architecture takes another, more interesting approach that, fascinatingly, leads to up to three orders of magnitude (100x) reductions in memory requirements and latency while being competitive, performance-wise, with current models, which sounds like a miracle. I think there is something to this new architecture and holds promise. Read more about it at: https://lnkd.in/g7yu64C8 https://lnkd.in/gBWFWtFk https://lnkd.in/g8E93Vxc #YOCO #TransformerArchitecture #LLMs #LanguageModels #AIResearch #MachineLearning #ArtificialIntelligence #DeepLearning #InnovativeTechnology #MemoryOptimization #LatencyReduction #PerformanceEngineering #TechInnovation #FutureOfAI #AIModeling #TechTrends #AIArchitecture #NeuralNetworks #ComputationalEfficiency

Excited to share a new efficient small language model architecture with parallel Mamba and Attention fusion - Hymba. Details: https://lnkd.in/gFKfdG4h We study the tradeoff between Mamba and Attention, exploring how they can be combined, how the attention sink and forced-to-attend phenomena can be mitigated, and how the KV cache can be shared across layers. The team delivered an end-to-end solution featuring a novel architecture, selecting data, a five-stage training setup, and trained both Base and Instruct models. Release is with open license. A standout feature is that the Hymba-1.5B Base model outperforms LLaMA 3.2-3B, despite being trained on 7× fewer tokens and achieving 12× cache reduction. Model weights are coming soon (hopefully tomorrow). Stay tuned for more details on the architecture tomorrow.

Our new Hybrid model is on arxiv!

🚀 Excited to announce our new paper, VideoGPT+: Integrating Image and Video Encoders for Enhanced Video Understanding. ✨ Key Contributions: - Hybrid Model Architecture: VideoGPT+ combines the strengths of image and video encoders, ensuring detailed spatial and robust temporal understanding. - Novel Dataset: Introduced a 112K video-instruction set, leveraging a semi-automatic annotation pipeline to boost model performance. - Comprehensive Benchmark: VCGBench-Diverse evaluates video conversation LMMs across 18 video categories with 4,354 Q&A pairs, testing dense captioning, spatial and temporal comprehension, and complex reasoning. Check out our full paper for more insights and results! Paper: https://lnkd.in/dtrYgTPw Code: https://lnkd.in/dYU-JkRD Team: Muhammad Maaz, Salman Khan and Fahad Khan

This approach is pivotal for developing specialized Small Language Models tailored to each vertical domain.

🚀 Optimizing Next-Gen Sequence Generation Models 🧠💡 The next generation of sequence generation models hinges on optimizing compute-intensive Multi-Headed Attention blocks. This critical innovation drives advancements in efficiency and scalability for Generative AI. Enter Hymba by NVIDIA, leveraging Selective State Space—a groundbreaking approach that’s quickly positioning itself as a leader in compute-efficient Generative AI models. This is a game-changer for unlocking faster, more cost-effective, and high-performing AI solutions. Excited to see how these innovations redefine what's possible in AI! 🌟 #GenerativeAI #AIInnovation #DeepLearning #ComputeEfficiency

Is revolution coming in deep learning architecture world? Is it early signal? Look https://lnkd.in/geWuTJkA . It beats Mumba easily. Although in early stage, but it is a potential candidate to change the transformer world. One not so contextual point: This paper is resulted from more than one year effort from the team. Salute to the authors 👏🙏. #deeplearning #RNN #transformer #llm