Amar Ali-bey’s Post

I was reading through a number posts by Mikhail Gorelkin on research papers about generative AI trends. I couldn't fail noticing most of them are coming from PRC universities. They are doing scarily fantastic work summarizing all the existing Western research. In general, that seems to be the general way the PRC develops, stealing from the truly creative and ramping up the scale far beyond what sclerotic Western industry is capable of nowadays. Big question is if they are capable of innovation themselves or if some spark of Western genius will succeed to out-invent the PRC for the one decisive breakthrough, like the one below. OpenAI Q* reasoning => it gives more solid mathematical grounding to the output content. Something that prevents absurd hallucinations when answering STEM questions. Nicolas M. Chaillan Brian Morrison https://lnkd.in/eGqJhkim

Research day - Topic - 📜 "Generating Long Sequences with Sparse Transformers" by Rewon Child, Scott Gray, Alec Radford, and Ilya Sutskever. Research paper link - https://lnkd.in/eUezTNRP 🚀 Exciting research from the #OpenAI team! 🙌 🔍 The paper tackles the challenge of efficiently generating long sequences using Transformer models. Standard Transformers struggle with very long sequences due to quadratic time and memory complexity. 🧩 💡 The authors propose Sparse Transformers, which introduce sparse factorizations of the attention matrix. This allows for more scalable and computationally efficient handling of long sequences, up to a million tokens or more! 🚀 🌍 Real-world applications include: Long-form content generation Document summarization Story writing Language modeling Teaming up with Mark Hodierne, Maria, Igor Special thanks to Besart Shyti and Izaak Sofer (The Machine Learning Institut) for facilitating this. #NLProc #DeepLearning #LanguageModels #AI #Research #transformers #longtextgeneration #machinelearning #ai #SparseTransformers

If you are new #RAG (Retrieval Augmented Generation) and you don't know where to start from in the current #LLM landscape? I have a fix for you with the following video. This is "RAG 101", you can learn about creating embeddings through OpenAI and store them into Chroma DB, which is an open-source embeddings database and you can query your embeddings through text easily. It supports its own #distance functions like cosine similarity, Euclidean L2 distance etc so you don't have to worry about writing your own methods. Link To Video 👉🏼 https://lnkd.in/g_s4twZc #rag #ai #generativeai #vectordb #vectordatabase #openai #retrieval #tech #artificialintelligence #naturallanguageprocessing

Now doing FREE prototypes https://lnkd.in/dV2B__iy. #automation #data #annotation #detection #MachineLearning #ArtificialIntelligence #computerscience #computervision #uefachampionsleague #tagging #NLP #SupervisedLearning #DataScience #deeplearning

🔍 What is Retrieval-Augmented Generation (RAG) and Why It Matters? Retrieval-Augmented Generation (RAG) is an innovative AI architecture revolutionizing the utilization of large language models (LLMs). It enables AI systems not only to generate responses but also to incorporate specific content—such as PDFs, internal databases, or websites—into those responses. References for Further Exploration: ➡️ Mastering Retrieval-Augmented Generation (RAG) Architecture - A detailed exploration of RAG's mechanics and applications. https://lnkd.in/er_cBbtF ➡️ What is Retrieval-Augmented Generation (RAG)? - IBM Research discusses how RAG enhances LLM outputs with external knowledge. https://lnkd.in/eu8M2taY ➡️ RAG Meets LLMs: Towards Retrieval-Augmented Large Language Models - A comprehensive review of research studies on retrieval-augmented large https://lnkd.in/e3N258mk models. #AI #MachineLearning #RAG #LLM #GenerativeAI #PromptEngineering #AIAugmentation #DigitalTransformation #TechInnovation

📢❗️If you're interested in self-supervised learning, have a look at our #WACV 2025 paper led by two of my brilliant former students Alin Dondera & Anuj Singh. We introduce MAGMA: a simple plug-and-play regularization technique that boosts the performance of Masked Autoencoders (MAE) with no significant computational overhead! It excels in data-efficient settings, boosting accuracy on ImageNet subsets by up to 11% and demonstrating improvements even on the full ImageNet dataset. 📋paper: https://lnkd.in/emiFN6JD ☑️ code: https://lnkd.in/eAP282F6 #ai #computervision #deeplearning TU Delft | AI Shell

Check out this performance comparison of leading AI language models! 📊✨Claude 3.5 Sonnet stands out with impressive scores across various tasks, including: Graduate Level Reasoning: 59.4%Undergraduate Knowledge: 88.7%Code Evaluation: 92.0%Multilingual Math: 91.6%Reasoning Over Text: 87.1%Mixed Evaluations: 93.1%Math Problem-Solving: 71.1%Grade School Math: 96.4%Other models like Claude 3 Opus and GPT-4o also show strong performances but with some variations.Explore the future of AI with these powerful tools! 🚀🤖 #AI #MachineLearning #LanguageModels #Claude3Sonnet #Claude3Opus #GPT4o #AIResearch #TechInnovation #digitaldaddy

🚀 Google just dropped Selective Attention for Transformers! In a recent pre-print published in ArXiv, Google researchers introduced a simple parameter-free change to the standard attention mechanism in Transformers, reducing attention to unneeded elements. And this approach is super effective! A range of transformers trained with the language modelling objective on C4 with selective attention performs equivalently to standard transformers with ∼2X more heads and parameters in their attention modules. In addition, Selective attention decreases the size of the attention’s context buffer, leading to meaningful reductions in the memory and compute requirements during inference. For example, transformers with 100M parameters trained on C4 with context sizes of 512, 1,024, and 2,048 need 16X, 25X, and 47X less memory for their attention module, respectively, when equipped with selective attention, as those without selective attention, with the same validation perplexity. Check out the complete paper here: https://lnkd.in/enyVAdUm Follow along if you want to stay updated with the latest AI research daily. Medium: https://lnkd.in/eaey9sPK LinkedIn Newsletter: https://lnkd.in/et9MMbfX Substack: https://lnkd.in/eqihb9G7 #artificialintelligence #ai #ml #machinelearning #datascience #google

🌟 Demystifying LSTM: Understanding Long Short-Term Memory Networks 🌟 Let's dive into how machines understand sequences in data, like predicting stock prices or text generation? Let’s break down LSTM (Long Short-Term Memory) networks—the backbone of many advanced AI applications. LSTM is like a supercharged memory unit in AI. Imagine your brain processing information—short-term memories (like what you had for breakfast) and long-term memories (like your childhood home). Similarly, LSTM networks learn to remember important details (long-term dependencies) while not getting distracted by irrelevant data (short-term noise). Here’s a simple comparison: 🧠 Regular Memory (Vanilla RNNs) : Like a notebook where you write information down. It’s useful but limited in holding onto details over time. 🚀 LSTM Memory : Think of it as a smart organizer with folders for short-term notes and a vault for long-term archives. It selectively stores and retrieves information, ensuring relevance and context. Applications? LSTM shines in: - Time Series Prediction : Forecasting trends with historical data. - Natural Language Processing : Understanding context in sentences. - Speech Recognition : Handling nuances in spoken language. #MachineLearning #LSTM #DataScience #ArtificialIntelligence

Get ready to witness the magic of neural networks! These powerful algorithms have the ability to approximate any function, no matter how complex, with the right set up and training. In this experimental video, we put this to the test by training a neural network to perform simple mathematical operations such as addition and multiplication. But that's not all, we take it to the next level by creating a neural network architecture that can approximate any function, whether it be linear, non-linear or even piecewise, without even knowing what the actual function is! This is just the tip of the iceberg, imagine the possibilities that data science enthusiasts can create with this concept. Don't miss out on this mind-blowing journey and be sure to like, share and subscribe for more exciting neural network content. Video Link: youtu.be/4PvGKuqRQTE YT channel Link:  youtube.com/@datatrek #datatrek #datascience #machinelearning #statistics #deeplearning #ai