Arne Mauser

We just released a blog on whether the use of multimodal feature transfer can detect deception (phenomena like sarcasm, irony, and condescension) in conversation. The short answer... Yes. This particular write-up profiles a subset of our work at Symbl.ai on multimodal large language models, and is research led by one of our summer interns, Benjamin R. (from Prof. Larry Heck's AVA Lab at Georgia Institute of Technology). We have a paper currently under submission on this work, and we can't wait to share the full details with all of you soon. Blog Link: https://lnkd.in/g6m_wFDi #multimodal #featuretransfer #LLMs #deception #conversation #sarcasm #irony #language #AI #ML #NLP #LargeLanguageModels #SummerInternship #ResearchPapers #ACL #Llama2 #SymblDotAI

16

In this clip, Chip Huyen breaks down her new blog post on building generative AI systems. We explore: 🏗️ Common GenAI platform components ❌ AI failure types: information vs. behavior-based 🔍 RAG and context enhancement strategies 🛠️ Output improvement techniques 🔄 Fine-tuning complexities 📊 Output format considerations Full chat link in 🧵! #GenerativeAI #MachineLearning

17

1 Comment

🚀 Unlocking the Power of RAG: What You Need to Know! 🚀 🔍 What is RAG? Retrieval-Augmented Generation (RAG) is revolutionizing the way AI generates responses by combining retrieval mechanisms with generative models. This powerful combination results in more accurate and contextually relevant outputs. 📊 Check out the infographic for a quick overview of RAG! 1️⃣ What is RAG? RAG combines retrieval mechanisms with generative models to produce more accurate AI responses. 2️⃣ How it Works RAG retrieves relevant data, augments it, and then generates a contextually relevant response. 3️⃣ Benefits RAG improves accuracy, enhances contextual relevance, and boosts efficiency. 4️⃣ Use Cases RAG powers smarter chatbots, creates better content, and personalizes recommendations. 5️⃣ The Future RAG will see ongoing improvements, broader adoption, and innovative hybrid models. #AI #MachineLearning #RAG #Innovation #TechTrends #FutureOfAI

4

This is a great first step in understanding how scaling dynamics extend beyond language to other types of sequential data. Time is a universal axis, understanding how models learn optimally over data collected over time is a crucial step in the broader AI research agenda.

26

👏CLOC enables high-quality regional embeddings for image region recognition and retrieval tasks, and can be a drop-in replacement of CLIP to enhance MLLMs, especially on referring and grounding tasks

25

#StreamMultiDiffusion: #RealTime #Interactive Generation with Region-Based Semantic Control A fancy way of saying that you can use something like #stablediffusion (and other models), where you draw with up to 8 brushes, each one representing something specific to your goal (body of water, grass, brick, road, whatever). It's kind of like #Nvidia's #Canva, but on steroids, since one can paint as it's being rendered, after rendering, #inpaint, and more. Try it on #HuggingFace: https://lnkd.in/g6K_Xfxw Run this baby locally: https://lnkd.in/g6K_Xfxw

5

The future of open-source AI is truly 𝐛𝐫𝐢𝐠𝐡𝐭.

2

How to optimize Ollama for CPU-only inference? I am wondering if anyone has documentation, blog posts, ideas, ... really anything(!) about optimizing inference for CPU-only. The reason is that most people do not have nice GPU(s) in their notebooks and rely on API providers or remote hardware. Would be great to give those people a good hint on which model they could use locally. We are currently on a good way of doing a massive benchmark about CPU-only inference so i appreciate your input greatly!

📊 The Power of High-Quality, Diverse, Real-World Data Fine-Tuning LLMs 🌍 When it comes to training Large Language Models (LLMs), the mantra isn’t just “more data”—it’s better data. While having more data can be beneficial, it’s the quality, diversity, and real-world relevance of the data that truly drive impactful results. Here’s why: 1️⃣ Higher Quality: clean, well-curated data improves model accuracy and reduces noise, making the model more reliable and stable. Poor-quality data often leads to inconsistent responses and increased hallucinations. Unbiased labeling consistency is key. 2️⃣ Diversity: it ensures the model can generalize across different contexts and user scenarios. This makes the model more robust, safe and inclusive, which is especially important in NLP for avoiding bias and ensuring representation. 3️⃣ Real-World Datasets: training on real datasets helps models perform better in real-world applications, as it learns from actual patterns and usage scenarios, making the model more practical and effective. For LLMs, there’s a general playbook to prepare data effectively: (1) Collect Instruction-Response Pairs: gather examples of how tasks should be answered, covering various domains and complexity levels. (2) Concatenate Pairs: organize the data into structured sequences, allowing the model to learn from end-to-end interactions. (3) Tokenize: process the data into tokens, using padding and/ or truncation as needed to maintain uniform length. I particularly love 🤗 transformers library's AutoTokenizer to identify which tokenizer to use on every model. (4) Split into Train/Dev/Test Sets: partition your data into training, validation, and testing sets to ensure proper model evaluation and tuning. Ultimately, data quality prevails over quantity, specially as we live in the LLMs era when models have been pre-trained with almost all the public data available in the 🌎 . A well-prepared dataset tailored to your application will make all the difference in the performance and reliability of your model. #AI #MachineLearning #DataQuality #LLM #NLP #ModelTraining #DataPreparation

5

It was probably 2006 when I saw Geoff Hinton give his now famous talk at NeurIPS about Deep Belief Networks and Restricted Bolzmann Machines. This talk really was the catalyst event that rekindled interest in "deep learning". It was a controversial idea at the time that most considered "fringe". As a 2nd year PhD student, many faculty warned me not to pursue it. They called it an "eddy current" that would dry out and go nowhere. They cited Cybenko's Universal Approximation Theorem, which they summarized as showing that a two layer network was all you needed to approximate any function, and "depth wasn't needed". This "proof" probably set the field back 10 years after it was introduced in 1989, and killed a lot of interest in deep learning. But Geoff's talk in 2006 blew me away, as I had never seen in the few years of my PhD any model that could generate anything as 'complex' as handwritten digits (we used to think MNIST was complex). So I walked up to Geoff Hinton at NeurIPS, and told him I wanted to work on Deep Learning, and wanted to join the faithful. Luckily he agreed, and my advisor Tom Mitchell, although skeptical, was gracious enough to let me go to Toronto for four months to work in Hinton's lab. It was a magical summer and it's hard to overstate the impact of the people who were there: Laurens van der Matten, who was inventing t-SNE (and nearly 20 years later my team shows me t-SNE plots almost everyday); Graham Taylor who applied generative models to animation decades before it was cool; Alex Krizhevsky, who helped change the field of Computer Vision overnight with AlexNet; Ruslan Salakhutdinov who is a patient soul who taught me all the DBN math and still inspires me today, and of course Ilya Sutskever, who co-founded OpenAI and is one of the greatest minds I've ever known. Hats off to all of you - and congratulations to Geoff for your Nobel Prize today, for persevering through years of skepticism, following your wonderful intuition, and for building one of the most productive laboratories in the history of science.

205

6 Comments

If you think compute scaling will plateau, think again. We are in the baby states of inference time compute. VLLMs coupled with computer tools, reward learning and RL can leverage orders of magnitude more inference time compute than what we have seen with foundation models. Here’s a concrete training architecture (replace \pi with a large pre-trained vLLM and add it to the reward learner too, \R is the space of all computer tools)

22

1 Comment

This post from Rowan Cheung has a great summary of agentic approaches by the various LLMs. @rowancheung is a great follow on X. https://lnkd.in/gW_kZ9xx

1

Hallucinations in LLMs are inevitable, no seriously! Josh Fritzsche and I have worked hard at Voze to find a way to tune, prompt and even plead 🙏 with LLMs to prevent them from the occasional 🍄 hallucination. It seemed like the better we have done the harder the edge cases have been to solve and eventually we realized that they tend to frequently happen when LLMs predict or make anticipations on "nothing". I remember at one point nearly going insane with frustration because when LLMs had a hallucination, they were EPIC! 😱 Then we pushed through it and benefited for the effort! How did we solve this common problem? We didn't (Not completely), we got better at identifying the issues as they happened. Once we identified in the data we were able to start looking for patterns. The same catalysts that cause it to happen once if left unchanged caused it over and over again. Our solution is astoundingly simple. Once you know that it's likely to happen..... let it! Accept it will 🍄, but also be preemptive in not focusing on it and use alternative data sources, generators or providers. Change the conditions that cause it but continue to monitor it as it happens. As Dan Caffee is fond of saying to me on a regular bases. "Dusty we need the system to be adaptive and provide mechanisms to learn from it's past." Well he is right! All we needed was data and the people that contribute to it's quality. Huge props 👏 to strong support from Kathy and Janelles audit teams. We were armed with a knowledge and a desire to learn from past mistakes. Armed with confidence that we could correct these problems we could focus on the data to prevent their impact going forward. When doing millions of LLM (Generative AI) calls regularly Josh and I have learned that hallucinations will happen. We also know exactly how we will handle and prevent it going forward! Failure is not doing anything and accepting defeat.

4

2 Comments

Ensemble of smaller LLMs (PoLL) is less biased, faster, 7x cheaper when evaluating LLMs performance. Works great on QA & Arena-hard evals arxiv.org/abs/2404.18796

9

#ObsessivelyEnthusiasticInvestor #UnreasonablyEnthusiasticInvestor #IrrationallyEnthusiasticInvestor at REFASHIOND Ventures: The Industrial Transformation Fund. We look for the people we believe are doing the best work in Industrial Transformation & Supply Chain Technology; Attempting the impossible. Then we go all-in on them. With obsessive and unreasonable enthusiasm. Join us. #IndustrialTransformation #SupplyChain #Innovation #Technology #Startups #VentureCapital #DemandTheImpossible #UnreasonableEnthusiasm #ObsessiveEnthusiasm #IrrationalEnthusiasm 

6