Dr. Thomas Müller

Retrieval Augmented Generation (RAG) is a straightforward concept that any company can implement with little effort. You can think of RAG as an NN-based document summarization, like merging the results of a simple elastic search/apache solr query with a text summary. The original concept involved retraining the Llm to have a similar latent space, however in applied concepts the retraining has been dropped and replaced with a vector database and prompt engineering that can be paired with any Llm that exists. Based on Meta's LLAMA3, LlamaIndex, Qdrant, Ollama, Angular, and FastAPI, I built the RAG demo about funding lines for universities of applied sciences in hesse. The demo is available at (not well optimized for mobile use): https://lnkd.in/eBYQwKh4 This demo is a weekend project when the little ones were asleep, so it took me less than 3 days. Imagine your company creating a bot like this with more manpower and effort. Best of all, it's completely free and costs you nothing. Your data stays in your company without any security or GDPR violations. P.S.: On my own behalf: I also work as a speaker and consultant on AI topics. Through my research in context-aware fault diagnosis in industrial production lines, I know the state of the art in AI and have implemented various AI use cases myself using different neural networks under industry constraints. If you need some advice, just get in touch with me. I'm looking forward to talking with you about self-built solutions without buying expensive cloud services.

11

2 Kommentare

Want to know more about the implementation of 1-bit quantization for LLMs - this might be helpful Team from the Mobius Labs GmbH has just shared a blog post detailing there work on the 1-bit and 2-bit quantization of Llama2-7B. The models are also available on Hugging Face to try and they have also shared a google colab. Worth reading the article and playing around. Google Colab -> https://lnkd.in/d_hBzsjK Hugging Face -> https://lnkd.in/dv-4xfTC Blog Post -> https://lnkd.in/dUHhGb3R _____________________________________ Follow me for regular updates and insights on Generative AI and my journey as a CTO in AI product development.

8

My appreciation for Rust's stdlib grows by the LeetCode problem I solve. 🦀 🧰 Was able to solve "345. Reverse Vowels of a String" in a pretty readable way (to me at least!) 👇 https://lnkd.in/efP5_8P8 Learning about String::match_indices() and String::replace_range() was nifty to make this fairly readable! I could probably do some method chaining here to make it even more idiomatic? 🤔 (curse LinkedIn not supporting some sort of code formatting like Markdown) ```rust impl Solution { pub fn reverse_vowels(s: String) -> String { let mut reversed_vowel_s = s.clone(); let vowels_indices: Vec<(usize, &str)> = s.match_indices(['a', 'e', 'i', 'o', 'u', 'A', 'E', 'I', 'O', 'U']).collect(); let (indices, mut vowels): (Vec<_>, Vec<_>) = vowels_indices.into_iter().unzip(); vowels.reverse(); for (index, reversed_order_vowel) in indices.into_iter().zip(vowels) { reversed_vowel_s.replace_range(index..=index, reversed_order_vowel); } reversed_vowel_s } } ```

2

Inexpensive token generation and agentic workflows introduce fresh prospects for training Large Language Models (LLMs) with synthetic data, potentially elevating output quality. Direct pretraining of LLMs on their generated responses proves ineffective, but employing agentic workflows could yield superior results, akin to humans learning from their own thinking processes. LLM training typically comprises pretraining on vast amounts of unlabeled text data, followed by fine-tuning and alignment to human values. However, sourcing adequate training data remains a hurdle, prompting exploration of innovative methods like training smaller models on the output of larger ones. Despite the substantial cost of token generation, agentic workflows present promising avenues for creating high-quality synthetic data, ultimately enhancing LLM performance.

11

💡 The 2024 🇨🇭 Open Source Study confirms that 97% of Swiss corporations are using #opensource software …. and 40% are already using #opensource #ai tools and models. Here my Top 5 takeaways from the study ⬇ 1. There are already 700’000+ open source #ai models on Hugging Face 2. #TensorFlow and #PyTorch are the top used #ai open source tools. 3. #Llama and #Mistral are by far the most used open source #ai models. 4. #opensource is used where nobody can really see it! None of the #cloud environments we’re using in private and professional live would work without Open Source! 5. #linux, #docker and #kubernetes are key technologies to run today’s Cloud environments. A lot of interesting articles about the economics of #opensource and real world examples are complementing the study. Thanks to CH Open, swissICT and Berner Fachhochschule BFH for running an providing this interesting study! Here you can find the details (in German): https://oss-studie.ch/ Matthias Stürmer Alain Gut Christian Keller Richard Zobrist Katharina Göppinger Lise Crettaz Philipp Spaeti

33

4 Kommentare

💎 When building #GraphRAG, you may want to explicitly define the graph yourself, or use the LLM automatically extract the graph. Both have tradeoffs: the former takes a lot more human effort but gives you full control over the data representation, and the latter automates human effort but can be unreliable. With LlamaIndex, you can do either or both! 🌟

17

WHY #MOBILE #DATA SO EXPENSIVE? Some countries stand really out in their prices of … … mobile internet data. Single GB of data transfer cost ranges from as little as $0.1 (India🇮🇳 or Italy🇮🇹) to hefty $7.3 (in Switzerland 🇨🇭) How much internet costs in given country has a lot to do with how much #infrastructure has been built in given country (think Canada 🇨🇦 or New Zealand 🇳🇿) , but also … How much of the #competition there is (see Czech Republic 🇨🇿 with solid infrastructure but little competition costing amazing 30x of Italy 🇮🇹). Luckily enough, if you are only turist to given country, your home provider levels the price for you into #roaming package.

6

2 Kommentare

I like Agnieszka Mikołajczyk-Bareła's paper reviews because she really has an eye for relevancy in the field of LLMs: 💡 As someone with a PhD she is familiar with many scholars and academics so when a paper comes out, like xLSTM, she can recognise relevant academics. Like Sepp Hochreiter in this case who was one of the original authors of the original LSTM paper back in the 90's which had a profound impact on the field of ML. 💡She explains papers in a concise and comprehensive way, so that you don't need to be spending hours reading the actual paper, you can spend 5 minutes reading her post in order to understand if it makes sense for you to read the entire thing. My thoughts on xLSTM: 📌 xLSTM definitely appears as a cheaper alternative to the Transformer based models (which is most LLMs at this point in time) 📌 My intuition is that Transformer-based models will outperform xLSTM on most benchmarks, however, I'm looking forward to the first industrial applications of xLSTM, as well as review/benchmark papers comparing it to the most popular LLMs today. 📌 Overall impression: So far, it looks like the advantage of xLSTM technology is price, so I believe xLSTM might have some industrial applications. 📌 However, I believe that for the forseeable future, transformer-based models will continue to be the base for most chatbot-like applications. Let me know your thoughts ✨

13

2 Kommentare

What Karl says. There's going to be two amazing talks about Hopsworks based #ai and #ml systems: * Build a personalized Bitcoin (BTC) virtual assistant in Python with Hopsworks and LLM function calling by Javier de la Rúa Martínez (https://lnkd.in/eQfWpFRU ) * Build TikTok's Personalized Real-Time Recommendation System in Python with Hopsworks by Jim Dowling (https://lnkd.in/eXEHePjG) If you are in the #berlin area next week - go and take a look!

7

⚡ 𝗵𝗼𝘄 𝘄𝗲 𝗰𝗮𝗻 𝗺𝗮𝗸𝗲 𝗹𝗮𝗿𝗴𝗲 𝗹𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗺𝗼𝗱𝗲𝗹𝘀 (𝗟𝗟𝗠𝘀) 𝗺𝗼𝗿𝗲 𝗲𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝘁 𝗮𝗻𝗱 𝗰𝗼𝘀𝘁-𝗲𝗳𝗳𝗲𝗰𝘁𝗶𝘃𝗲? This recent study from Google DeepMind introduces an innovative approach called 𝗥𝗲𝗹𝗮𝘅𝗲𝗱 𝗥𝗲𝗰𝘂𝗿𝘀𝗶𝘃𝗲 𝗧𝗿𝗮𝗻𝘀𝗳𝗼𝗿𝗺𝗲𝗿𝘀, which reduces the size of #LLMs through parameter sharing across layers while maintaining performance. 🟢 LLMs are expensive to deploy, but parameter sharing can reduce their size and cost. 🟢 Recursive Transformers share parameters across layers, with minimal performance loss. 🟢 The study introduces Relaxed Recursive Transformers, which add flexibility to parameter sharing via low-rank adaptation (#LoRA) modules. 🟢 These models outperform similar-sized vanilla models and can recover most of the performance of full-size models. 🟢 A new inference paradigm, Continuous Depth-wise Batching, can lead to significant gains in inference throughput. ✨ I find this research incredibly promising. By making LLMs more efficient, we can deploy them more widely and effectively, driving innovations across various industries. 👉 Paper https://lnkd.in/dvKZxgBR #AI #MachineLearning #Innovation #TechLeaders

7

This might be the biggest moment for Open-Source AI. Meta just released Llama 3.1 and a 405 billion parameter model, the most sophisticated open model ever released. It already outperforms GPT-4o on several benchmarks. Some features: ▪️You can fine-tune, distill and deploy it anywhere. ▪️Supports multiple languages & tools, with performance on par with GPT-4o. ▪️Optimized to run on a single server using cutting-edge quantization techniques. ▪️Advanced data preprocessing & quality assurance methods implemented in Llama 3.1 405B. ▪️Integrating image, video, & voice capabilities—still under development. ▪️Open-source & updated licensing to foster tech sharing and applications. ▪️What is your opinion of open source models vs. proprietary models? Which one will win? #ai #meta #opensourceai

56

3 Kommentare

🔵 Week 3 : AI Paper by Hand ✍ --Paper 3 ➡️ RuAG: Learned-Rule-Augmented Generation For Large Language Models-- Combine RAG and Monte Carlo Tree Search (MCTS) and we get RuAG, a very interesting approach to instill data into an LLM. Here is a pdf of my annotated version of the paper, hope this helps you grab the central essence of this cool idea! 💡 🟢 A quick summary of what I understood: ----------------------------------------------------- RuAG (learned-rule-augmented generation) is a novel framework that distills large volume of information into logic rules using MCTS which are translated into natural language for seamless augmentation of an LLM's knowledge. --------- Central Idea ---------- The way RuAG works can be summarized as: 1. It starts with features (body predicate) and labels (target predicate). Target predicate is defined to be task-relevant while the body predicates are initialized as all the data attributions in the dataset. Based on task and description, LLM generates new target predicates and eliminates most irrelevant body predicates. 2. Next, RuAG uses an automated formulation for MCTS to generate set of first-order logic rules. It involves building a search tree and simulating outcomes to estimate the value of actions. To do so, it starts by build a search tree, selects the most promising child node, calculates the reward for the new node and propagates it up the tree. 3. Finally, the search rules are cleaned and translated into natural language which is then used for improved LLM generation. --------- Results ---------- - For Relation Extraction, RuAG achieves an F1-score of 60.42% - For Anomaly Detection, RuAG achieves an F1-score of 92.59%. In both cases, RuAG outperforms all baselines 💡 Paper : https://lnkd.in/g7BGrrv3 [Repost ♻️ ] Please help me share this resource to your network! #aibyhand #deeplearning #neuralnetworks #ai

84

1 Kommentar

🎶 Finding Harmony in AI: My Weekend Dive into Music LLMs🎶 This weekend, I took a deep dive into the fascinating world of Music Large Language Models (LLMs) by exploring the recent paper "Foundation Models for Music: A Survey." As a musician at heart, I was curious—and a bit skeptical—about how AI could truly understand and create music. But what I found was nothing short of inspiring. The research sheds light on how models like #ChatMusician treat music as a second language, using notations and structures to generate compositions that are not just coherent, but also deeply expressive. These models can craft entire musical pieces based on chords, motifs, and styles that resonate with the kind of creativity that feels almost human. What struck me most was the potential for these technologies to bridge gaps in music education and global cultural exchange. Imagine a world where anyone, regardless of their musical background, can compose a piece that captures the essence of a #Bach fugue or explore the rhythmic complexities of global music traditions—all with the help of AI. But beyond the technical achievements, there’s a deeper connection here. As I read through the paper, I found myself reflecting on how these models might help us connect with the music within ourselves, offering new tools to explore and express our inner melodies. Of course, this journey into AI-driven music isn’t without its challenges—ethical considerations around transparency and copyright are significant. But with careful stewardship, the fusion of AI and music could lead to an era where technology amplifies human creativity in ways we’ve only begun to imagine. I’m excited to see where this path leads and to explore how AI might help us all find our own unique musical voices. For those curious to explore more, you can find the full paper here - https://lnkd.in/g-_7xA7v. #AI #MusicLLMs #Creativity #Innovation #MusicTech #ArtificialIntelligence #EthicsInAI

48

9 Kommentare

𝗠𝘆 𝗘𝘅𝗽𝗲𝗿𝗶𝗺𝗲𝗻𝘁 𝘄𝗶𝘁𝗵 𝗖𝗵𝗮𝘁𝗚𝗣𝗧: 𝗗𝗲𝗯𝘂𝗻𝗸𝗶𝗻𝗴 𝘁𝗵𝗲 𝗛𝘆𝗽𝗲 𝗔𝗿𝗼𝘂𝗻𝗱 𝗧𝗼𝗸𝗲𝗻 𝗟𝗶𝗺𝗶𝘁𝘀 𝗶𝗻 𝗟𝗟𝗠𝘀 Recently, I tried an interesting experiment with #ChatGPT. I wanted to test how it would argue both sides of an increasingly popular debate in AI: the significance of high token limits in large language models (#LLMs). So, I asked it to provide justification from both sides: 𝗢𝗻𝗲 𝗲𝗺𝗽𝗵𝗮𝘀𝗶𝘇𝗶𝗻𝗴 𝘁𝗵𝗲 𝗶𝗺𝗽𝗼𝗿𝘁𝗮𝗻𝗰𝗲 𝗼𝗳 𝗵𝗮𝘃𝗶𝗻𝗴 𝗵𝗶𝗴𝗵𝗲𝗿 𝘁𝗼𝗸𝗲𝗻 𝗹𝗶𝗺𝗶𝘁𝘀 (even up to a million) and how they enhance the capabilities of LLMs. 𝗔𝗻𝗼𝘁𝗵𝗲𝗿 𝗰𝗵𝗮𝗹𝗹𝗲𝗻𝗴𝗶𝗻𝗴 𝘁𝗵𝗶𝘀 𝗽𝗲𝗿𝘀𝗽𝗲𝗰𝘁𝗶𝘃𝗲 and arguing that real-world projects don’t need such extreme token capacities. Case 1: 𝗧𝗵𝗲 𝗖𝗮𝘀𝗲 𝗳𝗼𝗿 𝗛𝗶𝗴𝗵𝗲𝗿 𝗧𝗼𝗸𝗲𝗻 𝗟𝗶𝗺𝗶𝘁𝘀 The first article outlined the potential benefits of having higher token capacities, highlighting scenarios such as: Comprehensive legal document review. Long, context-rich conversations in customer service. Complex, multi-document analysis for strategic insights. These use cases painted a picture of LLMs as powerful tools in intricate, data-heavy situations. But while these scenarios are impressive, they come with notable trade-offs: increased computational costs and infrastructure demands that most organizations aren't prepared for. Case 2: 𝗥𝗲𝗮𝗹-𝗪𝗼𝗿𝗹𝗱 𝗣𝗿𝗮𝗰𝘁𝗶𝗰𝗮𝗹𝗶𝘁𝘆 The second article argued against the need for million-token limits in most real-world projects. It pointed out: 𝗘𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝗰𝘆 𝗮𝗻𝗱 𝗖𝗼𝘀𝘁-𝗘𝗳𝗳𝗲𝗰𝘁𝗶𝘃𝗲𝗻𝗲𝘀𝘀: How smaller, more focused applications of AI can achieve impactful results without breaking the bank. Strategic Token Management: Methods for structuring AI interactions to avoid unnecessary token usage. Applicability: Most business challenges don’t require LLMs to process extreme token limits to deliver value. 𝗪𝗵𝗮𝘁’𝘀 𝗥𝗲𝗮𝗹𝗶𝘀𝘁𝗶𝗰? When I asked ChatGPT to review both perspectives, it concluded that the second argument, for practical, leaner token use was closer to the real challenges and solutions most organizations face. While high token limits can be beneficial in specific, complex scenarios, most practical applications don’t need them. The focus should be on strategic token use, cost management, and building efficient, targeted AI solutions that align with real-world business needs. 𝗠𝘆 𝗧𝗮𝗸𝗲𝗮𝘄𝗮𝘆: This experiment reaffirmed, 1) Prompt chatGPT without preconceived notion on a topic. 2) It’s tempting to chase the allure of advanced AI capabilities like million-token limits but, real impact comes from well-thought-out applications, smart resource use, and models designed for actual business problems, not just theoretical or extreme cases. What’s your experience with LLMs and token limits? Have you found yourself balancing between practicality and ambition in your AI projects? Let's share insights and build towards #AI solutions that are effective and sustainable

9

🌟 The Best Analogy for Understanding #StableDiffusion and #DALLE This is the best analogy I have ever heard for how stable diffusion and DALL-E work: “It's kind of like looking up 👀at a cloudy sky ☁️, finding a cloud ☁️that looks kind of like a dog 🐶, and then being able to snap your fingers 🫰🏻 to keep making it more and more dog-like ☁️🐶.” This perfectly captures the magic of these AI technologies! "Stable Diffusion and DALL·E 3 both rely on a process called diffusion. The image generator starts with a random field of noise, and then edits it in a series of steps to match its interpretation of the prompt. By starting with a different set of random noise each time, they can create different results from the same prompt." Read more here: https://lnkd.in/dQp6iesE #AI #GenAI #AIGC #ChatGPT

8

🚀 Demystifying OpenAI’s New Model: #o1 🤖 The buzz around #OpenAI’s o1 model is impossible to ignore—but let’s take a moment to break it down, beyond the hype. Many people see these advanced AI models as magical, mysterious black boxes. But here’s the thing: AI, especially models like o1, aren’t magic—they’re systems designed with layers of logic and reasoning, much like how our own brains approach complex problems. Imagine you’re in a brainstorming session. 🧠 You start with basic ideas, then gradually add layers of depth, considering different angles, combining thoughts, and eventually reaching a refined conclusion. That’s how models like o1 work too. Just as a deep neural network has layers—each contributing something new and unique—the o1 model processes information in layers of reasoning. It starts with simple, foundational understanding and builds toward more complex insights. It’s like how a human would reason through a problem, one step at a time. Rather than being a mystical force, the o1 model is a tool that mimics this human-like reasoning, breaking problems down into understandable pieces and working through them logically. It’s closer to having a collaborative brainstorming partner than anything magical. Let’s embrace the real power of these models: making reasoning more accessible and scalable—not mysterious. #AI #OpenAI #DeepLearning #NeuralNetworks #AIReasoning #Innovation #Brainstorming #FutureTech #LogicNotMagic #LetsBreakItDownLogically

28