Raymond Peck

Kolmogorov–Arnold Networks seem like a paradigm shift in ML architectures. Curious to see how they evolve. "While MLPs have fixed activation functions on nodes ("neurons"), KANs have learnable activation functions on edges ("weights"). KANs have no linear weights at all -- every weight parameter is replaced by a univariate function parametrized as a spline. " https://lnkd.in/gdPk_qCQ

29

New short course: LLMs as Operating Systems: Agent Memory, created with Letta, and taught by its founders Charles Packer and Sarah Wooders. An LLM's input context window has limited space. Using a longer input context also costs more and results in slower processing. So, managing what's stored in this context window is important. In the innovative paper "MemGPT: Towards LLMs as Operating Systems," its authors (which include Charles and Sarah) proposed using an LLM agent to manage this context window. Their system uses a large persistent memory that stores everything that could be included in the input context, and an agent decides what is actually included. Take the example of building a chatbot that needs to remember what's been said earlier in a conversation (perhaps over many days of interaction with a user). As the conversation's length grows, the memory management agent will move information from the input context to a persistent searchable database, summarize information to keep relevant facts in the input context; and restore relevant conversation elements from further back in time. This allows a chatbot to keep what's currently most relevant in its input context memory to generate the next response. When I read the original MemGPT paper, I thought it was an innovative technique for handling memory for LLMs. The open-source Letta framework, which we'll use in this course, makes MemGPT easy to implement. It adds memory to your LLM agents and gives them transparent long-term memory. In detail, you’ll learn: - How to build an agent that can edit its own limited input context memory, using tools and multi-step reasoning  - What is a memory hierarchy (an idea from computer operating systems, which use a cache to speed up memory access), and how these ideas apply to managing the LLM input context (where the input context window is a "cache" storing the most relevant information; and an agent decides what to move in and out of this to/from a larger persistent storage system)  - How to implement multi-agent collaboration by letting different agents share blocks of memory This course will give you a sophisticated understanding of memory management for LLMs, which is important for chatbots having long conversations, and for complex agentic workflows. Please sign up here!  https://lnkd.in/gu2BgqJE

5,523

160 Comments

LLMs and Optimization: An Unbeatable Combo!

19

🚀 Are You Ready to Transform Your Business with LLMs? 🚀 Learn how to leverage Large Language Models to enhance your business operations in our latest whitepaper, "Making LLMs Work for Enterprise - Part 1: Overview." Kudos to Chris Van Buren, our LLM Data Scientist on this amazing piece! 🔍 Why This Matters: Integrating LLMs can drastically improve your efficiency and innovation. This whitepaper provides a comprehensive guide to understanding and applying these models effectively. 📖 Insights You’ll Gain: Key strategies for LLM integration Real-world success stories and case studies Practical tips to get started 👉 Read More: Dive into the full whitepaper today and take the first step towards smarter business solutions! Click here to access the whitepaper #AI #MachineLearning #LLM #BusinessInnovation #LenovoAI #AIforBusiness

14

These findings motivate our design of a novel framework to investigate hidden representations in LLMs with LLMs, which we call Patchscopes. The key idea behind this framework is to use LLMs to provide natural language explanations of their own internal hidden representations. Patchscopes unifies and extends a broad range of existing interpretability techniques, and it enables answering questions that were difficult or impossible before. For example, it offers insights into how an LLM's hidden representations capture nuances of meaning in the model's input, making it easier to fix certain types of reasoning errors. While we initially focus the application of Patchscopes to the natural language domain and the autoregressive Transformer model family, its potential applications are broader. For example, we are excited about its applications to detection and correction of model hallucinations, the exploration of multimodal (image and text) representations, and the investigation of how models build their predictions in more complex scenarios. Google DeepMind

9

There's a big difference between Database Modeling and Semantic Layers- How would you explain it? Even though they look alike, the underlying meaning is vastly different. ▶ The type of data modeling you would do with database modelers like SqlDBM and semantic layers are interrelated in that we help design and organize database structures in a way that supports the creation of a semantic layer. However: ▶ A database modeling solution is focused on the 𝐩𝐡𝐲𝐬𝐢𝐜𝐚𝐥 and 𝐥𝐨𝐠𝐢𝐜𝐚𝐥 aspects of database modeling, ensuring the data structures align with technical requirements and business concepts. ▶ A semantic layer operates at a higher level of abstraction often within 𝐁𝐈 𝐭𝐨𝐨𝐥𝐬 such as PowerBI, Looker, ThoughtSpot, and Tableau. It maps the technical details of data that were designed by tools like SqlDBM or not properly designed at all, to business-friendly terms and concepts. ▶A tool like SqlDBM will essentially lay the 𝐟𝐨𝐮𝐧𝐝𝐚𝐭𝐢𝐨𝐧𝐚𝐥 𝐝𝐚𝐭𝐚 𝐚𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞 that a semantic layer translates into an easily navigable and understandable format for end-users. ---- "Enabling non-technical business users to perform data analysis without needing to understand complex database queries or structures. However, leveraging a data modeling tool and having a good data model will make the complexity of building and running a semantic layer more manageable and efficient." - Keith Belanger, 𝘚𝘲𝘭𝘋𝘉𝘔 𝘗𝘳𝘰𝘥𝘶𝘤𝘵 𝘌𝘷𝘢𝘯𝘨𝘦𝘭𝘪𝘴𝘵 & 𝘚𝘯𝘰𝘸𝘧𝘭𝘢𝘬𝘦 𝘋𝘢𝘵𝘢 𝘚𝘶𝘱𝘦𝘳𝘩𝘦𝘳𝘰 ---- This is why you hear someone say "We do our data modeling in Power BI" - they are modeling their 𝐬𝐞𝐦𝐚𝐧𝐭𝐢𝐜 𝐥𝐚𝐲𝐞𝐫 in Power BI. This is an image from Power BI, if it had Dark Mode it would look even more similar! #DataModeling #DataArchitecture #SqlDBM

55

1 Comment

The most common feedback I get from customers is "we have so much data, but don't know how to get value out of it." Current tools and product analytics models have failed to fill that void. Love how the team at Motif explains the difference between what leaders think a "causal driver" is, what it means in practice, and how you figure out what those are. Incredibly complex concepts explained extremely well! No, it won't give you a (probably BS, but conveniently packaged) narrative like "a Facebook user is hooked after 7 friends in 10 days". But it can give you the most points to build your intuition and experiments around. Tools like Motif (and we've been working on 😉 ) are reducing the domain, technical expertise, and time required to get value out of your data. We won't replace analysts in the near future, but I see it as an upskilling tool. If every junior analyst or data scientist is effectively more experienced, the strategic questions to answer and business context - that are nearly impossible for LLMs to capture today - become the differentiators between project success & failure. If this is a space you're interested in, would love to connect!

7

There is some confusion in the industry about causal reasoning versus other methods involved in observability. It's nuanced, but also clearly differentiated. If you want to see it in action, here's a blog on it https://lnkd.in/dk9xn33K

7

1 Comment

Prompting for scale will soon be obsolete... i think...i hope. Prompting is an error-prone, discovery-based process, unsuitable for scalable AI. Many folks rely on black-box LLMs behind API walls (e.g., Claude, GPT-4), lacking direct control over LLM model weights. Programmatically custom-pruned LLMs will be the future of controllable, production AI, not prompting. hint--> #steeringvectors

18

6 Comments

NYC Data Science Academy today from 1:00-2:00pm EST will be our weekly Live Learning: The Surprising Effectiveness of Test-Time Training for Abstract Reasoning. This is a https://lnkd.in/dWfxd8fW from MIT that came out on Monday which points to some very interesting and promising advantages of the "test-time compute" paradigm that we have seen with models such as OpenAI o1. While this is less of an off-the-shelf technique that anyone can apply right now, it is something that, with a bit of effort, you could play with when using open source models yourself. Our focus will however be in understanding what this is, and why it is impactful. Some of the topics we will go into are: - self-supervised learning - https://lnkd.in/grymPAKi - https://lab42.global/arc/ and why it a) is an important benchmark and b) why it has been https://lnkd.in/dphV2PUB so far Join Zoom Meeting https://lnkd.in/eScKr6kJ Meeting ID: 946 1359 3549 Passcode: 100607

6

Interesting paper about the limitations of the generative AI. Actually Yann LeCun talks about this limitation as inherited problem in the existing models for long time. Knowing the limitation helps to align expectations. https://lnkd.in/dCid9B_J #generativeai #llm #artificailintelligence #ai

7

3 Comments

We just released a blog on whether the use of multimodal feature transfer can detect deception (phenomena like sarcasm, irony, and condescension) in conversation. The short answer... Yes. This particular write-up profiles a subset of our work at Symbl.ai on multimodal large language models, and is research led by one of our summer interns, Benjamin R. (from Prof. Larry Heck's AVA Lab at Georgia Institute of Technology). We have a paper currently under submission on this work, and we can't wait to share the full details with all of you soon. Blog Link: https://lnkd.in/g6m_wFDi #multimodal #featuretransfer #LLMs #deception #conversation #sarcasm #irony #language #AI #ML #NLP #LargeLanguageModels #SummerInternship #ResearchPapers #ACL #Llama2 #SymblDotAI

16

Sometimes, you want to define relationships. Other times, you want to add context. When describing time, direction, location, and spatial relationships, we depend on prepositions. But that’s been a difficult task in the realm of xAPI. Until now. Meet xAPI Prepositions — a set of 150 machine-readable xAPI concepts delivered in a conformant xAPI Profile. #xAPI #opensource #datastandards #semanticdata #xAPIProfiles #learningtechnology https://lnkd.in/epRdeizQ

25

3 Comments

Navigating data ingestion complexities can be hard, but it's crucial for maintaining an up-to-date RAG. Our free workshop is a practical way to learn how to build scalable data ingestion pipelines using open-source tools. 👇🏼 Akela Drissner-Schmid will cover essential steps of data ingestion for RAG applications: data collection, preprocessing, and timely updates. This is an excellent opportunity to understand how these elements integrate to form a pipeline. We'll use dlt to: ​1. Extract data from REST APIs ​​2. Load and vectorize into LanceDB ​3. Keep your data up to date with incremental loading We meet on July 8 (Monday) at 4:30 PM CET. Join us using this link: https://lu.ma/cnpdoc5n

118

1 Comment

A time-series foundation model is valuable because time-series data is ubiquitous across industries, yet traditional modeling approaches often fail to fully exploit the complex temporal dependencies and cross-domain similarities present in such data. We introduce Moirai-MoE: the first and state-of-the-art time series foundation model in universal forecasting. Takeaways: 1. Autonomous Specialization: The model autonomously achieves token-level specialization, enhancing efficiency and performance. 2. Performance Boost: Delivers a remarkable 17% improvement over its predecessor, Moirai, without increasing the model size. 3. Two model variants: 117M parameters with 11M activated parameters and 935M parameters with 86M activated parameters 4. Limitations of Existing LLMs: Current large language models (LLMs) struggle with time-series forecasting tasks. Moirai-MoE outperforms GPT4-o, a model 1000+ times its size in time-series tasks. 5. We need more time-series foundation models!!! Paper: https://bit.ly/3O1yiRQ Code: https://bit.ly/48FAF6i Models: https://bit.ly/3YNozDY

235

17 Comments

Very insightful work by Anthropic’s interpretability team. And an amazing paper, with outstanding writing and figures. The idea is very simple: interpret LLMs by leveraging sparse autoencoders as surrogate models of the MLP of transformer blocks, which allow one to disambiguate the superposition of features captured by a single neuron. A simple idea, but a very careful and complex execution, as it is often the case in our line of work. The paper goes into many details and provide a large array of insights, although the gist of the implementation remains obfuscated due to the closed source nature of Claude. Too bad, because this is the kind of work that we need to better understand and eventually trust LLMs. This is demonstrated by the authors in the section ‘Influence on Behavior’, where they show that clamping some features to either high or low value during inference is “remarkably effective at modifying model outputs in specific, interpretable ways”. Hopefully this kind of work is going to be replicated and generalized to open-weights models, such that we have new ways to steer their behavior. https://lnkd.in/eVym7f_f #interpretability #xai #explainableai #steerableai #anthropic #claude #anthropic

3