Letta

Excited to see Letta in TechCrunch's most disruptive startups of 2024 list 👀 Memory is the most important problem in AI today. LLMs are the new building block for AI systems, but LLMs are inherently stateless compute units - we need fundamental advancements in memory systems to get to anything resembling AGI. 👾 If you're interested in advancing the frontier of memory for AI, reach out! We're hiring across product, infra, and research! (link in comments) Note: our office is in SF, not Berkeley ;)

The AI agents stack in late 2024, organized into three key layers: agent hosting/serving, agent frameworks, and LLM models & storage. Read more at https://lnkd.in/dP7JAzFr

The death of scaling laws 📈 ☠ ⁉ Over the past week there’s been a lot of chatter around “the death of scaling laws”: we have leaks from various sources coming out saying that efforts at frontier labs to scale base models are stalling, meanwhile Sam and Dario are making the rounds on podcasts "claiming AGI by 2025/2026". My take on this as someone that's been working on "agents" research for 7+ years (previously RL + LLM agents at Berkeley / now exclusively LLM agents at Letta): it’s been clear that traditional scaling laws (more parameters + more data = 🧠 📈 ) have been dead for a while now. Look at the trajectory of public model releases from OpenAI - whereas GPT-3.5 to GPT-4 was a clear increase in capabilities (yay scaling laws 🎉 ), all model releases beyond GPT-4 have existing along some tradeoff curve: - GPT-3.5 → GPT-4 (huge bump in reasoning performance*) - GPT-4 → GPT-4-Turbo (worse reasoning, better speed) - GPT-4-Turbo → GPT-4o (back to ~GPT-4 reasoning though slightly worse in many tasks, but better multi-modal support) - GPT-4o → o1 (”better” than GPT-4 in some tasks, though trading off w/ >10s-1m+ response times - in many cases GPT-4 is a better user experience) *note: “performance” here is my own personal take as someone who spends their day-to-day getting LLMs to work as agents (not referring to just public benchmarks), YMMV That being said, I strongly disagree that the death of (traditional) scaling laws implies that AI has “hit a wall” or that AI progress is about to stall. As OpenAI showed with o1, there’s a lot more you can do to improve AI beyond just scaling LLM parameters + data. To date the majority of progress in LLM-based intelligence has come from the base models - as this progress slows, the focus of researchers and engineers will shift to the layer directly above it (the LLM OS / context management layer). Relatively little focus went into the system building over the past ~2 years because of lack of stability - it’s incredibly hard to build systems around LLMs (in particular choosing the right abstractions) when the capabilities of the base models is changing so quickly. Now that LLMs are becoming more of a known quantity (we understand their limitations and their strengths as auto-regressive token predictors) - it’s off to the races to build the full agentic system around the LLM. I would love to be wrong - if GPT-5 exists it would be an amazing early Christmas gift (please drop the weights via magnet link -- thank you in advance Sam 🙏🎄)

Q: How do we get from LLMs-as-chatbots to LLMs-as-agents? A: Programatic memory (context) management through an LLM OS. We're super excited that our (free!) course on LLMs as OS with Andrew Ng and DeepLearning.AI is finally out!! Andrew is one of the “godfathers of modern AI” and has turned DLAI into the de facto platform for learning about the state-of-the-art LLM technology (everything spanning research to new frameworks). Sarah Wooders and I strongly believe the next wave of advancements in AI/agents will come from memory management, but despite our conviction there’s not much material online to help developers and builders understand (1) the importance of memory and (2) how to add memory management to their LLM systems to create better agents. The biggest problem today with LLM agents today is memory. Not just “memory” in a semantic sense (how can we get LLM agents to remember facts+preferences over time, similar to how a human does?), but also memory management - what tokens do we put into the context window at each LLM inference step, and why? The memory management (or “context management”) problem is the fundamental problem in programming LLMs to turn them from autocomplete engines into compound agentic systems that can interact with the world and learn from experience. JSON mode, tool use / function calling, RAG, chain-of-thought: these are all early forays into building the “LLM OS” for context management. We’re still extremely early in the history of LLM OS development. In our DLAI course, we distill the main ideas behind what it even means to do “LLM memory management” into clean and concise example and guide you through building a version of MemGPT (one of the early examples of an LLM OS) yourself, entirely from scratch.

Welcome Kevin!! 👾

Welcome Matt Zhou!

Welcome shubham I̶̛̛̒̒̇̀̈́̐̾́̔̓̐̋͌͗̃͑̿̄͑̾̓̌͒̊̀͐̈͆̿̈́̐̊͛͛̐̕͠͝͝ naik!