Johannes Hagemann

NVIDIA's Jensen said "ChatGPT democratized computing, Llama2 democratized generative AI" ...and now Llama3 is the next level 🔥 It's fascinating to watch Meta's strategic moves. With Llama3, they prioritized training on a massive 15T token dataset to pack all into a lean 70B param model, instead of building a massive model. This allows Llama 3 to match trillion+ parameter models like GPT-4, but at 1/10th the compute, storage and inference costs! 💰 This technique was published by Google DeepMind a few years ago https://lnkd.in/gx7VU8aA Meta is not a dark horse in AI anymore. They might be the top dog. https://lnkd.in/gRRJYuxt #llama2 #llama3 #llm #chatgpt #gpt4 #ai #ml

16

5 Comments

Anthropic AI Releases Claude 3.5: A New AI Model that Surpasses GPT-4o on Multiple Benchmarks While Being 2x Faster than Claude 3 Opus Anthropic AI has launched Claude 3.5 Sonnet, marking the first release in its new Claude 3.5 model family. This latest iteration of Claude brings significant advancements in AI capabilities, setting a new benchmark in the industry for intelligence and performance. Claude 3.5 Sonnet is available for free on Claude.ai and the Claude iOS app. The model is accessible via the Anthropic API, Amazon Bedrock, and Google Cloud’s Vertex AI. Enhanced rate limits are provided for Claude Pro and Team plan subscribers. The pricing structure is set at $3 per million input tokens and $15 per million output tokens, with a 200K token context window, making it cost-effective and highly efficient. Quick read: https://lnkd.in/g6DZAvHD Try it: https://lnkd.in/gfVBhz8S Anthropic Blog: https://lnkd.in/gSuW2WcX Anthropic

44

Among the most impressive aspect of today's Llama 3.1 release is the accompanying research paper! Close to 100 pages of deep knowledge-sharing on LLMs like we haven't seen very often recently What a treat! It covers everything, pretrainining data, filtering, annealing, synthetic data, scaling laws, infrastructures, parallelism, training recipees, post-training adaptation, tool-use, benchmarking, inference strategies, quantization, vision, speech, videos... Mind-blown! Maybe the single paper you can read today to join the field of LLM from zero right to the frontier Read it here and feel the open-science https://lnkd.in/exykbWCu

1,772

46 Comments

Ok this could be HUGE. Matrix multiplication is at the core of LLM training and inference. It also underpins crypto mining and graphics acceleration. This paper outlines a potential approach to matrix-multiplication-free large language modeling which would dramatically reduce the cost / thermal waste of AI processing: https://lnkd.in/gpccwppG #AI #LLMs #matrixmultiplication #airesearch

12

1 Comment

How reliable are LLMs for RAG applications? The results might shock you! I recently conducted a surprise test to evaluate the efficiency of top proprietary models in answering queries based on context from a Vector Data Store. The contenders: (a) Gemini 1.5 Pro (b) Claude 3.5 Sonnet (c) GPT-4 Can you guess which model came out on top? Watch the video to discover the eye-opening results! This experiment raises crucial questions about model reliability. Why do some models falter on prompts that others excel at? If these proprietary models are constantly improving, why do we still see inconsistent performance across simple context-based queries? The implications are significant: How can we build dependable AI applications when their core functionality relies on potentially unpredictable LLMs? This challenge presents exciting opportunities for engineering teams. Building LLM-based applications isn't just about implementation – it's about navigating the complexities of these powerful yet sometimes erratic models. If you want to know what strategies we can deploy for having consistent performance from LLMs for RAG solutions, write to me in the comment and I will share it with you. #AIReliability #LLMChallenges #RAGApplications #AIEngineering #openai #claude #googlegemini

13

6 Comments

We finally have a definition for #OpenSource #AI https://lnkd.in/e7Teq_Mf #GenerativeAI @yannon_ @odonnell_jm @techreview

23

"At the application layer, economies of scale aren’t a viable mechanism for defensibility either — because computational costs are an order of magnitude lower compared to the model layer. Your ability to pay for an OpenAI’s API or compute for your app isn’t a sustainable advantage over a future competitor. Some applications like Character.ai have attempted to avoid this problem by vertically integrating, i.e. building their own customized models. Here again, the efficacy of economies of scale is questionable for the same reasons I mentioned earlier. This leaves network effects and switching costs as the only realistic modes of defensibility for most applications. So how do you create real, meaningful network effects with an AI application? Based on what we’ve talked about so far, the app needs to have an AI-enabled multiplayer interaction which involves a lot more than AI-generated output created by another user. These applications use AI to enable a (higher friction) multiplayer interaction that was previously impossible." https://lnkd.in/gW4zPGGp

3

AI Architecture: Test Time Training (TTT) more efficient than transformers? Worth to do a knowledge deep dive. #artificialintelligence #ai #technology #tech #deeptech #venturecapital #vc #startups #invest #business #funding

4

2 Comments

🧩 One thing we've noticed with #VentureIQ technology is how well it can distill and communicate complex technical material 🧩 Whether it's a #deeptech venture, a novel #AI application, or even a patent submission, we can quickly distill the business and strategic implications for investors...and sometimes even the founders/PIs themselves! 🤯 Based on our work so far, the tech can handle a wide array of research areas and technological sectors -- anything from blockchain algorithms to vaccine development. It handles the nuance of the field, the specific terminology, and extracts the key concepts which matter for venture-building. 🤓 To demonstrate, I took a patent I helped write with Eric Laber on reinforcement learning. I ran it through #Ventureiq to build a commercial assessment (key differentiators, target markets). Then for fun I had #NotebookLM make a podcast out of it. I've uploaded the result it here (DM me if you'd like the actual report): https://lnkd.in/eiTejsny This is more than just automation or streamlining operations - it's about unlocking clarity and insight when it was previously unattainable, and the ability to do it over and over again no matter the sector or vertical. Interested in more info? DM me so we can chat.

11

2 Comments

Join C10 Labs and their AI Venture Studio for a roundtable discussion on building AI-first ventures. This is an opportunity to meet founders from the 1st cohort, network with other applicants from the 2nd cohort, and connect with early-stage entrepreneurs in the AI community. Don't miss out on the chance to learn from the experts and build your own AI venture. Register now! #AI #entrepreneurship #C10Labs #CIC

48

1 Comment

The pace of innovation happening in AI is mindblowing. A great example is Tenyx's latest open-source model, which has become the first proprietary learning technology to outperform any variant of GPT-4. “The implications of Tenyx’s breakthrough are profound, potentially leveling the playing field by providing businesses and researchers with access to state-of-the-art language models without the high costs and restrictions associated with proprietary offerings," writes Michael Nuñez for VentureBeat. Congrats Itamar Arel and team on this achievement! cc: Ron Chrisley, Adam Earle, Romain Cosentino Ph.D., John Robb, Amy Stringer, Vijay Chattha, Nele Heinemann, Point72, Henry Sohn, Morado Ventures, LLC, Caleb Hamilton, Riley Munks, Anne Sophie Hurst

5

1 Comment

Anthropic just released its new multimodal AI model: Claude 3.5 Sonnet. The benchmarks sure need a refresh as AI models keep improving at breakneck speed. But more importantly, each benchmark represents a real-world application, so it's crucial to steer clear of thin wrappers over any benchmarked capability. #claude #anthropic #chatgpt #gpt #openai

5

1 Comment

Hot take: small and specialized models are as underrated as using an appropriate model size/type for a given task. Token recall curve limits (basically, LLM attention span) generally come with quadratic complexity. Right now, there's a trend to make frontier LLMs into "universal problem solvers" with things like Agentive Cognitive Architecture -> https://lnkd.in/gQKBPc5w TLDR: Procedural Memory (LLM/Agent Code) <-> Decision Procedure Decision Procedure <-> Working Memory / Reasoning Loop Working Memory / Reasoning Loop <-> Semantic & Episodic Memory Working Memory / Reasoning Loop <-> Observations In, Actions Out Here's the pickle: running this cost effectively at scale is a pipe dream unless you optimize for speed. Why speed? Because faster almost always means less compute intensive in ML. When you stack up most ML systems against high performance traditional systems, end to end API response times of >50ms are abysmal if you are to have any reasonable chain of calls via Apollo / GraphQL. This should be the goal for performant ML modules. The only way to do this right now is to break them down into small (<5B), highly specialized networks - transformers or otherwise. People need to understand that frontier LLM's performance is equivalent to outdated large monolith repos (not google monorepo style, bad code style). And the solve is the same: break 'em down. If you think about an effective ML architecture, VERY few tasks outside of orchestration and arbitering, if any, require the full power of a frontier LLM. If this isn't true for your design, you need a better design. Would you hire a 1.5M+/yr E10+ from FAANG to build a 3-page static NextJS app? Because that's a VERY accurate analogy for using frontier models for everything. So how DO we efficiently handle quadratic attention scaling for the parts that require a hefty window despite being well designed? Obvi, jury's still out, but here's something new! Peep this: https://lnkd.in/g76_YCGg TLDR: Instead of using predefined patterns/heuristics to approximate attention sparsity like we mostly do now, this uses learned sparsity. This 'lil gem strikes a sane balance between speed and accuracy by utilizing a learnable gate to adaptively select the critical blocks of the attention map while leaving the rest as sparse. The result is 5-6x more speed, therefore 5-6x less cost. Prolly more for larger context window tasks. Maintainable system architecture is important. Especially if you want to effectively integrate ML into your traditional systems. Don't start the project off by littering massive frontier LLMs with abysmal cost efficiency. Those are a last resort. Always start with small and fast models for nearly atomic (in the sense of human logic) tasks you can run in-cluster and tune them as necessary.

22

2 Comments

Pleased to share that Convergence's agent, Proxy, outperformed Claude Computer Use (Anthropic), Runner H (H Company), and Agent E (Emergence AI), achieving state-of-the-art results on the Web Voyager benchmark. Performance Results: Proxy: 82% Agent E: 73% Runner H: 67% Claude Computer Use: 50% Proxy has also demonstrated the ability to dynamically spin-up and launch relevant, specific agents (and full clones of itself) to complete appropriate sub-tasks in parallel. Once tasks are completed, it aggregates the results and delivers a comprehensive response to the user. Evaluation methodology of tasks were run firstly using the Web Voyager auto- evaluator based on GPT-4o. Results were then checked by human evaluators. ✨ Read more below ✨ https://lnkd.in/eg3_4-Dg

16

2 Comments

DeepStack: Enhancing Multimodal Models with Layered Visual Token Integration for Superior High-Resolution Performance Instead of feeding a long sequence of visual tokens into the language model’s first layer, DeepStack distributes these tokens across multiple layers, aligning each group with a corresponding layer. This bottom-to-top approach enhances the model’s ability to process complex visual inputs without increasing computational costs. After testing the LLaVA-1.5 and LLaVA-Next models, DeepStack shows significant performance gains across various benchmarks, particularly in high-resolution tasks, and can handle more tokens efficiently than traditional methods. Recent advancements in LLMs like BERT, T5, and GPT have revolutionized natural language processing (NLP) using transformers and pretraining-then-finetuning strategies. These models excel in various tasks, from text generation to question answering. Simultaneously, LMMs like CLIP and Flamingo effectively integrate vision and language by aligning them in a shared semantic space. However, handling high-resolution images and complex visual inputs remains challenging due to high computational costs. The new “DeepStack” approach addresses this by distributing visual tokens across multiple LLMs or Vision Transformers (ViTs) layers, enhancing performance and reducing overhead. DeepStack enhances LMMs using a dual-stream approach to incorporate fine-grained visual details without increasing context length. It divides image processing into a global view stream for overall information and a high-resolution stream that adds detailed image features across LLM layers. High-resolution tokens are upsampled and dilated, then fed into different LLM layers. This strategy significantly improves the model’s ability to handle complex visual inputs efficiently. Unlike traditional methods that concatenate visual tokens, DeepStack integrates them across layers, maintaining efficiency and enhancing the model’s visual processing capabilities. Quick read: https://lnkd.in/gDMurnGY Paper: https://lnkd.in/gPqUiJBV GitHub: https://lnkd.in/gs_QviZ6 Microsoft Research

34

1 Comment

New trending model, 1 million token context window Llama3. Fine tuned version of llama3 released by Gradient to reach advanced context levels used by google's Gemini. Context is very important, the longer the context the less forgetful a model is. Very important for domains like programming. Longer context also removes most of the need for R.A.G & Fine tuning. Using in context learning. Try it out, use vllm to serve the model on a EC2 instance. Comes after the recent paper explaining how to do this Raw model: https://lnkd.in/g_7Be-iZ Ollama: https://lnkd.in/gKmNMRDJ Papers: https://lnkd.in/gWpE28Uf & https://lnkd.in/gdeEjH_5

2

4 Comments

World Labs, co-founded by AI pioneer "Fei-Fei Li", has raised $230 million to develop large world models (LWMs) that bring AI from 2D to 3D, enabling spatial intelligence in both virtual and real environments. The company's mission is to create models capable of perceiving, generating, and interacting with 3D worlds, allowing artists, designers, and engineers to build virtual spaces with realistic physics and semantics. This groundbreaking technology could have immediate applications in gaming, design, AR/VR, and more, with future potential in robotics. The funding round, led by Andreessen Horowitz, NEA, and Radical Ventures, highlights World Labs’ rapid rise to unicorn status just four months after its formation. With Li's expertise, particularly her pivotal work in computer vision through ImageNet, the company aims to revolutionize industries such as healthcare and manufacturing by advancing the spatial reasoning capabilities of AI. ➜ Check the news at: https://lnkd.in/dd-xBV5Q ➜ Vendorvise's official Instagram: https://lnkd.in/ebK2KQ2C 🤖 If you enjoy reading my posts about AI and tech news, hit the like button and share with your friends to support my content! #ai #artificialintelligence #ainews #aipowered #news #blog #worldlabs #230million #feifeili #spatialai #a16z #lwm #aimodel #generativeai #aitech #technology #content #socialmedia #future #new #business #techtrends #innovation #launch #success #markets #people #industry

3