Hyung Won Chung

Introducing RouteLLM: a sophisticated routing framework developed with Berkeley-LMSys in collaboration with Anyscale. RouteLLM optimizes query handling by dynamically selecting between high-performance proprietary LLMs and cost-effective open-source models, cutting costs by over 2x without sacrificing quality. Using human preference data and LLM-as-a-judge for data augmentation, our routers evaluate query complexity to choose the appropriate model. Rigorous testing on benchmarks like MMLU and GSM8K confirms our cost-efficient, high-quality performance. Explore our open-source code, models, and preference data on GitHub: https://lnkd.in/gJyijZRx, and try our online demo: https://lnkd.in/gME-GC2a Explore More about RouteLLM and learn how it can revolutionize your LLM applications: Read our Blog here: https://lnkd.in/gfPD-u-y LLMsys Blog here: https://lnkd.in/ga_MgERE Full research paper here: https://lnkd.in/gqRy7Pjy

81

1 Comment

MIT Researchers Propose Boltz-1: The First Open-Source AI Model Achieving AlphaFold3-Level Accuracy in Biomolecular Structure Prediction A team of MIT researchers has introduced Boltz-1, the first open-source and commercially accessible model that matches AlphaFold3-level accuracy in predicting biomolecular complexes. Unlike its predecessors, Boltz-1 is fully accessible to the public, with the model weights, training, and inference code released under the MIT license. This openness aims to foster global collaboration and advance biomolecular modeling. Boltz-1 follows the general framework used in AlphaFold3 but introduces several architectural and procedural innovations, including new multiple sequence alignment (MSA) pairing algorithms, a unified cropping approach for efficient training, and an enhanced confidence model. These innovations allow Boltz-1 to deliver high accuracy while remaining accessible and significantly lowering the computational burden. The researchers demonstrated Boltz-1’s capabilities through various benchmarks. On CASP15, a competition for protein structure prediction, Boltz-1 showcased strong performance in protein-ligand and protein-protein prediction tasks, achieving an LDDT-PLI of 65%, compared to Chai-1’s 40%. Moreover, Boltz-1 had a DockQ success rate of 83%, surpassing Chai-1’s 76%. These results highlight Boltz-1’s reliability and robustness in predicting biomolecular interactions, especially in protein-ligand complex prediction, where it excelled in aligning small molecules with their respective binding pockets.... Read the full article here: https://lnkd.in/gP-GTp2e Technical report: https://lnkd.in/gCSrNYBY Code/Model: https://lnkd.in/gwdzH2kw MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) Massachusetts Institute of Technology Jeremy Wohlwend Gabriele Corso Saro Passaro Mateo Reveiz Ken Leidal Wojtek Swiderski Tally Portnoi Itamar Chinn Jacob Silterra Regina Barzilay

75

2 Comments

Mixture-of-Agents as an Event-Driven System 🤖☎️ This paper by Wang et al. shows you how to ensemble smaller LLMs to create a system that can outperform state-of-the-art larger models. We’ve implemented this paper in a fully async, event-driven workflow system thanks to Ravi Theja Desetty - treat each “small LLM” as an event-driven step that can process incoming events and respond to it, independently and in parallel. ✅ Take full advantage of processing an entire batch of requests  ✅ Cleaner, readable code LlamaPack: https://lnkd.in/gRuqQGk2 Learn more about workflows: https://lnkd.in/gqWwz7Xs

311

9 Comments

Check out this fascinating research by Peichen Pan, Chang-Yu Hsieh, Tingjun Hou, et al. from Zhejiang University! 'A flexible data-free framework for structure-based de novo drug design with reinforcement learning' This work introduces a unqiue structure-based de novo drug design method that utilises Monte Carlo tree search alongside a carefully curated fragment library. This approach effectively overcomes the limitations of traditional atom-by-atom molecular generation methods. Read here for free via Diamond💎 Open Access: https://lnkd.in/eWtRSN9M

1

Check out the latest research on high-performance graph processing from a hardware/software co-design perspective! This paper dives deep into innovative strategies to optimize computational efficiency. https://lnkd.in/gUXumc5x 廖小飞 Hai Jin Long Zheng Zhiyuan Shao

More fun and mystery in AI land! You may remember that the drama a couple of weeks ago was the piece on Yann LeCun in the WSJ - "This AI Pioneer Thinks AI Is Dumber Than a Cat". "Today’s models are really just predicting the next word in a text, he says. But they’re so good at this that they fool us. And because of their enormous memory capacity, they can seem to be reasoning, when in fact they’re merely regurgitating information they’ve already been trained on." So, lots of support for the idea that all an LLM is doing is super-fancy pattern matching. Anyway, completely contradicting that view is a new paper from Google DeepMind - "Grandmaster-Level Chess Without Search" The basic idea is that it's well understood that the software chess engines that are now the world's best chess players don't operate anything like the humans they can beat. Specifically they rely on a computer's ability to exhaustively examine a vast next possible moves in a chess game and pick the the best one; one of the most powerful chess engines, Stockfish, can "look into the future" up to 20 moves ahead to pick the best possible next move. So the bottom line is that these engines are programmed from the ground up to play chess, and use a very-non-human-like algorithm for doing so, examining many millions of possible moves before deciding the next move. The Deepmind folks asked the question - "what if we let an LLM watch a huge number of games and see if it can learn to play" - without any programming, and without any search capability (LLM's don't do search in any traditional sense). Anyway, it worked! Their LLM trained on 10 million games learned to play chess without any programming or any search - and it plays at the grandmaster level! What does this have to do with the "Dummer that a cat" issue? Playing chess well as a human is REALLY HARD and requires lots of cognitive skills, for example: planning, strategy, tactics, and decision making under uncertainty. And the LLM does this very, very well - the level of a chess grandmaster. So not dummer than a cat at all 🤔🤔🤔 Clearly we have no idea what we're doing here, which I just love 🤣🤣🤣

14

15 Comments

Liquid Foundation Models are "state-of-the-art performance at every scale, while maintaining a smaller memory footprint and more efficient inference." Are you looking at LFMs? https://lnkd.in/dhSZuzSS

4

🚀 New Research Alert: Unlocking the Potential of Open-Access LLMs! 🚀 We are delighted to share an exciting research paper titled "ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG Capabilities" by Peng Xu, Wei Ping, and their team at NVIDIA. 🔍 Why This Paper is a Game-Changer for You: In the rapidly evolving world of AI, staying ahead of the curve with the latest advancements is crucial. This paper introduces ChatQA 2, a Llama3-based model that significantly narrows the performance gap between open-access large language models (LLMs) and leading proprietary models like GPT-4 Turbo. Here's why this research is invaluable: ➡️ Extended Context Window: The model extends the context window from 8K to 128K tokens, enabling it to process and understand large volumes of text more effectively. This is a major leap for handling extensive documents and data-heavy applications. ➡️ Enhanced Instruction Tuning: Through a three-stage instruction tuning process, the model's ability to follow instructions and perform retrieval-augmented generation (RAG) tasks is significantly improved. This means better performance in real-world tasks requiring long-context understanding. ➡️ Superior Performance: ChatQA 2 not only matches but often surpasses GPT-4 Turbo in many long-context understanding tasks and RAG benchmarks. This makes it a powerful tool for applications in natural language processing, conversational AI, and more. ➡️ Practical Benefits: The model's dual capability in long-context and RAG makes it versatile, offering efficiency and accuracy across various tasks, from summarizing entire documents to efficient query-based retrieval. 💡 Key Takeaways for You: ➡️ Innovative Techniques: Learn about the cutting-edge methods used to extend context windows and improve model instruction-following. ➡️ Performance Benchmarks: Understand how open-access models can rival and even outperform proprietary models in specific tasks. ➡️ Application Insights: Gain insights into practical applications and how these advancements can be leveraged in your projects. 🔥 Explore more cutting-edge strategies and network with top industry leaders at the DataHack Summit 2024. Join us in defining the new world order in Generative AI this August in Bengaluru: https://lnkd.in/gAsFp6w7 #AnalyticsVidhya #GenerativeAI #ResearchPaper #DataScience

37

Now on Arxiv: Linsen Li, Pratyush Anand and Kaiming He tackle real-time estimation & control in quantum systems using reinforcement learning and Transformer models. Their theoretical research acieves: 1. Quantum Resource Scheduling with Monte Carlo Simulation: Formulating the scheduling problem in a digitized environment, allowed them to develop a rule-based greedy heuristic that significantly outperforms random scheduling baselines. 2. Reinforcement Learning & Transformer-on-QuPairs Framework: By training reinforcement learning agents in an interactive probabilistic environment, they utilized a self-attention mechanism on sequential information of inhomogeneous qubit pairs. The Transformer-on-QuPairs scheduler achieved over 3x improvement compared to rule-based methods! 3. Scalable & Versatile Framework: This approach is scalable, handling larger sets of input variables and supporting longer sequences as the number of qubits increases. This paves the way for enhanced co-design of physical and control systems in quantum communication, networking, and computing. This RL-based framework for system optimization is versatile, applicable to various large-scale quantum systems, and maximizes the effectiveness of existing hardware platforms. Read the full paper here: https://lnkd.in/gjVuZ3vM Thank you Dirk Englund, MIT Research Laboratory of Electronics, MITRE and National Science Foundation (NSF)! #QuantumComputing #ReinforcementLearning #AI #QuantumScience #Innovation #Research #QuantumTechnology

8

An interesting paper on ArXiv. Sort & Slice: A Simple and Superior Alternative to Hash-Based Folding for Extended-Connectivity Fingerprints https://lnkd.in/eFq73fxZ

1

🦗 Relatively quiet day on the quantum front — but there is quantum excitement to be had in Berkeley Lab’s process for identifying and fabricating quantum materials. Plus, if there were QIS standards for secondary schooling, these would be it. ⚆ Researchers at Berkeley Lab have combined rapid computational predictions with precise fabrication techniques to accelerate the discovery of quantum materials; the results are shared in a publicly available database called the Quantum Defect Genome ⚆ IBM has become a partner in the Superconducting Quantum Materials and Systems Center at Fermilab, focusing on advancing superconducting quantum computing technology through tackling challenges in quantum computing, communication, large-scale deployment, and the workforce ⚆ Quandela and Welinq have teamed up to combine their expertise in quantum processors and interconnect technology to create custom quantum links for photonic quantum computers ⚆ attocube systems Inc is providing essential tools for the quantum technology supply chain to improve energy efficiency for scalable quantum computing applications ⚆ The case for integrating quantum informatics into secondary school curricula is made by positioning it within traditional computer science education, aligning it with the Great Principles of Computing, and establishing standards for expected student outcomes ⚆ Don't miss a single qubit -- follow Cierra Choucair & subscribe to the free newsletter for your daily quantum computing news & research insights. hashtag #quantumcomputing #quantumtechnology #innovation #quantumscience #quantumphysics

1

1. 🚀 TextGrad uses text feedback for automatic differentiation, offering a new method for prompt optimization in large language models, effective in Q&A, molecular optimization, and radiotherapy planning. 2. 🔧 DSPy provides the algorithmic foundation for TextGrad to optimize LLMs' prompts and weights, reducing the need for manual adjustments. 3. 🌐 Automatic differentiation is crucial in deep learning, and TextGrad applies it to the text domain, optimizing through text feedback. 4. ✨ ProTeGi leverages natural language gradients and prompt editing to effectively optimize LLMs' prompts, finding the optimal configuration. 5. ❓ Despite TextGrad's innovation, its practical application may be limited by the performance of the teacher model and the need for extensive API calls. #AI #TextGrad #Innovation #DSPy #LLMOptimization #Algorithm #AutoDiff #DeepLearning #ProTeGi #PromptOptimization #AIOptimization #AIChallenges #TechLimitations

1

🚀 Announcing GPUStack 0.2! We’re excited to introduce powerful new features: multi-GPU inference, distributed inference across workers, CPU inference, and binpack & spread strategies. Whether you need to assign specific workers or manually select GPUs, GPUStack 0.2 provides the flexibility and performance for handling advanced LLM workloads. Learn more about these capabilities and how they can enhance your AI infrastructure by clicking below: https://lnkd.in/gcD-uFQD #AI #LLM #GenAI #OpenAI #LLaMA #TechInnovation

2

Matt's on the road and he’s got a lot to say about scaling LLMs! 🧑‍🏫 We’ve spent quite a lot of time putting open source models into production, and for the most part, it’s fairly straightforward The challenge is when you want the system to scale to high traffic – the inference speed of LLMs becomes a bottleneck, and throwing more hardware at the problem isn’t always the solution Plus, when there’s additional wrappers around the model, such as guardrails, inference time increases significantly In his talk at MLOps World (hosted by Toronto Machine Learning Society (TMLS)), Matt is going to dig into this, detailing the lessons we’ve learned building and running LLMs for our customers at scale, including the surprises that caught us out along the way It’ll be a cracker 🧨 The event is virtual and it’s free to attend, and Matt’s talk is on Thursday 9th May at 17:20 BST (12:20 EST) – the link to sign up and for more information is in the comments! See you there! #mlops #llmops #scalability #opensource

20

1 Comment

Exploring In-Context Reinforcement Learning in LLMs with Sparse Autoencoders Read the full article: https://lnkd.in/ePyEX-YC Paper: https://lnkd.in/eeE_ErWt

2

This new article discusses our efforts in delivering tensor parallelism, weight-only quantization, vLLM integration, and the RBLN model zoo. It is definitely worth reading! 👉 https://lnkd.in/gRnG8_pR #TensorParallelism #WeightOnlyQuantization #vLLM #ModelZoo

84