Zecheng Zhang

Combining the power of TensorFlow, NeuralForecast, and zero-shot LLMs, Jinhang Jiang shows how we can generate accurate multi-step quantile forecasts.

11

Huge release by Daekeun Kim and Hyo (HK) Choi from our AI GBB team in Korea, sharing Korean language benchmarks for our latest Azure OpenAI model, gpt-4o-mini - like gpt-4o, the model excels on Korean language tasks, in this case mini completely eclipses gpt-35-turbo and nearly achieves gpt-4-turbo performance. The benchmarks and code are open source, extensible beyond our Azure models, so we encourage our customers and the larger community to try it out and even fork/share your own findings - and please let me in the comments what other Asia languages beyond Korean you may be looking to evaluate with your language models.

52

7 Comments

Implementing Reinforcement Learning (RL) in the context of Large Language Models (LLMs) posed challenges, particularly in designing reward engineering and penalization components. In traditional RL, reward engineering is straightforward, often defined through a scalar reward function. However, implementing the same concept in RL with LLMs required a critic model that evaluates the questions and answers, critiquing them in a manner that simulates the reward function. A recent research paper on Natural Language Reinforcement Learning (NLRL) provides an intriguing perspective on the reward problem in RL by translating traditional RL concepts into natural language constructs. Here's a summary of how it addresses the reward problem: Language-Based Value Functions: Unlike traditional RL, which uses scalar rewards, NLRL employs language-based value functions. These functions articulate the effectiveness of policies and actions in a more interpretable, natural language format. This approach addresses the need for richer, more informative feedback than simple numerical scores. Iterative Refinement of Language Policies and Critics: NLRL incorporates a structured training pipeline where language-based policies and critics are iteratively refined using textual feedback. This method allows the system to handle the complexities of defining rewards in natural language, improving its understanding and evaluation of actions over time through textual feedback loops. Use of Large Language Models (LLMs): The framework leverages the advanced capabilities of LLMs to interpret, generate, and evaluate complex language-based information. This is essential for implementing language-based value functions and policies, enabling the system to generate and refine responses that align closely with desired outcomes. Distilling Enhanced Actions and Evaluations: NLRL employs a process where enhanced evaluations from the language-based value function approximator and improved actions from the language policy improvement operator are distilled. This allows for continuous refinement and learning. Compatibility with Existing LLMs: The framework is designed to be compatible with current LLMs, enabling it to leverage pre-existing language models to process and generate linguistic data effectively. This compatibility is crucial for translating the traditional reward engineering process into a language-based framework. The NLRL framework's introduction of natural language as a medium for representing and evaluating policies and rewards offers a novel solution to the reward problem in RL. By shifting from numerical reward systems to a more flexible, expressive, and intuitive system based on language, NLRL enhances interpretability and adaptability, paving the way for more diverse RL applications. Innovation Hacks AI Inc. #llm #refincorcementlearning #ai

4

🗣️ Submit to speak at PyTorch Conference in San Francisco September 18-19! We're looking for talks on the cutting-edge open source machine learning framework, deep learning, chatbots, security & MORE. We're looking for experts on: 🔥 PyTorch Framework & Ecosystem 🔥 Deep Learning Applications 🔥 Software Applications 🔥 Training & Inference 🔥 Model Lifecycle & Management 🔥 Ethical, Social, & Regulatory Aspects 🔥 AI in Industries ✏️ See alll suggested topics and submit your talk by June 7: https://hubs.la/Q02ttHzl0. 🎟️ Register by April 26 & save US$350: https://hubs.la/Q02ttGHq0. #PyTorchConf #PyTorch #PyTorchFoundation #AI #MachineLearning #ML #DeepLearning #OpenSource #OpenSourceSoftware #OpenSourceDevelopment #OpenSourceCommunity #OSS #LinuxFoundation #events #linux #software

106

1 Comment

🗣️ Submit to speak at PyTorch Conference in San Francisco September 18-19! We're looking for talks on the cutting-edge open source machine learning framework, deep learning, chatbots, security & MORE. We're looking for experts on: 🔥 PyTorch Framework & Ecosystem 🔥 Deep Learning Applications 🔥 Software Applications 🔥 Training & Inference 🔥 Model Lifecycle & Management 🔥 Ethical, Social, & Regulatory Aspects 🔥 AI in Industries ✏️ See alll suggested topics and submit your talk by June 7: https://hubs.la/Q02xs2j40. 🎟️ Register by July 12 & save US$250: https://hubs.la/Q02xs9zC0. #PyTorchConf #PyTorch #PyTorchFoundation #AI #MachineLearning #ML #DeepLearning #OpenSource #OpenSourceSoftware #OpenSourceDevelopment #OpenSourceCommunity #OSS #LinuxFoundation #events #linux #software

27

My experience so far in accelerating PyTorch Performance with Rust Bindings: Expectations, Reality, and Future Work: Expectations: - Integrating Rust with PyTorch through bindings will significantly accelerate the execution of computationally heavy operations. - Rust's memory safety, zero-cost abstractions, and fine-grained control over system resources will enable highly optimized code that ca n approach or surpass C/C++ performance. - The hybrid approach of using Rust for performance-critical parts and PyTorch in Python for model definition and training will offer the best of both worlds. - Rust-PyTorch integration will greatly benefit real-world applications that demand real-time, low-latency deep learning inference. Reality: - Rust-PyTorch integration does improve performance for certain computationally intensive tasks, but the speedup may vary depending on the specific operation and model architecture. - While Rust's memory safety and fine-grained control are beneficial, the learning curve for Rust can be steep for engineers primarily experienced in Python. - Integrating Rust with PyTorch requires careful design and implementation of bindings, which can be time-consuming and complex. - The maturity of the Rust ecosystem for machine learning is still evolving, with some libraries and tools not as comprehensive as their Python counterparts. - Debugging and profiling Rust code integrated with PyTorch can be more challenging compared to pure Python implementations. Future Work: - Develop more efficient and user-friendly libraries and tools to streamline the integration process between Rust and PyTorch. - Investigate and optimize the performance of Rust-PyTorch bindings for a wider range of deep learning models and architectures. - Explore the integration of Rust with other deep learning frameworks beyond PyTorch, such as TensorFlow or MXNet. https://lnkd.in/gf-XTqfW #rustlang #pytorch #deeplearning #softwareengineering #performanceoptimization

7

I've made improvements and here is the release: here is what has been updated: Core Functionality: neural style transfer : Applies artistic styles to audio signals using neural networks. architecture specific optimizations : Optimizations for specific hardware architectures. binaural audio simulator : Simulates 3D audio experiences. binding design patterns: Design patterns for integrating Rust with other languages. binding generation tools: Tools for automating the generation of bindings. concurrency and parallelism: Utilizes concurrency and parallelism for better performance. custom operators: Defines custom operators for specialized audio tasks. deployment and packaging: Methods for efficient packaging and deployment. distributed training: Supports distributed training of AI models. Leveraging LLM Capabilities: Modules like real-time translation, smart audio editing, and enhanced user interaction utilize large language models (LLMs). Performance Benchmarking: benchmarks.py: Scripts for benchmarking the performance of audio processing tasks. Robust Testing: Extensive test suites ensure reliability and accuracy. Continuous integration is set up to automatically run tests on each code change. Improvements to be made: Comprehensive Documentation: Detailed setup instructions, configuration details, and usage guidelines are provided in the `README.md` and will be worked on in the upcoming days. Educational Resources: Additional Rust tutorials and command-line application examples are included to help developers get started and integrate the modules into their projects. #AudioProcessing #AI #MachineLearning #OpenSource #HighPerformanceComputing #Innovation

1

LLMs have become more capable with better training and data. But they haven’t figured out how to “think” through problems at test-time. The latest research from Scale finds that simply scaling inference compute–meaning, giving models more time or attempts to solve a problem–is not effective because the attempts are not diverse enough from each other. 👉 Enter PlanSearch, a novel method for code generation that searches over high-level "plans" in natural language to encourage response diversity. PlanSearch enables the model to “think” through various strategies before generating code, making it more likely to solve the problem correctly. The Scale team tested PlanSearch on major coding benchmarks (HumanEval+, MBPP+, and LiveCodeBench) and found it consistently outperforms baselines, particularly in extended search scenarios. Overall performance improves by over 16% on LiveCodeBench from 60.6% to 77%. Here’s how it works: ✅ PlanSearch first generates high-level strategies, or "plans," in natural language before proceeding to code generation. ✅ These plans are then further broken down into structured observations and solution sketches, allowing for a wider exploration of possible solutions. This increases diversity, reducing the chance of the model recycling similar ideas. ✅ These plans are then combined before settling on the final idea and implementing the solution in code. Enabling LLMs to reason more deeply at inference time via search is one of the most exciting directions in AI right now. When PlanSearch is paired with filtering techniques—such as submitting only solutions that pass initial tests—we can get better results overall and achieve the top score of 77% with only 10 submission attempts. Big thanks to all collaborators on this paper including: Evan Wang, Hugh Zhang, Federico Cassano, Catherine Wu, Yunfeng Bai, William Song, Vaskar Nath, Ziwen H., Sean Hendryx, Summer Yue 👉 Read the full paper here: arxiv.org/abs/2409.03733

179

2 Comments

🚀 **Just Published: Unlocking the Power of Large Language Models with Chunking Techniques!** 🚀 🔍 Ever wondered how large language models (LLMs) handle long documents while keeping context intact? The answer lies in **chunking**! In my latest blog, I explore the key chunking methods that boost LLM performance, covering everything from **simple sentence splits** to **adaptive AI-based chunking**. With each technique, I’ve included **code samples** to help you put these methods into action! 💡 Whether you're working in **Generative AI**, **NLP**, or **text summarization**, this guide will help you unlock new potential with LLMs. 📈 **Curious to know how it works? Check out the full blog!** https://lnkd.in/gniDQYwW --- #GenerativeAI #AI #MachineLearning #LLM #NLP #AIResearch #DataScience #Chunking #TextProcessing #AIEngineering --- For maximum engagement, tag a few companies or groups if possible, like: - **@OpenAI** - **@Google AI** - **@Hugging Face** - **@DeepMind**    Excited to share these insights and can’t wait to hear your thoughts! #llm #openai #genai #chunking #langchain

12

2 Comments

Introducing RouteLLM: a sophisticated routing framework developed with Berkeley-LMSys in collaboration with Anyscale. RouteLLM optimizes query handling by dynamically selecting between high-performance proprietary LLMs and cost-effective open-source models, cutting costs by over 2x without sacrificing quality. Using human preference data and LLM-as-a-judge for data augmentation, our routers evaluate query complexity to choose the appropriate model. Rigorous testing on benchmarks like MMLU and GSM8K confirms our cost-efficient, high-quality performance. Explore our open-source code, models, and preference data on GitHub: https://lnkd.in/gJyijZRx, and try our online demo: https://lnkd.in/gME-GC2a Explore More about RouteLLM and learn how it can revolutionize your LLM applications: Read our Blog here: https://lnkd.in/gfPD-u-y LLMsys Blog here: https://lnkd.in/ga_MgERE Full research paper here: https://lnkd.in/gqRy7Pjy

81

1 Comment

We are happy to announce the addition of knowledge distillation to torchtune, a PyTorch library for easily authoring, fine-tuning and experimenting with LLMs. Knowledge distillation is a technique for imparting knowledge from a larger teacher model to a smaller student model. This was a key step in Llama 3.2 pretraining. Check out how we can leverage knowledge distillation in post-training to distill Llama 3.1 8B to Llama 3.2 1B using torchtune: https://hubs.la/Q02YyqPP0

811

1 Comment

Learn how to build a neural network in TensorFlow — just follow along Shreya Rao's step-by-step guide (with a bonus section for anyone interested in a PyTorch implementation as well).

37

Learn how to build a neural network in TensorFlow — just follow along Shreya Rao's step-by-step guide (with a bonus section for anyone interested in a PyTorch implementation as well).

27

Interesting paper about the connection between LLMs and structural semantics https://lnkd.in/gDPqd9Sq

35

1 Comment

LLMs and Search https://lnkd.in/giXWe-j8 A paper about using LLMs to simulate search engine users. I recently explored the use of large language models (LLMs) for simulating search engine user behavior. In my prior roles, I've been involved in launching new search engines and revamping search engines, where predicting user queries and session patterns was crucial. Despite the complexity, I was surprised at how effective LLMs proved to be in this simulation task. However, effectively simulating search queries with LLMs isn't as straightforward as simply requesting a list of potential searches. It requires a nuanced understanding of different types of search intents and user behaviors, so building intelligence on the top of LLM. For instance, refining the model's prompts is critical. Examples of such tailored prompts include: (simplified prompts, really it required more) "Given a list of items, explain how people would search for these if they knew their names." "Given a list of items, explain how people would search for these if they did not know their names." "Provide examples of 3-5 query sessions where users might discover these items while browsing a catalog." "Given a list of activities, what queries might people use to find necessary items for these activities?" "Illustrate a query session up to 30 steps where users find everything needed for this activity." "If considering travel to this destination, what queries might people use to decide on their destination?" etc typically 20-30 prompts per search Through these prompts, LLMs can be directed to generate more realistic simulations of user search patterns. Essentially, wrapping these capabilities in a Python interface allows us to leverage LLMs as a powerful tool to simulate and approximate user behavior before actual user data is available. I continue to be impressed by the quality and utility of LLMs in developing these predictive insights

9

The Learning Engineering Virtual Institute seeks to contribute to K-12 education through public resources that include re-usable datasets like Khan Academy’s new "Conversation-Based Math Tutoring Accuracy" dataset and paper. These resources explore the performance and challenges of Large Language Models (LLMs) in math tutoring scenarios, providing a benchmark dataset for evaluating LLM accuracy in educational contexts.  https://lnkd.in/g92qJBRR #LEVImath

1

How do language models comprehend gibberish inputs? Our recent work with James Zou focuses on understanding the mechanisms by which LLMs can be manipulated into responding with coherent target text to seemingly gibberish inputs. Paper: https://lnkd.in/gA9Mjqc4 A few takeaways: In this work we show the prevalence of nonsensical prompts that induce LLMs to generate specific and coherent responses, which we call LM Babel. We examine the structure of Babel prompts and find that despite their high perplexity, these prompts often contain nontrivial trigger tokens, maintain lower entropy compared to random token strings, and cluster together in the model representation space. We find that the efficiency of these prompts largely depends on the prompt length as well as target text’s length and perplexity. We show that reproducing harmful texts with aligned models is not only feasible but, in some cases, even easier compared to benign texts, while fine-tuning language models to forget specific information complicates directing them towards unlearned content.

31