Yuyang (Deo) Chen

Awesome to see the Meta #Llama 3.2 models on #Azure #AI Model Catalog through managed compute inferencing #MaaS and soon on serverless API deployments while MACC Eligible providing customers with Enhanced Security and Compliance, Content Safety Integration, Simplified Assessment of LLM flows, Client integration, and Simplified Deployment and Inference. https://lnkd.in/g4Zpb3Dm

9

1 Comment

⚡ New blog post about our work on hybrid retrieval for #Instacart search While it's widely recognized that adopting a hybrid retrieval system (keyword + vector) can enhance performance, creating a bespoke solution tailored to our specific business needs proved to be quite a challenge! In our latest blog post, we explore: 1. The current search retrieval architecture at Instacart. 2. The challenges we faced, including the variance in search query distribution and the number of relevant documents per query. 3. Our methods for contextually adapting the recall set from each retrieval source to boost the precision and recall of the final results. Super proud of the team who made this possible Vinesh Gudla Prakash Reddy Putta Ankit mittal Andrew Tanner Guanhua S. Xiao Xiao Taesik Na Alex Charlton Xukai Tang Akshay Nair Haixun Wang

52

Organizations have insufficient technical capabilities for the speed and scale of their ML model implementations. My colleague Sumit Agarwal just published a very helpful research with a capability view and sample vendors for defining a modular, flexible and scalable MLOps solution. https://lnkd.in/gnB_T9bq #MLOps #LLMOps #AI

149

17 Comments

🌟 Unlock the Power of Generative AI with Andrew Ng! 🌟 DeepLearning.AI has launched "Generative AI for Everyone"—a groundbreaking course designed to make generative AI accessible to all. 🚀 🎓 What You'll Learn: Core Concepts: Understand the fundamentals of generative AI and how it works. Real-World Applications: Discover how tools like ChatGPT, DALL·E, and more are transforming industries. Business Strategies: Learn how to integrate generative AI into your organization to boost productivity and creativity. Practical Insights: Hands-on exercises to help you grasp the technology, even without a technical background! 👩‍💻 No coding experience? No problem! This course is tailored for everyone—professionals, business leaders, and anyone curious about AI's potential. ✨ Ready to shape the future with generative AI? Enroll now and start your journey today! ➡️ deeplearning.ai #GenerativeAI #ArtificialIntelligence #Upskill #AIInnovation

5

Are you an iOS / Apple Apps Builder? Check out our new five part blog series that gets you from no AI Infra experience to building AI features right in your Apple Ecosystem Apps with Weaviate! Huge thanks to Randy Fong for the community contribution and collaboration. Stoked to do more with you, Randy!

11

Good detailed overview of Retrieval-Augmented Generation (RAG), one of leading techniques to improve accuracy of LLMs. When developing a production-grade LLM-based system, it's essential to understand the advantages and limitations of different techniques (RAG, prompt engineering, fine-tuning etc), across training and serving/inference phases, to improve LLMs for specific applications (see my previous post https://lnkd.in/giuJdJJt) A balanced approach to LLM applications that takes into account accuracy, latency, and cost is crucial for successful productization at scale. #artificialintelligence #largelanguagemodels #GenerativeAI #airevolution

10

Learn how to build an autonomous research assistant using LangGraph with a team of specialized AI agents #llms #chatgpt4 #gemini #aiml #llmops https://lnkd.in/d6TUbZNj

2

Attending #DataAISummit next week? Swing by the O'Reilly booth (#127) at the Data+AI Summit on June 12 from 3:15PM - 3:45PM PT for a book signing with me! We're giving away my earlier book "Modern Data Engineering with Apache Spark" but we can talk Delta, Spark, or anything really or all things The Delta Lake Definitive Guide. #opensource #oss #deltalake #freebooks

86

6 Comments

Spatial Reasoning did not ‘emerge’ spontaneously in Large Language Models (LLMs) the way so many reasoning capabilities did. Humans have specialized, highly capable spatial reasoning capabilities that LLMs have not replicated. But every subsequent release of the major models — GPT, Claude, Gemini- promises better multimedia support, and all will accept and try to use uploaded graphics along with texts. #llms #chatgpt4 #gemini #llmops #aiml #ml https://lnkd.in/dFSWKqKs

2

This is a win for #sustainability. In effect, ML applications may better leverage GPU capabilities, saving more time and energy 🚀

1

The idea of agents has been around for a while. What’s new here? In his talk, Andrew Ng explains agentic systems that use design patterns such as reflection, tool use, planning, and multi-agent collaboration to outperform the results of zero-shot prompting. #llms #aiagent #ml #rlhf #mlops #datascience #llmops https://lnkd.in/d2Q8Yj7X

6

In this article we are providing the major architectural building blocks and blueprint for building end-to-end GenAI-based applications that are ready for production. I’d like to call out several key considerations as you go about implementing and designing these llm-based applications. #llms #llmops #chatgpt4 #aiml #gemini #mlops #data https://lnkd.in/dMev5ncq

8

𝐀𝐠𝐞𝐧𝐭𝐢𝐜 𝐖𝐨𝐫𝐤𝐟𝐥𝐨𝐰: 𝐏𝐢𝐨𝐧𝐞𝐞𝐫𝐢𝐧𝐠 𝐭𝐡𝐞 𝐅𝐮𝐭𝐮𝐫𝐞 Recently I happened to see this nice video ( 𝘩𝘵𝘵𝘱𝘴://𝘸𝘸𝘸.𝘺𝘰𝘶𝘵𝘶𝘣𝘦.𝘤𝘰𝘮/𝘸𝘢𝘵𝘤𝘩?𝘷=𝘴𝘢𝘭78𝘈𝘊𝘵𝘎𝘛𝘤 ) by Andrew Ng in which he talks about Agentic workflows. In Language Model (LLM) development, we provide a zero-shot prompt and receive an answer. We may review it and add more prompts to get it to a desired shape. This is exactly what the agentic workflow does. Basically, 𝘢 𝒃𝒖𝒏𝒄𝒉 𝒐𝒇 𝑨𝑰 𝒂𝒈𝒆𝒏𝒕𝒔 𝒘𝒐𝒓𝒌 𝒕𝒐𝒈𝒆𝒕𝒉𝒆𝒓 𝒊𝒏 𝒂𝒏 𝒊𝒕𝒆𝒓𝒂𝒕𝒊𝒗𝒆 𝒘𝒂𝒚 𝒕𝒐 𝒑𝒓𝒐𝒅𝒖𝒄𝒆 𝒂 𝒅𝒆𝒔𝒊𝒓𝒆𝒅 𝒐𝒖𝒕𝒑𝒖𝒕. Moreover, he even claims the output produced by agentic workflow is much better than the output produced single LLM which might sound obvious but what surprised me was, as per the “𝑯𝒖𝒎𝒂𝒏𝑬𝒗𝒂𝒍” coding benchmark, agents-based approach beating plain LLM by quite a margin (see the picture below). The plain 𝐆𝐏𝐓 3.5 was at 48% but 𝑳𝑫𝑩+𝑹𝒆𝒇𝒍𝒆𝒄𝒕𝒊𝒐𝒏 (𝘈 𝘓𝘢𝘳𝘨𝘦 𝘓𝘢𝘯𝘨𝘶𝘢𝘨𝘦 𝘔𝘰𝘥𝘦𝘭 𝘋𝘦𝘣𝘶𝘨𝘨𝘦𝘳 𝘷𝘪𝘢 𝘝𝘦𝘳𝘪𝘧𝘺𝘪𝘯𝘨 𝘙𝘶𝘯𝘵𝘪𝘮𝘦 𝘌𝘹𝘦𝘤𝘶𝘵𝘪𝘰𝘯 𝘚𝘵𝘦𝘱-𝘣𝘺-𝘴𝘵𝘦𝘱) fared at impressive 95.1% with the same GPT 3.5. The plain 𝐆𝐏𝐓 4 was at 67% but 𝑨𝒈𝒆𝒏𝒕𝑪𝒐𝒅𝒆𝒓 (𝘔𝘶𝘭𝘵𝘪-𝘈𝘨𝘦𝘯𝘵-𝘣𝘢𝘴𝘦𝘥 𝘊𝘰𝘥𝘦 𝘎𝘦𝘯𝘦𝘳𝘢𝘵𝘪𝘰𝘯 𝘸𝘪𝘵𝘩 𝘐𝘵𝘦𝘳𝘢𝘵𝘪𝘷𝘦 𝘛𝘦𝘴𝘵𝘪𝘯𝘨 𝘢𝘯𝘥 𝘖𝘱𝘵𝘪𝘮𝘪𝘴𝘢𝘵𝘪𝘰𝘯) fared at whooping 96.3% with the same GPT 4.  You can get more information about the evaluation and the models at 𝘩𝘵𝘵𝘱𝘴://𝘱𝘢𝘱𝘦𝘳𝘴𝘸𝘪𝘵𝘩𝘤𝘰𝘥𝘦.𝘤𝘰𝘮/𝘴𝘰𝘵𝘢/𝘤𝘰𝘥𝘦-𝘨𝘦𝘯𝘦𝘳𝘢𝘵𝘪𝘰𝘯-𝘰𝘯-𝘩𝘶𝘮𝘢𝘯𝘦𝘷𝘢𝘭. He also mentioned the following are the 4 design patterns that are emerging in the agentic workflow. 1.    Reflection 2.    Tool use 3.    Planning 4.    Multi-agent collaboration If you are interested in learning more, instead of me explaining each of these patterns, I would encourage watching the video :) #ai #agent #agenticworkflow #gpt

23

Building a data mesh? Don't overlook the X and Y axes of data sharing and independence. The X-axis defines how widely you'll distribute data; the Y-axis determines the level of autonomy for teams. A high X and high Y scenario strongly points towards a data mesh architecture, granting domain-driven data ownership while enabling seamless data access across the organization. Image source #datamesh #dataengineering #leadership #dataarchitecture

9

I'm excited to share two papers from the Retrieval team at Walmart that were presented at CIKM last week. They're both about improving the relevance of Embedding-Based Retrieval. 𝗥𝗲𝗹𝗲𝘃𝗮𝗻𝗰𝗲 𝗙𝗶𝗹𝘁𝗲𝗿𝗶𝗻𝗴 𝗳𝗼𝗿 𝗘𝗺𝗯𝗲𝗱𝗱𝗶𝗻𝗴-𝗯𝗮𝘀𝗲𝗱 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹. https://lnkd.in/gNQe7QhA The first is about how to intelligently choose a query-dependent cutoff on the cosine score to remove irrelevant items from the retrieved set. Since cosine scores aren't comparable across queries, we introduce a "Cosine Adapter" to map the scores to interpretable scores. 𝗘𝗻𝗵𝗮𝗻𝗰𝗶𝗻𝗴 𝗥𝗲𝗹𝗲𝘃𝗮𝗻𝗰𝗲 𝗼𝗳 𝗘𝗺𝗯𝗲𝗱𝗱𝗶𝗻𝗴-𝗯𝗮𝘀𝗲𝗱 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 𝗮𝘁 𝗪𝗮𝗹𝗺𝗮𝗿𝘁. https://lnkd.in/gdqkc2hR The second is about model training techniques we learned to improve the relevance of embedding models. In particular, we describe how to use a teacher model to provide relevance labels for training Juexin (Joy) Lin Nicholas Rossi, PhD Feng LIU Zhen Yang Sachin Yadav Praveen Reddy Suram Satya Chembolu Prijith Chandran Hrushikesh Mohapatra Alessandro Magnani Ciya Liao

147

1 Comment

When *open* isn't *open source* 🙋🏼‍♂️👀 Reflecting on the open-source landscape for Large Language Models, I explore the essentials: permissive licensing, transparency in training data and model architecture & more #LLM #LLMs #AI #GenerativeAI #opensource https://lnkd.in/gMxbN4Nb

35

2 Comments

Training of LLMs to generate multi-facet structured substances (JSON) using Targeted Denoising. (EMNLP-2024). https://lnkd.in/g66F_S9x

39

1 Comment

More details about the mighty phi-3 family of SLMs. Such a key development as GenAI Dev becomes more about always optimizing and orchestrating with LLMs routing to LLMs and SLMs continuously to balance complexity, latency, cost and more! https://lnkd.in/gGQYqNKt

9