Muyang Jin

Combining the power of TensorFlow, NeuralForecast, and zero-shot LLMs, Jinhang Jiang shows how we can generate accurate multi-step quantile forecasts.

11

More opportunities

12

Hi LinkedIn Community,   Short version:  Grad student building low-code tool for ML data pipelines. Goal: Make ML dev work enjoyable again, not just endless debugging. Ask: Shadow sessions (see Calendly link below) Longer version: As ML devs, we often leave the office thinking, “Do I actually love doing my work?” because we had spent most of the day on inefficient whack-a-mole debugging and maintenance work. I want to get us back to where we can love coding for those reasons we originally chose to pick up coding in the first place.   A lot of AI tools are restrictive and end up mangling codebases, and I believe the current state of AI coupled with strong design principles can enable much better code native developer experiences. I’ve gone through prototypes/testing and have some competing hypotheses. I’d love to observe coders working on ML data pipelines (happy to sign NDAs if necessary, can also be personal projects instead). If you're open to sharing your expertise, please schedule a shadow session: https://lnkd.in/eBW6p6qE Let's make ML data development more efficient and actually enjoyable! #MachineLearning #DataPipelines #LowCode #DevTools #AI #SoftwareEngineering #GradStudent #Research #MLOps #DataScience #ArtificialIntelligence #ProductivityTools #DeveloperExperience #CodeQuality #TechInnovation #ComputerScience #DataEngineering #CloudComputing

84

8 Comments

🚀 𝗥𝗲𝗰𝗲𝗻𝘁 𝗛𝗶𝗴𝗵𝗹𝗶𝗴𝗵𝘁𝘀 𝗶𝗻 𝗔𝗜 𝗥𝗲𝘀𝗲𝗮𝗿𝗰𝗵 🚀 𝗚𝗿𝗼𝗸𝗙𝗼𝗿𝗺𝗲𝗿: Graph Fourier Kolmogorov-Arnold Transformers Introducing a novel Graph Transformer network that surpasses conventional self-attention mechanisms. 1. Tackles limitations of existing GTs in modeling complex node label patterns. 2. Incorporates learnable activation functions in the graph spectrum. 3. Demonstrates superior performance on multiple node and graph classification datasets. Paper: https://lnkd.in/gDKKk-2x 𝗦𝗧𝗔𝗥 𝗔𝗧𝗧𝗘𝗡𝗧𝗜𝗢𝗡: Efficient LLM Inference Over Long Sequences Recent advancements in Large Language Models (LLMs) now handle contexts extending to millions of tokens. 1. New attention mechanism designed to address computational challenges during inference while preserving accuracy. 2. Opens up new applications, including repository-level code analysis, multi-document summarization, and large corpus retrieval. Paper: https://lnkd.in/gtj29ghz 𝗚𝗹𝗶𝘁𝗰𝗵 𝗧𝗼𝗸𝗲𝗻𝘀 𝗶𝗻 𝗟𝗮𝗿𝗴𝗲 𝗟𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗠𝗼𝗱𝗲𝗹𝘀: 1. Categorization Taxonomy and Effective Detection. 2. Addressing the issue of glitch tokens that lead to hallucinations and erratic behavior in LLMs. 3.Introduction of GlitchHunter, a novel clustering-based detection method. 4.Constructs a Token Embedding Graph (TEG) and uses clustering for efficient detection. 5.High precision and recall across multiple LLMs. Paper: https://lnkd.in/gTVr8jgK #MachineLearning #AIResearch #GraphTransformers #LLM #DeepLearning

4

🚀 I'm excited to share that our paper, "Enhancing E-commerce Product Title Translation with Retrieval-Augmented Generation and Large Language Models," has been accepted at CIKM 24' - Data-centric AI. Thanks for the co-authors Stephan Walter and Taichi Nakatani for the collaboration! 😀 Our study explores how we can improve the machine translation of e-commerce product titles using LLMs, particularly languages that the LLMs do not primarily support. The results of our experiments show that our proposed approach leverages large volumes of bilingual product data, and utilizes a probabilistic retrieval model to retrieve similar bilingual product information for few-shot learning, can improve overall product title translation quality by up to +15.3% in chrF scores. Notably, we observe that language pairs where the LLM has limited proficiency can benefit the most from our method, addressing a common challenge in this domain. As new and more capable LLMs continue to emerge, our RAG-based approach allows for the quick integration of these models (without extensive fine-tuning). By using the large volume of bilingual product data, including titles, bullet points, and descriptions, as few-shot learning examples, our method provides a flexible and adaptable solution that is particularly suitable for industrial applications, where the ability to keep pace with changing requirements is crucial. #amazonscience #LLM #CIKM #RAG #machinetranslation #searchengine #ecommerce #translation #GenerativeAI #search #MultilingualSearch #mixtral #localization #retail

66

2 Comments

I wanted to share some insights from a recent paper from Amazon Science in which authors tackled an important challenge in language modeling: minimizing sequence truncation. In language models, truncating input sequences might lead to a loss of crucial information, especially when the models are dealing with long-context tasks such as document summarization or story generation. The authors introduce a clever solution by leveraging a heuristic from the Best Fit Bin Packing problem, an NP-hard problem, to reorder input documents and reduce truncation. The proposed approach helps retain more information compared to other strategies by optimizing how sequences are packed into the model's context window. Authors show that the bin packing solution had biggest performance gain reading comprehension (+4.7%), natural-language inference (+9.3%), context following (+16.8%), and program synthesis. They also found that best-fit packing helped prevent closed-domain hallucination. Authors claim that while the experiments primarily focused on LLM pretraining, best-fit packing is broadly applicable to fine tuning as well. https://lnkd.in/gWquw3ud

28

3 Comments

Run open-source large language models like Llama 3 on your local machine using LangFlow and Ollama. A step-by-step guide, from downloading Ollama to setting up Langflow for local inference using Llama 3.

56

1 Comment

Three insights from the TEDAI San Francisco X Factory X AI at Meta CodeGen Hackathon. Had the pleasure of judging a fantastic hackathon in SF yesterday. Here's the biggest things I took away from iteration on the cutting edge of GenAI for code: 1) 𝙂𝙚𝙣𝘼𝙄 𝙘𝙤𝙙𝙞𝙣𝙜 𝙨𝙩𝙖𝙧𝙩𝙚𝙙 𝙬𝙞𝙩𝙝 𝙖𝙘𝙩𝙞𝙫𝙚𝙡𝙮 𝙖𝙨𝙨𝙞𝙨𝙩𝙞𝙣𝙜 𝙙𝙚𝙫𝙨 𝙖𝙩 𝚠̲𝚛̲𝚒̲𝚝̲𝚎̲ 𝙩𝙞𝙢𝙚, 𝙗𝙪𝙩 𝙂𝙚𝙣𝘼𝙄 𝙖𝙜𝙚𝙣𝙩𝙨 𝙣𝙖𝙩𝙪𝙧𝙖𝙡𝙡𝙮 𝙛𝙞𝙩 𝙖𝙩 𝙙𝙞𝙛𝙛 𝚛̲𝚎̲𝚟̲𝚒̲𝚎̲𝚠̲ 𝙩𝙞𝙢𝙚. Iterative agenic workflows can run at review with automated vetting of generations, and only serve edit suggestions shown to improve a tested outcome (i.e. memory efficiency). Bonus is that this can scale with inference time compute to try more or less variations or revision cycles. 2) '𝘾𝙤𝙙𝙚' 𝙖𝙨 𝙖𝙣 𝙖𝙗𝙨𝙩𝙧𝙖𝙘𝙩 𝙘𝙤𝙣𝙘𝙚𝙥𝙩 𝙤𝙥𝙚𝙣𝙨 𝙖 𝙬𝙤𝙧𝙡𝙙 𝙤𝙛 𝙥𝙤𝙨𝙨𝙞𝙗𝙞𝙡𝙞𝙩𝙞𝙚𝙨 𝙛𝙤𝙧 𝙣𝙤𝙣-𝙨𝙤𝙛𝙩𝙬𝙖𝙧𝙚-𝙙𝙚𝙫 𝙚𝙣𝙜𝙞𝙣𝙚𝙚𝙧𝙞𝙣𝙜 𝙙𝙞𝙨𝙘𝙞𝙥𝙡𝙞𝙣𝙚𝙨. Many tools in these areas involve cumbersome UI's that ultimately translate points and clicks into complex config files, and LLM's are pretty good at editing these underlying files directly. 3) 𝘾𝙤𝙙𝙞𝙣𝙜 𝙖𝙗𝙞𝙡𝙞𝙩𝙞𝙚𝙨 𝙞𝙣 𝙢𝙤𝙙𝙚𝙡𝙨 𝙖𝙧𝙚 𝙜𝙚𝙩𝙩𝙞𝙣𝙜 𝙢𝙤𝙧𝙚 𝙘𝙤𝙢𝙥𝙖𝙘𝙩 𝙩𝙝𝙖𝙣 𝙚𝙫𝙚𝙧. One hackathon team was able to use a combination of synthetic data creation/refinement and model distillation (in this case from my near and dear Llama 405B) to fit highly advanced code translation ability in a 14B model. I have a feeling this trend will continue, which will open up more inference compute heavy coding opportunities like broad repo summarization. Bonus here - the resulting model was open sourced by Jason Lu: https://lnkd.in/gswMbCR3 Shout out to SHACK15 and Cerebral Valley for hosting. Incredible views and vibes.

19

3 Comments

In this episode, we discuss Hymba: A Hybrid-head Architecture for Small Language Models by Xin Dong, Yonggan Fu, Shizhe Diao, Wonmin Byeon, Zijia Chen, Ameya Sunil Mahabaleshwarkar, Shih-Yang Liu, Matthijs Van Keirsbilck, Min-Hung Chen, Yoshi Suhara, Yingyan Lin, Jan Kautz, Pavlo Molchanov. The paper introduces Hymba, a new family of small language models that combines transformer attention mechanisms with state space models for enhanced efficiency and performance. It employs a hybrid approach using attention heads and SSM heads for detailed recall and context summarization, along with optimizations like learnable meta tokens, cross-layer KV sharing, and partial sliding window attention to reduce cache size. Experiments show that Hymba-1.5B-Base outperforms other models under 2B parameters, with improvements in accuracy, cache size, and throughput.

1

LLMs have become more capable with better training and data. But they haven’t figured out how to “think” through problems at test-time. The latest research from Scale finds that simply scaling inference compute–meaning, giving models more time or attempts to solve a problem–is not effective because the attempts are not diverse enough from each other. 👉 Enter PlanSearch, a novel method for code generation that searches over high-level "plans" in natural language to encourage response diversity. PlanSearch enables the model to “think” through various strategies before generating code, making it more likely to solve the problem correctly. The Scale team tested PlanSearch on major coding benchmarks (HumanEval+, MBPP+, and LiveCodeBench) and found it consistently outperforms baselines, particularly in extended search scenarios. Overall performance improves by over 16% on LiveCodeBench from 60.6% to 77%. Here’s how it works: ✅ PlanSearch first generates high-level strategies, or "plans," in natural language before proceeding to code generation. ✅ These plans are then further broken down into structured observations and solution sketches, allowing for a wider exploration of possible solutions. This increases diversity, reducing the chance of the model recycling similar ideas. ✅ These plans are then combined before settling on the final idea and implementing the solution in code. Enabling LLMs to reason more deeply at inference time via search is one of the most exciting directions in AI right now. When PlanSearch is paired with filtering techniques—such as submitting only solutions that pass initial tests—we can get better results overall and achieve the top score of 77% with only 10 submission attempts. Big thanks to all collaborators on this paper including: Evan Wang, Hugh Zhang, Federico Cassano, Catherine Wu, Yunfeng Bai, William Song, Vaskar Nath, Ziwen H., Sean Hendryx, Summer Yue 👉 Read the full paper here: arxiv.org/abs/2409.03733

179

2 Comments

TensorFlow vs. PyTorch: Finding the best Deep Learning framework for your AI journey 🚀 Which framework will boost your career goals? Whether you’re excited about diving into cutting-edge research with PyTorch or eager to create scalable AI models with TensorFlow, each option has its own perks! From how easy they are to use and the support you can get from the community to their flexibility and scalability, understanding these differences can really set you up for success in the fast-paced world of AI. I think Juliette Carreiro's article for @ironhack is a fantastic resource for anyone starting their journey in Data Science and Machine Learning! On another note, I’m still looking for a German-speaking Data Manager to join a leading company on an exciting automotive project in Frankfurt. If this sounds like a good fit for you or someone you know, please reach out here or at [email protected]. Happy to share more details! #DeepLearning #AI #DataScience #MachineLearning #TensorFlow #PyTorch #CareerInTech #Hiring #ironhack #redsofa #datamanager

12

1 Comment

Happy to share our latest work "MSc-SQL: Multi-Sample Critiquing Small Language Models For Text-To-SQL Translation" published in the Table Representation Workshop, NeurIPS 2024. Text-2-SQL translation is an important problem to enable agentic workflows for automating database tasks. Recent advances in the space of text-to-SQL utilize closed proprietary models like GPT-4 that present challenges in accessibility, privacy, and latency. We address these issues by developing small, efficient open models (under 10 billion parameters) for text-to-SQL translation and obtain state-of-the-art results compared to other open models while remaining competitive with larger proprietary models at a much lower cost. Joint work with my colleagues at Layer 6 AI. Ilan Gofman, Zhaoyan Liu, Paul Wu, Noël Vouitsis, Guangwei Yu, Jesse Cresswell, Rasa Hosseinzadeh. Paper link: https://lnkd.in/ghi5mKwB

98

1 Comment

Excited to be at #NeurIPS2024! 🎉 Join us for the poster presentation on "OTTER: Effortless Label Distribution Adaptation of Zero-shot Models" 📍 Location: West Ballroom A-D, Poster #6700 🗓️ Date & Time: Friday, Dec 13, 4:30 PM – 7:30 PM PST Short thread ⬇ 🤔 Imagine you’re taking a standardized exam like the SAT and realize you’ve chosen option (d) on 15 out of 20 questions. What would you do? 🤔 💡 Knowing that standardized exams often have evenly distributed answer choices, you’d likely revisit uncertain questions and modify your answers to the next most plausible options. We translate this intuition into an optimal assignment problem: selecting the most plausible answer set to match the known answer (label) distribution. Our method (OTTER) effectively mitigating label bias and boosting performance by: 🔹 +4.8% for CLIP image classification 🔹 +15.9% for text classification tasks. 📖 Our theoretical results show OTTER addresses label shift efficiently, decomposing errors into label distribution specification and calibration errors. Curious to learn more? Check out our paper: https://lnkd.in/gChtxQ5M Swing by the poster session for a chat—I’d love to hear your thoughts! 🚀

54

Quentin H. and Yanwei (Wayne) Zhang on the Faire Data team recently did some great work fine-tuning Llama3 model to enhance search relevance. Here are some key insights from the work: • While prompt engineering is valuable, LLM fine-tuning is essential for a domain specific task like understanding semantic search relevance. • Fine-tuned open-source LLMs like Llama3 can deliver impressive performance, scalability, and cost-efficiency compared to using proprietary models. • Effective application of a fine-tuned LLM demands a well-defined problem, robust, high-quality labeled data, and ongoing model refinement. Ground-truth labeled data is crucial for maximizing model performance. • Fine-tuned Llama3-8b achieves performance on par with the larger Llama2–13b model. Read the blog post to dive deeper into these findings! https://lnkd.in/ecP_3E7S We are also hiring (https://lnkd.in/dHfRZcPi) #LLM #AI #Finetuning #LLAMA3 #SearchRelevance #Hiring

93

3 Comments

Food is more than just nourishment; it's deeply intertwined with our health, lifestyle, and culture. In the age of AI, eCommerce search is evolving from a tool into a source of inspiration. It's no longer just about showing you what you're looking for—it also offers delightful surprises you didn't know you wanted. Often, we don't know exactly what to add to our cart. Maybe we have a vague idea, a theme, or an impulse—craving traditional Asian cuisine, trying making sushi at home but needing guidance on the ingredients, planning a fun football night, aiming for a healthier lifestyle with gluten-free meals, or simply wondering, "What's for dinner?" Thanks to LLMs and their vast knowledge of brands, cuisines, recipes, nutrition, cultures, and more, we're making these experiences possible. It takes a dedicated team to seamlessly blend the world knowledge of LLMs with Instacart's insight into our users and product catalog. This involves careful engineering, thoughtful product design, and rigorous A/B testing with real-world traffic. It's been an exciting journey collaborating with such a talented team to supercharge eCommerce search with LLMs.

6

📣 📣 📣 [CFP; MRR 2024 Workshop @ SIGIR 2024 Conference]We're excited to invite researchers and practitioners across the spectrum of multimodal modeling, representation learning, and retrieval  to participate in our workshop on "Multimodal Representation and Retrieval." This event is proudly organized in association with SIGIR 2024. Multimodal data is available in many applications like e-commerce production listings, social media posts and short videos. However, existing algorithms dealing with those types of data still focus on uni-modal representation learning by vision-language alignment and cross-modal retrieval. In this workshop, we target to bring a new retrieval problem where both queries and documents are multimodal. With the popularity of vision language modeling,  large language models (LLMs), retrieval augmented generation (RAG), and multimodal LLM, we see a lot of new opportunities for multimodal representation and retrieval tasks. This event will be a comprehensive half-day workshop focusing on the subject of multimodal representation and retrieval. The agenda includes keynote speeches, oral presentations, and an interactive panel discussion. Our objective with this workshop is to capture the interest of researchers in this emerging challenge. By highlighting the novelty and significance of the problem, we aim to attract researchers who are eager to explore and contribute to this field. Join us as we welcome esteemed speakers including Tat-Seng Chua (National University of Singapore), and Dinesh Manocha (University of Maryland), who will share their invaluable insights and expertise. We invite original research & industrial application papers that present research on multimodal data representation and retrieval. 🔗 Workshop website: https://lnkd.in/gcpZtrdc 📅 Submission Deadline: May 5, 2024 (11:59 pm, AOE) 📅 Workshop Date: July 18th, 2024 🌆 Workshop Location: Washington D.C., USA Join us to shape the future of multimodal representation, retrieval, and related applications! ☀ My co-organizers Arnab Dhua (Amazon), Douglas Gray (Amazon), I. Zeki Yalniz (Meta), Tan Yu (Nvidia), Mohamed Elhoseiny (KAUST), Bryan A. Plummer (Boston University) and I look forward to hosting you.

74

4 Comments

Just Arrived in Vienna for #ICML2024, I will be presenting our work on improving GNNs' generalization ability with ensemble learning (https://lnkd.in/gimXe5fS) at Oral 5A (Thu 25 Jul 10:45 a.m. — 11 a.m.) and Poster 2407 (Thu 25 Jul 11:30 a.m. — 1 p.m.). Paper title: From Coarse to Fine: Enable Comprehensive Graph Self-supervised Learning with Multi-granular Semantic Ensemble. A heartfelt thank you to all my collaborators for their incredible support; this work would not have been possible without them. I am also looking forward to reconnecting with old friends and making new ones. If you are attending ICML or are interested in my research, please join me during my presentation session. Additionally, please feel free to reach out to exchange ideas if you are interested in topics such as (1) improving graph model generalization across different datasets, and (2) foundational models for time series forecasting. See you in Vienna! 🇦🇹 #ICML2024 #MachineLearning #ArtificialIntelligence

14

Fast and memory-efficient thanks to QLoRA In this article, we will see how to fine-tune LLMs for function calling #llms #chatgpt4 #llmops #aiml #rag https://lnkd.in/dTksNxB2

2