Ted Li

Activision Blizzard released a 25 page study that incudes an amazing A/B test where they progressively turned off Skill Based Match making (SBMM) TL;DR: SBMM works. Everyone hated SBMM being turned off, with more quitting, less playing and negative blowouts. Great paper, the gaming industry could do more of this publicly SBMM has become a controversial topic in gaming due to its often poor implementation. Essentially, it’s a system that aims to match players of similar skill levels by using hidden MMR or ranks, even in casual play. The idea is to create more balanced and enjoyable matches. However, many players feel it discourages experimentation because performing well can lead to tougher opponents, making games feel uneven.

148

11 Comments

🚀 Just spent some time diving deep into Retrieval-Augmented Generation (RAG) for LLMs, and it’s been eye-opening! For those who aren’t familiar, RAG combines the strengths of traditional information retrieval with generative capabilities, allowing models to “look up” relevant information in a large database before crafting a response. This means we’re not just relying on the model’s pre-trained “memory,” but also dynamically fetching information to enrich responses. The result? Way more relevant and informed outputs, especially for specialized or up-to-date topics. Here’s where I see some real potential: 1. Customer Support Automation: Imagine a chatbot that can instantly access a company’s entire support knowledge base, user manuals, and past customer interactions. Instead of relying on generic, canned answers, it could generate responses tailored to the user’s specific question and product version—leading to faster, more accurate support. 2. Personalized Education: In the field of e-learning, RAG can pull from vast educational content databases, offering tailored explanations based on a learner’s background. Whether it’s math, language learning, or science, this could help create a personalized tutor experience that adapts in real-time to each student’s needs and knowledge gaps. 3. Legal & Compliance Research: For industries like law or finance, where accurate referencing is crucial, a RAG-powered LLM could retrieve and cite relevant cases, regulations, or compliance guidelines while drafting responses or reports. It could save hours of manual research and improve accuracy by drawing on real-time, verified data sources. 4. Healthcare Support: A tool for healthcare professionals where RAG enables retrieval of the latest medical studies, treatment protocols, or drug interactions. Imagine a doctor getting instant recommendations based on the most recent research without having to sift through mountains of publications. 5. Content Creation & Ideation: Need fresh ideas? RAG can look up trending topics, recent publications, and customer insights to help marketers and content creators generate ideas that are relevant to current audience interests, backed up by solid references. Why this is so exciting… RAG’s ability to bridge generative AI with real-time, relevant information retrieval feels like the next step in making LLMs truly practical and innovative. It opens up possibilities for high-stakes applications where accuracy and context are critical—and where generative models alone may fall short. Really looking forward to experimenting further with RAG in my projects. The future of LLMs is looking more dynamic than ever! 🔥

2

2 Comments

Kicking off my first post about Kolmogorov-Arnold Networks (KANs) and their potential to replace MLPs in transformers. This could improve existing model performance through optimized activation functions with the addition of a simple class. My experiments show a mixed bag results for KANs in regular RGB super-resolution tasks, but more promising outcomes in hyperspectral super-resolution as seen in the HSR-KAN paper. Looking ahead, I'm exploring KANs in image-encoded language models. Their strength in hyperspectral SR suggests potential for encoding words within images, possibly boosting multimodal learning performance. While training dynamics remain an area for further study, the fusion of visual and textual data opens exciting possibilities in NLP. Papers: https://lnkd.in/ehrVbZqS https://lnkd.in/egkhBENp

5

📖 "Efficient Multimodal Large Language Models: A Survey" - a comprehensive review of recent progress in efficient MLLMs. Key advances include:   💡 Architecture: optimizations in vision encoder, vision-language projector, language model, vision token compression, and using efficient structures like MoE and Mamba. 💡 Training: pre-training, instruction-tuning, and fine tuning 💡 Data: pre-training data, instruction data and benchmarks   Efficient MLLM efforts aim to make the powerful models more accessible - reducing cost and latency, and enabling deployments on device for privacy protection.   Read the full paper: https://lnkd.in/gFEw_XQ8   #generativeai #MLLM #largelanguagemodels

21

🚀 𝗥𝗲𝗰𝗲𝗻𝘁 𝗛𝗶𝗴𝗵𝗹𝗶𝗴𝗵𝘁𝘀 𝗶𝗻 𝗔𝗜 𝗥𝗲𝘀𝗲𝗮𝗿𝗰𝗵 🚀 𝗚𝗿𝗼𝗸𝗙𝗼𝗿𝗺𝗲𝗿: Graph Fourier Kolmogorov-Arnold Transformers Introducing a novel Graph Transformer network that surpasses conventional self-attention mechanisms. 1. Tackles limitations of existing GTs in modeling complex node label patterns. 2. Incorporates learnable activation functions in the graph spectrum. 3. Demonstrates superior performance on multiple node and graph classification datasets. Paper: https://lnkd.in/gDKKk-2x 𝗦𝗧𝗔𝗥 𝗔𝗧𝗧𝗘𝗡𝗧𝗜𝗢𝗡: Efficient LLM Inference Over Long Sequences Recent advancements in Large Language Models (LLMs) now handle contexts extending to millions of tokens. 1. New attention mechanism designed to address computational challenges during inference while preserving accuracy. 2. Opens up new applications, including repository-level code analysis, multi-document summarization, and large corpus retrieval. Paper: https://lnkd.in/gtj29ghz 𝗚𝗹𝗶𝘁𝗰𝗵 𝗧𝗼𝗸𝗲𝗻𝘀 𝗶𝗻 𝗟𝗮𝗿𝗴𝗲 𝗟𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗠𝗼𝗱𝗲𝗹𝘀: 1. Categorization Taxonomy and Effective Detection. 2. Addressing the issue of glitch tokens that lead to hallucinations and erratic behavior in LLMs. 3.Introduction of GlitchHunter, a novel clustering-based detection method. 4.Constructs a Token Embedding Graph (TEG) and uses clustering for efficient detection. 5.High precision and recall across multiple LLMs. Paper: https://lnkd.in/gTVr8jgK #MachineLearning #AIResearch #GraphTransformers #LLM #DeepLearning

4

I'm in love with all the work Meta does for the open source community Their latest release of Co-Tracker3: (https://lnkd.in/enxCMWtv) is the kind of tracking algorithm I've been looking for to help me track all players in any given sports game - with a higher degree of accuracy My primary objective is to track and create movement based transcripts for each player on the field just from video The interesting thing about "movement of the body" Is that its the common denominator among athletes, sport coaches, athletic development staff, therapists etc... -- it affects us all I should also mention - each movement you do in any circumstance (gym, training practice, games) Contains so much data (kind of crazy to think about) and we can create movement based transcripts for better understanding how movement affects certain events from happening Information that can be found in these transcripts ranges from: Muscles... Joints... Force directions: horizontal push/pull, vertical push/pull, lateral push/pull, rotational left/right Planes: sagittal, frontal, transverse Movement patterns: rotate, hinge, lunge, squat, gait... Movement directions: forward, backward, upward, downward, left, right, clockwise, anti-clockwise, diagonal Energy Systems: aerobic, anaerobic I could go on and on here... These are all DATA points that describe "movement of the body" I often ask myself why we don't have tools that give us this level of detail We have all the components to make this happen (Vision Language Models (VLMs) and computer vision algorithms like Co-Tracker, YOLO etc... Note: we do not need expensive sensors to extrapolate all this kind of data. All you need is video of athletes moving around on a field, track, ice, gym (whatever)- thats it

3

Microsoft has launched its new Phi-3 LLMs shortly after the release of Llama 3, showcasing impressive performance benchmarks. Despite Phi-3 models being significantly smaller and trained on fewer data tokens, they surpass Llama 3's capabilities, even in their mini versions. Phi-3-mini has only 3.8 billion parameters, which is less than half the size of Llama 3 8B, yet it performs comparably to larger models like Mixtral 8x7B and GPT-3.5. #AIKnowledge #AIforBusiness #AILeadership #sushtend #AI #GenerativeAI #DigitalTransformation

4

This approach makes perfect sense. This is a common recommendation on training Lora's - improve your labeling - especially label everything in the scene not just the main subject. So wonderful to see so much progress

1

After 3 months with GPT-4, I finally purchased a subscription for Anthropic's Claude 3.5 Sonnet. Game-changer alert! As an ML practitioner, I can confirm: Sonnet's code generation isn't just marginally better—it's in a league of its own. Clean, bug-free code, consistently, almost EVERY SINGLE TIME. Benchmarks aside, the real-world difference is night and day.

279

21 Comments

🔮RAG + KG: Powerful Combination for Translation Multi-AI Agents Large language models (LLMs) have shown great promise in machine translation, but they still struggle with contextually dependent terms, such as new or domain-specific words. This leads to inconsistencies and errors that are difficult to address. Existing solutions often depend on manual identification of such terms, which is impractical given the complexity and evolving nature of language. While RetrievalAugmented Generation (RAG) could provide some assistance, its application to translation is limited by issues such as hallucinations from information overload. =================== In this paper, researchers propose CRAT, a novel multi-agent translation framework that: 》 leverages RAG and causality enhanced self-reflection to address these challenges. This framework consists of several specialized agents: ✱ the Unknown Terms Identification agent detects unknown terms within the context, ✱ the Knowledge Graph (KG) Constructor agent extracts relevant internal knowledge about these terms and retrieves bilingual information from external sources, ✱ the Causality enhanced Judge agent validates the accuracy of the information, ✱ Translator agent incorporates the refined information into the final output. =================== This automated process allows for more precise and consistent handling of key terms during translation. Their results show that CRAT significantly improves translation accuracy, particularly in handling context-sensitive terms and emerging vocabulary. --- If your company is looking for an AI consultant to accelerate business growth, or if you are looking to build your career in AI, send me a message and I can help

4

Gigi v0.99.2b released https://lnkd.in/gp-MqKXR Shader asserts are in. Thanks Alexey Gladkov aka @Rewlion! Example: /*$(Assert: i < 5, "i was %f", i)*/ See UserDocumentation/GigiShaders_Documentation for more info. Functionality: * Set Variables have support for min and max * Subgraph loops can use CONSTANT variables for loop count. Dynamic variable support coming later. Compiler: * Better error messaging when subgraphs fail to inline. Says what the problem is. * Large graphs were having slow downs due to ReferenceFixupVisitor scanning the graph too deeply too often. A visited flag cleaned this up. Viewer: * Variables can have a UI scope to group them better in the UI * Conditional Copy Resource nodes now say whether their condition is true or false in the render graph window. * Better protection against using bad formats for depth textures * Better explanation when a texture can't be created (like, when it has an invalid size specified) * New snapshots interface to be able to save and load cameras, variable values, imported resource settings, or what window is being viewed. Generalizes a couple pre-existing features that needed more functionality. Techniques: * GigiJam2024 techniques added * A depth of field comparison demo at Techniques/PostProcessing/DepthOfField/dof_demo.gg

62

speechcluster -- "a speech database builder's multitool" -- is refreshed from the archives and now on the PYthon Package Index. My plan is to expand this into a frontend to PyTorch/JAX/etc for working on acoustic models (in particular, building from scratch, and tuning). Celebratory blog post forthcoming but for now I just thought I'd leave this here. https://lnkd.in/en_cAWfZ

3

🚀 Excited to share the second volume of my Unity Catalog series! In "How to Master Unity Catalog: Volume 2 From Concepts to Code," I dive into practical code examples for configuring and implementing Unity Catalog in Databricks. Whether you're just getting started or looking to refine your data management strategies, this guide will help you put key concepts into action. Check it out and let me know what you think! #databricks #unityCatalog #dataGovernance #cloudcomputing #azure #aws 😎

9

FlexAttention: Simplifying Attention in PyTorch _________________________________________________________________ FlexAttention is a new PyTorch API that combines the flexibility of PyTorch with the performance of optimized attention methods. It allows researchers to implement various attention mechanisms easily without needing to write custom kernels. Key Points: - Flexibility with Performance: -- Traditional attention methods offer high performance but limited flexibility. -- FlexAttention provides a flexible API that supports many attention variants with minimal effort. - How It Works: -- Standard Attention: Calculates attention scores using queries, keys, and values, normalizes them, and computes the weighted sum of values. -- FlexAttention: Allows for modifications to the attention scores through a flexible scoring function, enabling customization for different types of attention mechanisms. - Examples of Attention Variants: -- Relative Position Encoding: Adds positional information to attention scores based on relative positions. -- ALiBi Bias: Applies a bias to attention scores to account for position differences, enhancing model performance. -- Causal Mask: Masks future positions in the sequence to ensure the model only attends to previous positions, useful for autoregressive models. Handling Sparsity: -- BlockMask: Efficiently manages sparse attention masks, such as those used in causal attention, by creating masks that prevent unnecessary computations. Learn More: https://lnkd.in/g58pxB4A

8

We’ve all heard about RAG (Retrieval Augmented Generation), but now there’s something even more exciting—RIG (Retrieval Interleaved Generation) RIG not only taps into external knowledge like RAG, but interleaves that information directly into the text, creating smoother, more accurate and context-rich responses from large language models (LLMs). Why RIG is a game-changer: ✅ Improved Accuracy: Fact-checks in real-time to reduce hallucinations. 💡 Deeper Contextual Understanding: Captures the nuances of queries with ease. 📊 More Informative Responses: Effortlessly integrates relevant knowledge for richer outputs. More Curious how RIG is transforming the way LLMs work... Read more about it here: https://lnkd.in/g3cPC-tW #RIG #AI #LLM #NaturalLanguageProcessing #Innovation #FutureOfAI

21

Our friends at Manning Publications Co. have a new book in early access on "Knowledge-Graph Enhanced RAG" by Tomaz Bratanic and Oskar Hane. A problem with traditional RAG is that just splitting up documents into chunks in a vector DB doesn't always result in great relevancy to a given prompt; the more structured representation of a graph database can help. They also describe a hybrid approach of blending vector search with old-school full-text search for better results. I've already learned from the chapters they've published so far; check it out if you're so inclined. https://lnkd.in/e29FMt45

33

1 Comment

Intuitive Example, Principles, Code Explanation and Insights about CRAG Take your LLM performance to the next level. #llm #llmops #chatgpt4 #mlops #ai #datascience #ml #gemini tps://https://lnkd.in/dDkZiZ5m

13

Microsoft Research's Vasa-1 can create realistic, audio-driven talking faces from a single photo, potentially enhancing real-time digital interactions. Meanwhile, Hippocratic AI collaborates with NVIDIA to launch AI nurses costing less than $9 per hour, gaining high ratings for their bedside manner and patient education. ► Microsoft Research unveils Vasa-1, an AI that produces lifelike, real-time talking faces, enhancing virtual communications. ► Vasa-1's capabilities include detailed facial expressions and high-quality video generation, promising a breakthrough in real-time AI interactions by 2025. ► Hippocratic AI and NVIDIA introduce affordable AI nurses, praised for their interaction and educational skills with patients. ► AI nurses show superior technical performance compared to human nurses in various assessments. https://lnkd.in/gtU6r2Bv https://lnkd.in/gTcqk_BP #AIKnowledge #AIforBusiness #AILeadership #sushtend #AI #GenerativeAI #DigitalTransformation

2