Yi Wang

New 📝 Step-by-step instructions on how to ✅ Optimize LLMs with NVIDIA TensorRT-LLM, ✅ Deploy the optimized models with Triton Inference Server, ✅ Autoscale LLMs deployment in a Kubernetes environment. Technical Deep Dive: https://nvda.ws/3NyjtWX

216

1 Comment

These findings motivate our design of a novel framework to investigate hidden representations in LLMs with LLMs, which we call Patchscopes. The key idea behind this framework is to use LLMs to provide natural language explanations of their own internal hidden representations. Patchscopes unifies and extends a broad range of existing interpretability techniques, and it enables answering questions that were difficult or impossible before. For example, it offers insights into how an LLM's hidden representations capture nuances of meaning in the model's input, making it easier to fix certain types of reasoning errors. While we initially focus the application of Patchscopes to the natural language domain and the autoregressive Transformer model family, its potential applications are broader. For example, we are excited about its applications to detection and correction of model hallucinations, the exploration of multimodal (image and text) representations, and the investigation of how models build their predictions in more complex scenarios. Google DeepMind

9

Learn how an image-text multi-modality model can perform image classification, image retrieval, and image captioning #llms #llmops #image #opencv #chatgpt4 #gemini #aiml https://lnkd.in/gJFh9Ruf

1

A recent breakthrough titled "Matrix Multiplication-Free LLMs" demonstrates a huge advancement in the area of Large Language Models (LLMs) by reducing computational costs. The authors have eliminated MatMul operations from LLMs, claiming to 10 times reduction in memory usage and a 25.6% increase in training speed, all while maintaining strong performance at billion-parameter scales. Paper link: https://lnkd.in/ggph8qXc #AI #machinelearning #deeplearning #LLMs

16

👉 Learn how to simulate pre-trained models from #PyTorch and #TensorFlow directly in #Simulink with the new co-execution Simulink blocks. https://spr.ly/6044W3drs

2

Hallucinations in LLMs are inevitable, no seriously! Josh Fritzsche and I have worked hard at Voze to find a way to tune, prompt and even plead 🙏 with LLMs to prevent them from the occasional 🍄 hallucination. It seemed like the better we have done the harder the edge cases have been to solve and eventually we realized that they tend to frequently happen when LLMs predict or make anticipations on "nothing". I remember at one point nearly going insane with frustration because when LLMs had a hallucination, they were EPIC! 😱 Then we pushed through it and benefited for the effort! How did we solve this common problem? We didn't (Not completely), we got better at identifying the issues as they happened. Once we identified in the data we were able to start looking for patterns. The same catalysts that cause it to happen once if left unchanged caused it over and over again. Our solution is astoundingly simple. Once you know that it's likely to happen..... let it! Accept it will 🍄, but also be preemptive in not focusing on it and use alternative data sources, generators or providers. Change the conditions that cause it but continue to monitor it as it happens. As Dan Caffee is fond of saying to me on a regular bases. "Dusty we need the system to be adaptive and provide mechanisms to learn from it's past." Well he is right! All we needed was data and the people that contribute to it's quality. Huge props 👏 to strong support from Kathy and Janelles audit teams. We were armed with a knowledge and a desire to learn from past mistakes. Armed with confidence that we could correct these problems we could focus on the data to prevent their impact going forward. When doing millions of LLM (Generative AI) calls regularly Josh and I have learned that hallucinations will happen. We also know exactly how we will handle and prevent it going forward! Failure is not doing anything and accepting defeat.

4

2 Comments

SQLite getting stronger stand in VECTOR USECASES Will surely give it a try soon.

4

Put the power of quantum computing right into your workflows. Access everything you need: hybrid solvers, SDK, open-source tools, libraries, code examples, demos, and comprehensive documentation. It’s all available through the Leap™ quantum cloud service. Begin building quantum applications with ease today: https://lnkd.in/eNJUJMx

10

1 Comment

✅ DIY Code Completion with #Mamba 🐍 ➡️ https://nvda.ws/3SLBCDq ✨ ⚡ #TensorRT #LLM for speed 🤖 NIM for ease and ☁️ Deploy anywhere

2

New article: Easily Deploy and Serve a LLM with NVIDIA NIM In this article, we discover how to deploy and serve a large language model through NVIDIA NIM using their API catalog. Enjoy the read! https://lnkd.in/e8bJQmtu #llm #deployment #llmops #mlops #deeplearning #machinelearning #python

37

1 Comment

New paper: Shared Memory Parallelism in Modern C++ and HPX https://lnkd.in/ghYcgGkQ

39

Function calling is a powerful way to extend the capabilities of LLMs and AI agents by letting them use external tools. Our new short course Function calling and Data Extraction with LLMs, created with Nexusflow and taught by Jiantao Jiao and Venkat Srinivasan, demonstrates how to prompt LLMs to form calls to external functions. You'll work with NexusRavenV2-13B, a 13B parameter open-source model that excels in function calling tasks while still being small enough to host locally. Learn to use function calling to extract structured data from unstructured text and access web APIs, and build an end-to-end application that processes customer service transcripts. You'll learn how to build LLM-powered applications that can analyze feedback, automate data entry, and enhance search. Please get started here: https://lnkd.in/g9FpiCuH

5,028

111 Comments

🚀 Breaking News: Cerebras Systems Inference Gets 3x Faster! Llama 3.1-70B at 2,100 tokens / second⚡️  Here's what this means:  ⚡️ 16x faster than the fastest GPU solution  💪 8x faster than GPUs running Llama3.1-3B (a model 23x smaller!)  🎯 Equivalent to a full GPU generation upgrade (H100/A100) in one software release 🌐Real-World Applications Already Transforming Industries: 🧬 GSK (quote from Kim Branson, SVP of AI/ML): "Cerebras' inference speed is fundamentally improving our drug discovery process through innovative AI applications and intelligent research agents." 🎙️ LiveKit (powering ChatGPT's voice mode): "With Cerebras Inference, what was once the slowest stage in the pipeline is now the fastest" - Russ d'Sa, CEO 🛠️ Technical Improvements Under the Hood: Rewritten & optimized critical kernels (MatMul, reduce/broadcast, element-wise ops) Asynchronous wafer IO computation Advanced speculative decoding implementation Maintained 16-bit original weights precision Verified accuracy by Artificial Analysis ⚡️ What This Means for AI Development: Instantaneous voice & video processing Advanced reasoning capabilities Real-time intelligent applications 10x more reasoning steps without latency increase Enhanced multi-step agentic workflows  Experience the Speed Yourself: Try Cerebras Inference via chat or API at inference.cerebras.ai #LLM #Inference #GenAI

74

1 Comment

New use cases for LLMs are emerging every day, and a recent paper from MIT highlights an exciting application in anomaly detection. The researchers demonstrated how LLM models can effectively detect anomalies in time series data, eliminating the need for traditional deep learning and extensive ML training. This approach can be particularly valuable in the security space, where we can apply similar techniques to analyze time series data to enhanced security monitoring.

17

Effortless Integration for Smarter Insights: CloudEagle.ai Meets Bitbucket Integrating CloudEagle.ai with Bitbucket takes just minutes and gives you: ✔ Simplified App Tracking: No more manual reports, get the app usage data you need, when you need it. ✔ Centralized Control: Manage your Bitbucket security from the CloudEagle dashboard. ✔ Proactive Monitoring: Catch unusual login activity before it becomes a problem. Ready for a smarter, more secure development workflow? Book a demo today - https://lnkd.in/gXAhChrt #saas #saasmanagement #toolstuesday Atlassian #jira #toolstuesday

5

Huge release by Daekeun Kim and Hyo (HK) Choi from our AI GBB team in Korea, sharing Korean language benchmarks for our latest Azure OpenAI model, gpt-4o-mini - like gpt-4o, the model excels on Korean language tasks, in this case mini completely eclipses gpt-35-turbo and nearly achieves gpt-4-turbo performance. The benchmarks and code are open source, extensible beyond our Azure models, so we encourage our customers and the larger community to try it out and even fork/share your own findings - and please let me in the comments what other Asia languages beyond Korean you may be looking to evaluate with your language models.

52

7 Comments

For the budget sensitive AI teams who can not afford to buy H100 NVIDIA GPUs for their LLM training there is a new option in town: Pytorch Intel extension and Gaudi AI Accelerators ! Also great for Accelerated inference hosting (post training) https://lnkd.in/eXEd5NMk https://lnkd.in/ebbNuz86

1

3 Comments