Yixing Chen

Microsoft Semantic Kernel, their AI SDK, is growing in popularity. 💡 Check out our latest tutorial to discover how to build a simple movie recommendation bot using MongoDB Atlas, the sample movie collection, and Semantic Kernel with OpenAI in C#! https://lnkd.in/g9UfViTc #MongoDB #SemanticKernel #OpenAI #DotNET #GenAI #CSharp

9

Huge release by Daekeun Kim and Hyo (HK) Choi from our AI GBB team in Korea, sharing Korean language benchmarks for our latest Azure OpenAI model, gpt-4o-mini - like gpt-4o, the model excels on Korean language tasks, in this case mini completely eclipses gpt-35-turbo and nearly achieves gpt-4-turbo performance. The benchmarks and code are open source, extensible beyond our Azure models, so we encourage our customers and the larger community to try it out and even fork/share your own findings - and please let me in the comments what other Asia languages beyond Korean you may be looking to evaluate with your language models.

52

7 Comments

🚀 Exciting times for enterprises leveraging generative AI! As large language models (LLMs) evolve, understanding the critical performance metrics of throughput and latency is more important than ever. High throughput means handling more requests efficiently, while low latency ensures a seamless user experience. Finding the right balance between the two is key to delivering superior AI-powered applications. With NVIDIA NIM microservices, businesses can optimize performance, scale their operations, and enhance user satisfaction—all while driving down costs. 🌟Ready to transform your AI strategy? Let’s dive into the future of intelligent applications! #GenerativeAI #NVIDIA #AI #MachineLearning #UserExperience #Innovationhttps://lnkd.in/eMq6j3vB

10

A platform to learn and gain insights as per interest. Look into the GitHub Repositories to gain knowledge on Machine Learning and Artificial Intelligence.

1

How to write a killer Software Engineering résumé? By Terrence Kuo An in-depth analysis of the résumé that got me interviews at Google, Facebook, Amazon, Microsoft, Apple, and more. This résumé got me interviews at Google, Facebook, Amazon, Microsoft, and Apple.I obtained these interviews by sending my résumé to the résumé black hole, also known as applying online. Applying online is the most common way people go about applying for a job and therefore the least effective way to land an interview due to competition. Yet that is exactly how I obtained all my interviews. How did I accomplish this? In this article, I will go through a line-by-line analysis of my résumé for the following purposes: explaining the choices that I made in creating my résumé why I believe this résumé worked to help me land those interviews, and how you can create an even better résumé! I decided to write this article because I struggled a lot with landing interviews when I first started looking for a job. It would have been extremely helpful for me to have a real-life example résumé to look at. This article is organized into the following sections: The All Too Familiar Way of Not Landing an Interview - a short anecdote of my frustrations when I first started applying for jobs Evaluating the Options: Moving Forward - a reflection on different strategies to improve the odds of landing interviews Learning How to Write a Killer Résumé By Example - the step-by-step analysis of my résumé with each of the following sections corresponding to my résumé: 1.The Essentials from a Glance 2.Who Are You 3.Contact Information 4.Education 5.Employment 6.Personal Projects 7.Skills

112

1 Comment

How to create a small and efficient cluster for deploying optimization models with TensorRT, including creating Infrastructure as Code. -------- Triton Inference Server: is an open source inference serving software that streamlines AI inferencing. K3s: is a lightweight, easy to install, deploy, and manage version of stock Kubernetes (K8s) Flux CD: implements continuous delivery to Kubernetes clusters using a GitOps process Cloudflare R2 Storage: allows developers to store large amounts of unstructured data without the costly egress bandwidth fees associated with typical cloud storage services TensorRT: is an SDK for high-performance deep learning inference on NVIDIA GPUs. #JETSON #NVIDIA #TensorRT #K3S #Triton Ref. https://lnkd.in/gi8VHXmD

9

A recent breakthrough titled "Matrix Multiplication-Free LLMs" demonstrates a huge advancement in the area of Large Language Models (LLMs) by reducing computational costs. The authors have eliminated MatMul operations from LLMs, claiming to 10 times reduction in memory usage and a 25.6% increase in training speed, all while maintaining strong performance at billion-parameter scales. Paper link: https://lnkd.in/ggph8qXc #AI #machinelearning #deeplearning #LLMs

16

Check out this course from deeplearning.ai, explaining the advanced capability of function calling in LLMs. #LLMs #GAI #NexusAI

4

Facebook just released a very different kind of LLMs. These are focused on code optimization and compiler optimization. This is super important because it can fundamentally change the way code is executed on any machine. Here's how it works right now: Programs written in many languages like C/C++ have intermediate steps before they are run on the computer. For example they are converted into Byte Code and then converted into machine code via LLVM (Low level virtual machine). But Facebook's new model can take in the byte code as input and convert it to into a more efficient representation so that when this new code is converted into machine code, it will be faster. "In theory", it could mean that all existing code might be made faster without any change by developers. Our games can have higher frame rates on the same GPUs. Photoshop might run faster on the same laptop. Our software might complete tasks faster without any changes at all. But that will probably take quite a long time. To be honest I don't expect these models to actually be used in production since unless they were 100% as good as compilers they will probably break the code. And are they 100% good? No, not even close. But the speed at which improvements are happening, they will probably reach that level eventually. And that's when we will see models like these everywhere. 

11

April was an exciting month for AI at Meta! We launched MTIA v2 https://lnkd.in/gvC-UgVN, Llama3 https://lnkd.in/gdbfXa-6, presented a tutorial and paper on the PyTorch2 compiler at ASPLOS https://lnkd.in/gsT9DUr5, released PyTorch 2.3 https://lnkd.in/gYn3Vnzm and, to top it off, we launched the PyTorch ecosystem solution for mobile and edge deployments, ExecuTorch Alpha https://lnkd.in/gWv96jpm optimized for Large Language Models. What better than to combine all of these... running Llama3 on an a mobile phone exported with the PT2 Compiler's torch.export, and optimized for mobile deployment. And you can do all of this in an easy-to-use self-service format starting today, for both iPhone and Android as well as many other mobile/edge devices. The video below shows Llama3 running on an iPhone. (Makers will love how well models run on Raspberry Pi 5!) PyTorch is being developed by a multi-disciplinary team comprising ML engineers, accelerator experts, compiler developers, hardware architects, chip designers, HPC developers, mobile developers, and specialists and generalists that are comfortable across many of the layers involved in building end-to-end solutions. Even better -- if you're excited by the possibilities of AI, and solving the system design challenges of making AI run well across all hardware types, we are looking for YOU! The Pytorch team has openings across PyTorch core, compilers, accelerators and HW/SW co-design https://lnkd.in/gu2Y_KpY https://lnkd.in/gQVXBP6P and a broad range of positions that involve PyTorch from model development all the way to hardware deployments https://lnkd.in/gjDdUh5w #PyTorch #ExecuTorch #Llama3 #AICompilers #MTIA #AcceleratedAI #MetaAI #Meta

64

Cerebras and Neural Magic have reached a significant milestone in the realm of large language models (LLMs). Through innovative techniques and purpose-built hardware, we achieved up to 70% parameter reduction in LLMs without sacrificing accuracy. This breakthrough opens doors for more efficient training and deployment, democratizing access to LLMs across various sectors. #AI #MachineLearning #Innovation

9

1 Comment

💥💥💥 KOR-Bench: Benchmarking Language Models on Knowledge-Orthogonal Reasoning Tasks Kaijing Ma, Xinrun Du, Yunran Wang, Haoran Zhang, Zhoufutu Wen, Xingwei Qu, Jian Yang, Jiaheng Liu, Minghao Liu, Xiang Yue, Wenhao Huang, Ge Zhang Abstract In this paper, we introduce Knowledge-Orthogonal Reasoning (KOR), which minimizes the impact of domain-specific knowledge for a more accurate evaluation of models' reasoning abilities in out-of-distribution scenarios. Based on this concept, we propose the Knowledge-Orthogonal Reasoning Benchmark (KOR-Bench), encompassing five task categories: Operation, Logic, Cipher, Puzzle, and Counterfactual. KOR-Bench emphasizes the effectiveness of models in applying new rule descriptions to solve novel rule-driven questions, revealing that top-performing models like Claude-3.5-Sonnet and GPT-4o only achieve 58.96% and 58.00% accuracy, respectively. We conduct thorough analyses to identify bottlenecks in the Cipher task using Stepwise Prompting, discovering that two rounds of Self-Correction yield optimal results. Complex Task Processing evaluates model performance across three integrated tasks, while we also explore the impact of Tricks on the Puzzle task and visualize rule-focused attention to enhance our understanding of model behavior. We aim for KOR-Bench to be a valuable resource for enhancing models' reasoning capabilities and fostering further research in this field. 👉 https://lnkd.in/dzcZ3kpC #machinelearning

1

I have started writing three different topics on iThome Challenge this year! 1. Kubernetes (For rookie) https://lnkd.in/geX74nhn 2. AI research (Retrieval / RL / Multi-modal / Graph) https://lnkd.in/g9G8FvMb 3. LeetCode Daily (Python / C++) https://lnkd.in/gYkguFgi Traditional Chinese write the article. The event lasts for 30 days! Feel free to correct my concept if I have written the wrong information. 😀 #kubernetes #research #leetcode #ai #rag

12

2 Comments

Learn why the NVIDIA Jetson AGX Orin is the leading platform for running transformer models like GPT-J, vision transformers, and stable diffusion at the edge.

1

Lightning AI launched Lit-GPT, part of their Lit-* series, which aims to provide clean, optimized implementations of LLMs with support for pretraining and fine-tuning using techniques like LoRA and Adapter. This guide walks through the process of installing Lit-GPT, downloading model weights, preparing a dataset, and fine-tuning a model. Starting with cloning the repository and installing dependencies, it uses FlashAttention for efficient implementation. Then, it downloads and converts model weights, using the RedPajama-INCITE 3B parameter weights as an example. Then prepares the Dolly 2.0 instruction dataset(from Databricks) in the Lit-GPT format, and running a fine-tuning script. https://lnkd.in/exz22Xrn

9

An amazing article on using  Low Rank Adaptation (LoRA) and QLoRA (Quantized Low-Rank Adaptation)  for fine-tuning LLMs. https://lnkd.in/gHu7hfAE #lora #qlora #machinelearning #llms

16

🚀 Breaking News: Cerebras Systems Inference Gets 3x Faster! Llama 3.1-70B at 2,100 tokens / second⚡️  Here's what this means:  ⚡️ 16x faster than the fastest GPU solution  💪 8x faster than GPUs running Llama3.1-3B (a model 23x smaller!)  🎯 Equivalent to a full GPU generation upgrade (H100/A100) in one software release 🌐Real-World Applications Already Transforming Industries: 🧬 GSK (quote from Kim Branson, SVP of AI/ML): "Cerebras' inference speed is fundamentally improving our drug discovery process through innovative AI applications and intelligent research agents." 🎙️ LiveKit (powering ChatGPT's voice mode): "With Cerebras Inference, what was once the slowest stage in the pipeline is now the fastest" - Russ d'Sa, CEO 🛠️ Technical Improvements Under the Hood: Rewritten & optimized critical kernels (MatMul, reduce/broadcast, element-wise ops) Asynchronous wafer IO computation Advanced speculative decoding implementation Maintained 16-bit original weights precision Verified accuracy by Artificial Analysis ⚡️ What This Means for AI Development: Instantaneous voice & video processing Advanced reasoning capabilities Real-time intelligent applications 10x more reasoning steps without latency increase Enhanced multi-step agentic workflows  Experience the Speed Yourself: Try Cerebras Inference via chat or API at inference.cerebras.ai #LLM #Inference #GenAI

73

1 Comment

With NVIDIA AI Foundry, enterprises and nations can supercharge #generativeAI with a service spanning curation, synthetic data generation, fine-tuning, retrieval, guardrails, and evaluation to deploy custom AI at Meta Llama 3.1 NVIDIA NIM Microservices.

1