Guangdong Liu

Read to learn more from Intel about retrieval-augmented generation (RAG), and its impact on the journey toward intelligent and trustworthy #AI. https://lnkd.in/g_rg6j86

1

Using generative AI, NVIDIA operations built an AI Planner agent, developed on NVIDIA Inference Microservices (NIM). The agent leverages LLM, NeMo Retriever and CuOpt NIM to reduce re-planning time from hours to just seconds. https://nvda.ws/4bIfS2T

2

See how our #NVIDIAResearch team has developed a method to efficiently create smaller, accurate language models by using structured weight pruning and knowledge distillation - offering several advantages for developers: ✅ 16% better performance on MMLU scores ✅ 40x fewer tokens for training new models ✅ Up to 1.8x cost saving for training a family of models 🦙 The effectiveness of these strategies is demonstrated with the Llama 3.1 8B model, which was refined into the Llama-3.1-Minitron 4B. This new model will soon be available in the NVIDIA Hugging Face collection. ➡️ Technical Deep Dive

1

Using generative AI, NVIDIA operations built an AI Planner agent, developed on NVIDIA Inference Microservices (NIM). The agent leverages LLM, NeMo Retriever and CuOpt NIM to reduce re-planning time from hours to just seconds. https://nvda.ws/4bIfS2T

20

1 Comment

Massive news for AI enthusiasts (and anyone looking to give their business a serious edge)! Did you hear about Google's new Tensor Processing Unit (TPU) v6, codenamed Trillium? This is a game-changer for AI processing power. Ready to supercharge your AI efforts? ⚡️ Reach out to the team CloudWerx today, and let's chat about how TPUs can revolutionize your business!

20

New on Runtime: Anyscale is built around Ray, an open-source project that was designed to help AI workloads scale. But in recent years, commercial pressures have forced several companies with similar open-source origin stories to put restrictions on their projects to ward off competition. How will Anyscale resist the same pressure as it grows? As Robert Nishihara put it, "The value of our platform, of our product, can't just be managing the open-source project. That's not enough." Part one of our interview with Robert and new Anyscale CEO Keerti Melkote ran in last week's Runtime newsletter, and in this Q&A, we focus on the opportunities and pitfalls of building an infrastructure business around an open-source project.

11

1 Comment

Resharing it: This is a great summary shared by the LinkedIn engineering team about their #llmops and LLM deployment work and their key learnings. #llms #chatgpt4 #gemini #aiml #mlops #llmops https://lnkd.in/d2bH5GRR

2

🚀 How much does it costs to Training LLMs and Where Smaller Players Fit In? Training Large Language Models (LLMs) like GPT-4, LLaMA 3, and Gemini is not a means feat. It requires massive amounts of data, compute power, and financial investment. As these models grow, so do the resources needed. Here's a rough breakdown of the costs: 💡 GPT-4: OpenAI spent $100 million+ to train GPT-4, running tens of thousands of GPUs for weeks. The energy used was comparable to powering entire towns. 💡 LLaMA 3 (Meta): Meta trained LLaMA 3 using huge datasets and a distributed compute setup, likely spending similar amounts as GPT-4. 💡 Gemini (Google DeepMind): Google used its global data center infrastructure to train Gemini, a model with hundreds of billions of parameters, likely exceeding $100 million in costs. 🔋So where does these costs come from?: 1. Data: These models consume terabytes of text data, processed and structured before feeding into training pipelines. 2. Compute: Thousands of GPUs or TPUs power high-performance clusters that run continuously for weeks or months. 3. Energy: GPT-3 used 1.3 GWh of electricity during training, which could power 120 U.S. homes for a year. ⏩ How does the Future of the next Gen LLMs look: The next generation of LLMs could cost $500 million+ to train, pushing the boundaries of hardware and energy efficiency. Only big players like OpenAI, Google, Meta, and Microsoft will have the resources to lead an effort at this scale. 💡Where does this leave the Smaller Players: We will get into a future where we will start leveraging the best of both worlds. Smaller companies won’t train LLMs from scratch but will: - Fine-tune pre-trained models on specialized datasets. - Build niche applications by customizing LLMs for specific industries. - Focus on optimizing models for efficiency and scalability. - Rely on open-source models to innovate in targeted areas. The future will be a hybrid world where Big tech will continue to drive the training of foundational models, while smaller players will carve out unique applications, making AI accessible across industries. #LLMs #AI #GenerativeAI #MachineLearning #GPT4 #LLaMA3 #Gemini #AIResearch #TechTrends #AIInnovation #AIInfrastructure #FutureOfAI #AIStartups

30

3 Comments

Hello, colleagues, this Yahoo article is interesting and relatable for professionals. Here is a teaser: "Amazon has an incredibly granular list of criteria that need to be met before you can be considered for promotion, so I worked with my manager to identify the gaps in my performance, ... and I created a project for each gap that, if completed, could serve as evidence of my ability. I worked backward to create a strict timeline to finish my projects ... that helped me get systematic about my approach moving forward." https://lnkd.in/dsfvPeAk Take a lesson from this person. I seems Amazon gets very analytical when it comes to promotions. And in a company that size, it is impossible to not promote without rigid check-off type policies or whatever XYZ job promotion software. But, hey, there is a lot I will do for a $500,000 salary. Please take note of this last sentence, if anyone works for Amazon and wants a Quality Engineering Manager/Director anywhere in the world.

1

1 Comment

“Google was built for this moment. We’ve been pioneering GPUs for more than a decade,” Google CEO Sundar Pichai Introducing Trillium, Google's sixth-generation Tensor Processing Unit (TPU). Trillium was specifically designed to accelerate the development and deployment of advanced AI models. With significant improvements in compute power, memory, and networking capabilities Key advancements in Trillium include: ➡ Enhanced compute performance: 4.7X increase per chip, enabling faster model training and serving. ➡Increased memory and bandwidth: Doubled High Bandwidth Memory (HBM) capacity and bandwidth, along with Interchip Interconnect (ICI) bandwidth, for handling larger models and datasets. ➡SparseCore accelerator: third-generation SparseCore, Specialized for processing ultra-large embeddings common in advanced AI workloads. ➡Sustainability: 67% more energy-efficient than the previous generation. 👉 👉👉👉 https://lnkd.in/dyASZGWy #AI #ML

2

1 Comment

LLMs and Optimization: An Unbeatable Combo!

19

7 ways to slash latency in System Design: When I worked on Meta's distributed data store, managing massive data wasn't the most challenging part – fighting latency was. After years of trial and error, I'm sharing 7 proven best practices for building low-latency systems. 1. Choose the correct architecture Swap slower monolithic architectures for modular, scalable systems that cut latency by operating independently. 2. Optimize data management Choose SQL for structured data or NoSQL for flexibility, and use indexing, sharding, and replication to speed up data retrieval. 3. Improve network design Reduce data travel with CDNs, load balancers, and edge caching. 4. Use efficient communication protocols Use fast protocols like HTTP/2, WebSockets, and gRPC to streamline communication and cut round-trip times. 5. Optimize your code Use efficient algorithms, data structures, parallel processing, and I/O reduction to speed up execution and reduce bottlenecks. 6. Use high-performance hardware Opt for SSDs over HDDs and low-latency cloud services like AWS Global Accelerator for faster performance. 7. Implement smart caching strategies Cache at multiple layers with LRU/LFU eviction and sync updates to avoid stale data. Read the full blog for a closer look at these 7 practices and how to implement them. (Your system—and users—will thank you). https://educat.tv/3B5Dh0N #SystemDesign #SoftwareEngineer

20

This week in RL research, a new paper was published on leveraging contrastive learning for efficient robotic manipulation, and it's set to revolutionize the field 🚀 The researchers discovered that: • By using contrastive learning, they could reduce the number of required environment interactions by 90%, making robotic manipulation more efficient and cost-effective. • The proposed method outperformed existing state-of-the-art methods in terms of sample efficiency and task success rate. • The approach can be easily integrated into existing reinforcement learning frameworks, making it a promising solution for real-world applications. This research could be particularly helpful for improving robotic manipulation with reinforcement learning, as it enables robots to learn from fewer interactions and adapt to new tasks more quickly. Read the full paper to dive deeper into the methodology and results: https://lnkd.in/gQQR9dBn. Share your thoughts on this breakthrough research and let's discuss how it can be applied in real-world scenarios 💬 #ReinforcementLearning #RoboticManipulation #RLJobs

7

This is a good post that seems to reflect how many business people may be seeing the future. Here is a little reaction just to the GenAI-will-change-coding part: I enjoy having access to quick softmaxed reproductions of scraped code examples as much as everyone else, but really wish we would all stop spreading the "bots are going to do the coding" trope that so many people in a decision-making position seem to believe in, without much basis and lacking the understand of what software engineering really is - not only producing code, but specifying behaviour and being able to change systems in the face of changing requirements. Good luck with copying-and-pasting that!

19

2 Comments