About
I've led teams as CTO and Lead Engineer to drive innovation, optimized operations, and…
Services
Experience
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Education
Patents
-
In-Memory Database
US
-
Pattern Search
US
Languages
-
English
Native or bilingual proficiency
-
Chinese
Native or bilingual proficiency
Other similar profiles
Explore more posts
-
Anyscale
🚀 Looking to streamline fine-tuning for models like Llama 3, Mistral, and Mixtral? Our latest blog shows how Anyscale simplifies LLMs fine-tuning with Ray: 🔑 Key takeaways: • Scalable fine-tuning with LLM-Forge • Model Deployment with RayLLM Read more: https://lnkd.in/gQwD4A2U
-
Covalent
Attending PyCon US? Learn how to build an AI foundry with autonomous agents using Covalent Cloud, and deploy your next killer app with just a few lines of code. Don't miss "No Data? No Problem: Zero-Data Model Training with Foundational Models" presented by Santosh Kumar Radha and Ara Ghukasyan at 11:30am today. 👉 Github repo: https://lnkd.in/essje6nv Visit booth 221 to pick up some swag! Python Software Foundation #PyConUS #PyCon #python #llm #genai
-
Unite.AI
Optimizing Memory for Large Language Model Inference and Fine-Tuning - Large language models (LLMs) like GPT-4, Bloom, and LLaMA have achieved remarkable capabilities by scaling up to billions of parameters. However, deploying these massive models for inference or fine-tuning is challenging due to their immense memory requirements. In this technical blog, we will explore techniques for estimating and optimizing memory consumption during LLM inference and fine-tuning across various hardware setups. Understanding Memory Requirements The memory required to load an LLM is primarily determined by the number of parameters and the numerical precision used to store the parameters. A simple rule of thumb is: Loading a model with X billion parameters […] - https://lnkd.in/gDyxsFQn
-
Unite.AI
Optimizing Memory for Large Language Model Inference and Fine-Tuning - Large language models (LLMs) like GPT-4, Bloom, and LLaMA have achieved remarkable capabilities by scaling up to billions of parameters. However, deploying these massive models for inference or fine-tuning is challenging due to their immense memory requirements. In this technical blog, we will explore techniques for estimating and optimizing memory consumption during LLM inference and fine-tuning across various hardware setups. Understanding Memory Requirements The memory required to load an LLM is primarily determined by the number of parameters and the numerical precision used to store the parameters. A simple rule of thumb is: Loading a model with X billion parameters […] - https://lnkd.in/gDyxsFQn
-
Unite.AI
Optimizing Memory for Large Language Model Inference and Fine-Tuning - Large language models (LLMs) like GPT-4, Bloom, and LLaMA have achieved remarkable capabilities by scaling up to billions of parameters. However, deploying these massive models for inference or fine-tuning is challenging due to their immense memory requirements. In this technical blog, we will explore techniques for estimating and optimizing memory consumption during LLM inference and fine-tuning across various hardware setups. Understanding Memory Requirements The memory required to load an LLM is primarily determined by the number of parameters and the numerical precision used to store the parameters. A simple rule of thumb is: Loading a model with X billion parameters […] - https://lnkd.in/gDyxsFQn
-
Eksentricity
SF Bay Area Leads AI Tech and Funding Boom - AI Funding Dominance: Over 50% of global AI venture funding in 2023 went to Bay Area companies, with $27 billion raised. - Top AI Companies: Leading Bay Area AI firms, including OpenAI, Anthropic, and Inflection AI, have each raised over $1 billion and expanded their real estate footprints. - Startup Hub: The Bay Area attracts AI talent and startups, with major accelerators like Y Combinator and 500 Global seeing a significant portion of their AI-focused graduates setting up in the region. Subscribe to our daily newsletter here for more AI news https://lnkd.in/gM5dAd6j #ArtificialIntelligence #Innovation #Technology #VentureCapital #Startups #Entrepreneurship #BigData #Economy #Analytics #Future #AI
-
Octane OC
Exploring Frontiers in Generative AI: Innovations, Challenges, and Future Directions The first day of the Tech Innovation Forum wrapped up with an insightful panel discussion on generative artificial intelligence, moderated by Stuart Mathews, Director of Industry Alliances at UC Irvine's Enterprise Collaborations Group. This session explored generative AI's transformative impact across various sectors. Markus Bernhardt, AI Strategy Lead at the Learning Forum and Principal Consultant at Endeavor Intelligence, explored the use of AI in executive decision-making and the strategic challenges it poses. Anthony Chang, MD, MBA, MPH, MS, Founder of AIMed and Chief Intelligence and Innovation Officer at CHOC Children's, discussed AI’s transformative role in personalizing healthcare, stressing the importance of early education on AI capabilities. Neil Sahota, Chief Executive Officer of ACSILabs, Inc., and IBM Master Inventor highlighted the need for regulatory adaptations and explored AI’s potential as a powerful educational tool. Shanji Xiong, Chief Scientist of the Innovation Lab at Experian, explained how generative AI is reshaping the analysis of financial behaviors and helping make more informed decisions. These discussions not only illuminated the broad applications of generative AI but also effectively concluded the first day of the Tech Innovation Forum with valuable insights into the future of technology. #GenerativeAI #AIInnovation #OCInnovation
4 Comments -
Rebellions
"The true test of LLMs lies in their ability to deliver effective inference in real-world applications." Curious about the future of AI deployment? Learn more about our latest article on enhancing LLM serving and application performance: 👉 https://lnkd.in/ggvBhDZG Peek into our cutting-edge advancements: ✔️Tensor Parallelism: Distributing the inference workload across multiple devices ✔️Weight-Only Quantization: Compressed the Llama3-70B model's 16-bit weights to 4-bit weights ✔️vLLM Integration: Enhance inference speed and efficiency: Enhancing inference performance and efficiency ✔️Rebellions Model Zoo: Pre-trained models optimized for our hardware #AI #RebellionsTech #vLLM #Quantization #RebellionsScalableDesign
-
Enterprise Connect
#EnterpriseConnectAI Session Highlight: Interested in learning about how to structure different prompts and how to incorporate prompt engineering into a pilot? Hear from Jim Lundy, Founder, CEO, & Lead Analyst Aragon Research during his session at Enterprise Connect AI in Santa Clara this fall! Register now: https://lnkd.in/eRKxhmw6 #EnterpriseConnectAI #EnterpriseConnect #AI #ChatGPT #GenAI #GenerativeAI #EnterpriseAI #CXAI #LLMs #LargeLanguageModels #ArtificialIntelligence #Collaboration #Communications #cloud #CX #CustomerExperience #ContactCenter
-
Run:ai
Deploying LLMs at scale presents a dual challenge: ensuring fast responsiveness in high-demand times- while trying to manage the costs of GPUs! Getting this balance is no easy endeavor, so organizations often face a trade-off between provisioning additional GPUs for peak demand or risking SLA during spikes in traffic 🚨 And whether you decide to scale up aggressively from zero, with you users suffering through latency spikes, or deploy many replicas with GPUs to handle the worst-case traffic scenarios and pay for hardware that spends most of its time idling... Neither approach is ideal! We've had the pleasure of introducing you to Run:ai's model streamer, and you've heard a little about our GPU Mem Swap! Now we're bringing you our latest blog by Ekin Karabulut and Yoed Ginzburg to see how it can help you cut costs, without hindering performance--> https://hubs.li/Q02-RWtw0 #MLblog #aiblog #newblogalert #aiinfrastructure #AIOps #mlops #AIDevOps #GPUComputing #AIScaling #MachineLearningInfrastructure #runai #ml #LLM #opensource
-
Approved Networks, a brand of Legrand
At #SC24, Frank Yang saw the continued demand for #AI and how #Nvidia is driving its growth. Despite high costs, smaller companies can benefit from budget-friendly and scalable domain-specific models. Check out the full article for more insights from Frank! https://hubs.li/Q02-zrTK0
-
Better Sheets
How to Merge Cells Vertically https://lnkd.in/g5PQAF9K How to Merge Cells Vertically in Google Sheets Learn how to merge cells vertically in Google Sheets and why it's beneficial for organizing your data. This tutorial covers selecting cells, using the Merge Cells button and options, and aligning text. Additionally, see how color can enhance your merged cells and make your spreadsheets more visually appealing. Subscribe to BetterSheets for more tips! 00:00 Introduction to Vertical Cell Merging 00:04 Setting Up Your Lists for Merging 00:20 Steps to Merge Cells Vertically 00:32 Alternative Methods for Vertical Merging 00:41 Aligning and Coloring Merged Cells 01:11 Aesthetic Benefits of Vertical Merging 01:31 Conclusion and Further Learning
-
PyTorch
🗣️ Submit to speak at PyTorch Conference in San Francisco September 18-19! We're looking for talks on the cutting-edge open source machine learning framework, deep learning, chatbots, security & MORE. We're looking for experts on: 🔥 PyTorch Framework & Ecosystem 🔥 Deep Learning Applications 🔥 Software Applications 🔥 Training & Inference 🔥 Model Lifecycle & Management 🔥 Ethical, Social, & Regulatory Aspects 🔥 AI in Industries ✏️ See alll suggested topics and submit your talk by June 7: https://hubs.la/Q02ttHzl0. 🎟️ Register by April 26 & save US$350: https://hubs.la/Q02ttGHq0. #PyTorchConf #PyTorch #PyTorchFoundation #AI #MachineLearning #ML #DeepLearning #OpenSource #OpenSourceSoftware #OpenSourceDevelopment #OpenSourceCommunity #OSS #LinuxFoundation #events #linux #software
1 Comment -
PyTorch
🗣️ Submit to speak at PyTorch Conference in San Francisco September 18-19! We're looking for talks on the cutting-edge open source machine learning framework, deep learning, chatbots, security & MORE. We're looking for experts on: 🔥 PyTorch Framework & Ecosystem 🔥 Deep Learning Applications 🔥 Software Applications 🔥 Training & Inference 🔥 Model Lifecycle & Management 🔥 Ethical, Social, & Regulatory Aspects 🔥 AI in Industries ✏️ See alll suggested topics and submit your talk by June 7: https://hubs.la/Q02xs2j40. 🎟️ Register by July 12 & save US$250: https://hubs.la/Q02xs9zC0. #PyTorchConf #PyTorch #PyTorchFoundation #AI #MachineLearning #ML #DeepLearning #OpenSource #OpenSourceSoftware #OpenSourceDevelopment #OpenSourceCommunity #OSS #LinuxFoundation #events #linux #software
-
Streamlit
Not science fiction, but it IS a double feature! Come hear Amanda Kelly (one of our co-founders) talk about how Streamlit continues to accelerate data teams and make your lives a bit more fun. 🤠 Looking forward to hosting the (WiMLDS) Women in Machine Learning & Data Science at Snowflake HQ! #ai #machinelearning #womenindata #python
-
FriendliAI
Welcome to our new blog Series on Friendli Model Optimizer! Friendli Model Optimizer is a quantization tool that you can use to improve inference speed, lower resource consumption and maintain model accuracy. In this article, we’ll explore how to use the FMO library and understand its quantization feature. Stay tuned for part 2 of this blog, where we'll dive deeper into the performance analysis. Read Full blog 👉 https://lnkd.in/d-WU5MgJ Try friendli Suite 👉 https://suite.friendli.ai/ #FriendliAI #Friendli #FriendliModelOptimizer #LLM #Inference
-
Rebellions
#RebellionsTech Rebellions is pushing the boundaries of AI compute with native PyTorch 2.0 support and optimized LLM serving using vLLM. Our latest developments include: ✅Seamless integration with PyTorch 2.0's torch.compile ✅Continuous batching for LLM serving on our ATOM chip ✅Upcoming PagedAttention and FlashAttention optimizations Discover how we're enhancing developer experience and contributing to the AI/ML ecosystem: https://lnkd.in/eZYg2s9M #TorchDynamo #AOTAutograd #PrimTorch #TorchInductor #vLLM #PagedAttention #FlashAttention #ContinuousBatching
Explore collaborative articles
We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.
Explore MoreOthers named James Wong in United States
-
James Wong
-
James Wong
-
James Wong
5+ years in building and enhancing teams for top-tier fintech and blockchain companies. Areas of interest: HR, talent acquisition, financial markets. As much as I would like to, I may not get to your LinkedIn message.
-
James W.
-
James Wong
704 others named James Wong in United States are on LinkedIn
See others named James Wong