Syed Jafri

With NVIDIA now crowned the most valuable company in the world, it's time to reflect on this wild ride (I know, I am not an investor and I am not an AI expert). Scott Galloway said it best: "Nvidia, to justify its valuation, will need not only to continue to dominate the market for AI chips, forecasted to grow aggressively, but also to dominate another market of similar scale. Think about this: Built into the price of Nvidia is the expectation the company will find another market as big as AI, and achieve similar dominance." Hold onto your GPUs, folks! The AI hype train is at full speed, and who knows where it's headed next? 🌟💼💡 For the record, I’m definitely in the AI believer camp, but a bubble remains a bubble. #AI #NVIDIA #TechBubble

32

2 Comments

The game of large language models (LLMs) got a major plot twist! Meta's unveiling of Llama, a family of AI models pushing the boundaries of parameter size and processing power, is a game-changer. Why it matters? Llama boasts a staggering 405 billion parameter model, dwarfing previous heavyweights like Google's LaMDA and OpenAI's Jurassic-1 Jumbo. This sheer size unlocks unprecedented capabilities in: -> From scientific papers to philosophical treatises, Llama can digest vast amounts of data, leading to deeper comprehension. -> Imagine writing styles that perfectly mimic different authors or create compelling narratives on the fly. -> From machine translation to chatbot interactions, Llama holds the potential to transform how humans and machines communicate. -> Llama isn't just about brawn. Meta stresses its commitment to open-source access and responsible AI development. This fosters collaboration and helps mitigate potential biases often associated with large datasets. Meta's ambitious project is a harbinger of the future, where AI can not only process information but truly understand it. Imagine personalized education tailored to individual learning styles, or AI assistants that anticipate your needs before you even ask. https://lnkd.in/dYn_cTjB #MetaLlama #ai #LLMs #futureoftechnology #naturallanguageprocessing #opensource

6

Just published my latest article: "Transforming Business Operations with LLMs: A Path to a Production Ecosystem." a breakdown of the key components necessary for integrating LLMs into core business operations, including Human In The Loop (HITL), Feedback, Evaluation, and Decision Executor subsystems. Learn how these elements work together to start building an AI platform.

37

13 Comments

The latest Weekly UpdAIte: Microsoft and Apple forego board seats at OpenAI AMD acquires Silo AI Anthropic research on interpretable LLMs A blast from the past: Marvin Minsky Here's a short clip, full episode is linked in the comments. If you want a comedic twist on your weekly AI and Data news, subscribe! #ai #analytics #weeklyupdAIte

10

1 Comment

Unpacking Vinod Khosla's Future AI predictions at the Imagination in Action summit last week at MIT Media Lab. ✨ Evolution of Bipedal Robots Robots are evolving from static, rule-based imitation systems to dynamic, continuously learning systems. The ability to self-learn and improve from their experiences and surroundings will become integral to robotic systems. This development will empower robots to adjust to new circumstances and respond effectively to environmental changes. ✨ Paradigm Shift in Software Throughout computing history, we have been optimizing how we interact with computers and machines. Traditionally, through programming languages, operating systems, and application software, we've created structured systems designed to perform specific tasks. However, the future of software, enhanced by AI, will shift towards adapting more fluidly to human needs. We are already seeing the beginnings of this transformation in the functionality of chatbots and concepts such as 'prompt inversion' or 'reverse prompting', where software interacts with users to better understand and achieve their desired outcomes. This shift will move us from the rigidity of traditional software to a more adaptable and user-centered software paradigm. ✨ Internet of the AI Age AI agents are set to become the primary users of the internet, necessitating a redesign of web services and infrastructure to accommodate AI-first interactions. Within the next five years, AI agents are expected to dominate internet usage, transforming how web services are designed and information is processed. ✨ Natural Language Programming Programming is becoming more accessible thanks to natural language processing, effectively democratizing programming skills. This will lower barriers to entry, making programming nearly cost-free and allowing more rapid realization of ideas—an excellent opportunity for innovators, founders and creators. ✨ Super Intelligent AI Systems We are beginning to see AI systems assisting in training other AI systems, with early steps involving AI-generated data. This is akin to the initial stages of AlphaGo, which didn't achieve superhuman capabilities until it began self-play. Future AI systems will use synthetic data and develop their own reward models, engaging in self-play and self-improvement. ✨ Universal Access to Expertise AI will soon provide widespread access to educational and healthcare expertise at little to no cost. Advances in AI will address issues like hallucination and ungrounded responses, ensuring an abundance of reliable, expert advice. ✨ Continuous Learning As technology continues to evolve rapidly, the ability to continuously learn and adapt will be crucial. Embracing a mindset focused on lifelong learning and a diverse skill set will be essential for navigating the future successfully. The emphasis is on learning to learn and diversifying one's learning approaches. John Werner, Paul Baier #FutureofAI #MITForge2024 #AITrends #IIA

33

1 Comment

🚀 𝗥𝗲𝘃𝗼𝗹𝘂𝘁𝗶𝗼𝗻𝗶𝘇𝗶𝗻𝗴 𝗔𝗜: 𝗔𝗳𝗳𝗼𝗿𝗱𝗮𝗯𝗹𝗲 𝗚𝗣𝗨 𝗖𝗹𝗼𝘂𝗱 𝗖𝗼𝗺𝗽𝘂𝘁𝗶𝗻𝗴 𝗶𝘀 𝗛𝗲𝗿𝗲! 🖥️👁️ 🌟 The future of cost-effective GPU cloud computing has arrived, transforming how we train and deploy ML and computer vision models. Check out these game-changing startups offering incredibly affordable GPU solutions: 1️⃣ RunPod 🏃‍♂️💨 Empowering developers with on-demand GPU instances. RunPod's flexible pricing and user-friendly interface make high-performance computing accessible to all. 2️⃣ Vast.ai 🌊🧠 Pioneering a unique marketplace for GPU rentals. Vast.ai connects GPU owners with AI researchers, creating a win-win ecosystem for affordable compute power. 3️⃣ Lambda ☁️🔬 Offering turnkey GPU cloud solutions tailored for AI workloads. Lambda Labs' pre-configured environments streamline the development process for data scientists. 4️⃣ Paperspace 📄💻 Providing a seamless platform for GPU-accelerated applications. Paperspace's intuitive tools and collaborative features make it a favorite among AI teams. Paperspace is now part of DigitalOcean 5️⃣ CoreWeave 🔧☁️ Specializing in large-scale GPU deployments for enterprise clients. Coreweave's cutting-edge infrastructure supports the most demanding AI and rendering tasks. 6️⃣ Genesis Cloud ☁️🌱 Focusing on sustainability in GPU cloud computing. Genesis Cloud combines eco-friendly practices with high-performance GPU instances for guilt-free AI development. 7️⃣ FluidStack💧🥞 Harnessing idle GPUs worldwide to create a distributed supercomputer. Fluidstack's innovative approach offers cost-effective solutions for resource-intensive tasks. 8️⃣ Vultr 🚀💪 Delivering GPU-powered virtual machines with global reach. Vultr's extensive network of data centers ensures low-latency access to GPU resources worldwide. 9️⃣ JarvisLabs.ai 🤖🧪 Crafting AI-optimized cloud environments for researchers. Jarvis Labs' specialized tools and support accelerate breakthroughs in machine learning. 🔟 Banana 🍌💻 Simplifying ML model deployment with serverless GPU infrastructure. Banana.dev's user-friendly platform makes AI accessible to developers of all skill levels. These innovative companies are democratizing high-performance GPU computing, making it available to everyone from solo developers to large enterprises! 💪💼With prices that'll make your wallet smile 😄💰 and performance that'll blow your mind 🤯 #AIML #ComputerVision #AIStartups #CloudComputing #gpu #GPUPower Image credit : nvidia.com

3

While speaking at the GenAI Summit (GenAIx- Helping you to 10x your company with AI) last week, Rodrigo Liang, CEO and co-founder of SambaNova Systems, spoke with ABC7 News Bay Area, discussing how San Francisco has become the center of the AI boom and what the future of AI looks like.    Rodrigo says the Bay Area's AI innovation is incredible, and it is helping redefine and reshape San Francisco. Check out the full story below.    #AI #SanFrancisco

5

1 Comment

🏁 Bay Area vs. Los Angeles rivalry Exactly 3 years ago we moved to the Bay Area from LA, where I spent most of my life. Since then many have asked me what I feel the differences are. My general breakdown: 👩🏽‍💻 Technology Bay Area > LA - There is no place like the Bay for the sheer quantity and quality of big tech companies, startups, VCs, academic institutions, and talent. LA has Silicon Beach, which focuses mostly on media, entertainment, and consumer tech, while the Bay ecosystem is more diverse and dense. Some claim Silicon Valley has lost its appeal, I’d say the deep-rooted technology ecosystem, going back nearly a century with Bell Labs, HP, Shockley, etc., is not easily replicable. 🌴 Lifestyle Bay Area > LA - As my neighbor accurately put it: “In LA, success is measured by name-brand bags, luxury cars, and big houses. In the Bay, it's by how many exits you've had and the number of boards you're on." Both are ego driven, but I’d take the latter. LA > Bay Area - The cost of living in the Bay is infamously insane. (Cost of living in San Francisco is 23% higher than the state average and 71% higher than the national average. Housing is 187% more expensive than the U.S. average, while utilities are about 42% pricier.) LA > Bay Area - LA takes the crown for its diverse food scene, great restaurants, and affordability. Especially with Asian cuisine, it's hard to compete with the options in the San Gabriel Valley. LA > Bay Area - Style and cultural diversity are stronger in LA. Even though I ❤️ close to 30% of population in the Bay are AAPIs LA > Bay Area - grew up in Purple & Gold Bay Area > LA - Traffic, obvious Bay Area = LA - Weather, which is why we pay the infamous California "weather tax" 👥 People Bay Area > LA - Bay Areafolks are generally much nicer and more courteous than Angelenos. Bay Area > LA - The Bay has the most passionate people I've met when it comes to community involvement, activism, and social/political issues. Ingrained in the DNA and its cultural history. Which city is better? 🤼 “Let me serenade the streets of L.A. From Oakland to Sac-town, the Bay Area and back down Cali is where they put they mack down, give me love!” - Tupac

55

22 Comments

The keynote speech by Trevor Cai from #OpenAI at #HotChips 2024 was interesting. Trevor's presentation reiterated the ongoing hypothesis at OpenAI—that the AI models must continue to grow to achieve superior performance, with no apparent end in sight... He bolstered this argument using a log-log power law plot, which essentially plots the relationship between compute (in FLOPs) and model performance (i.e., loss) on a logarithmic scale. This approach has enabled OpenAI to accurately predict the performance of models like GPT-4 by curve-fitting the smaller models on the plot, thus demonstrating the scalability of this method. Trevor then drew a fascinating parallel to Moore's Law, which famously predicted the doubling of transistors on chips every two years. This prediction held true for decades, fueling massive advancements in semiconductor technology. He suggested that similar underlying principles might sustain the exponential growth of AI models for a significant time, though the mechanisms might evolve. However, Trevor notes that such growth isn't without its challenges. He highlighted several critical issues, including the need for advanced data centers, enhanced reliability, availability, and serviceability (RAS) systems, and more sophisticated power management strategies. The massive scale of synchronized training introduces vulnerabilities like power draw jitters and single-point failures that can cause widespread disruptions. If synchronized training causes so many challenges, I wonder if the research community is focusing on alternative algorithms for LLMs that don't require gradient synchronization after each iteration.? 🤔 💡 Trevor points out that as AI models became more intelligent, the demand for inference has grown significantly. He notes that while training infrastructure can be planned and scaled, inference capacity must be more responsive to actual usage and demands, making planning more challenging and dynamic. The technology and economics of inference will continue to evolve. While the conference is about hot chips, he has no specific recommendations on GPU/accelerator architectures, whether inference could benefit from custom hardware, etc. Overall, it is evident that openAI will continue to grow the models and datasets it trained on for each new release of GPT and will continue to challenge the AI/GPU and networking hardware industry to support this with technological advancements. At some point in the future, more than half the data on the Internet will have originated from the LLM models. At that point, how would the log-log plot look? 🤔 Wouldn’t models trained on their self-generated data collapse, and the losses start to grow? He acknowledges the problem in Q&A and says they are "investigating" techniques to overcome this…. Any thoughts? #AI #LLMs #datacenters #junipernetworks

83

8 Comments

This is something every #CEO should learn to do. They think #firing is the #smartest way to manage #business or #startup or #corporate or #company.

6

1 Comment

I haven’t tried this model fine tune and due to issues with gguf quant fine tune conversion, still working on adjustments. That being said llama3 and mixtral are my go to models. I’m using a context tuned version to 32k. Just not practically to run a 70b parameter local (yet) #llm #llama3 #genai #models

4

1 Comment

This is insane! NVIDIA co-founder & CEO Jensen Huang is now worth more than the entire market cap of Intel Corporation 😳 Jensen could theoretically buy #Intel in full and still have $19 billion to spare. Say for another #AI startup 🤖 The crazy part? Less than 5 years ago, both tech giants were worth about the same. But then #Nvidia decided to shift its strategic focus to AI and data centers, and the rest is history. It’s truly Jensen Huang's world, and we're just living in it. P.S. here's a list of the top 100 AI startups & pitch decks that have helped them raise $28B in 2024 already: https://lnkd.in/dZaQYZy7 🚀

7

2 Comments

🌟 Exciting News in AI/ML! 🌟 🚀 #Google’s #Gemini 1.5 Pro Doubles its Context Window to 2 Million Tokens 🚀 The latest update to Gemini 1.5 Pro has expanded its context window from 1 million to a staggering 2 million tokens! This leap is set to redefine possibilities in natural language processing. 🔍 What is a Context Window? In NLP, a context window is the amount of text the model can analyze at once. A larger context window means the model can generate more coherent and contextually relevant content, especially for long documents. 📜➡️📚 🚀 Why the Big Deal? 1️⃣ Enhanced Understanding: With a 2 million token context window, Gemini 1.5 Pro can maintain context over much longer texts, improving the quality and coherence of its outputs. 📖✨ 2️⃣ Tackling Complex Problems: This expansion is a boon for detailed analysis and reasoning across large documents, aiding in fields like legal analysis, academic research, and technical documentation. 🏛️🧠🔍 3️⃣ Boosting AI/ML Applications: Developers and researchers can now build more sophisticated models for tasks like summarization, translation, and sentiment analysis over extended content. 🛠️📊🌐 4️⃣ Real-World Impact: Industries like healthcare, finance, and automotive can benefit immensely from the ability to process and analyze large volumes of data accurately, leading to better decision-making and advanced predictive capabilities. 🚑💼🚗 🌟 Practical Implications 🔸 Technical Documentation: Engineers and technical writers can now create comprehensive, contextually accurate documentation with AI assistance. 📚🛠️💡 🔸 Legal and Compliance: Legal professionals can analyze extensive legal documents and contracts more effectively, identifying nuances and ensuring compliance. ⚖️📜✔️ 🔸 Scientific Research: Researchers can process and analyze vast amounts of scientific literature, enabling thorough literature reviews and meta-analyses. 🔬📘🧪 🔍 What Does a 10 Million Token Context Window Mean? Imagine the possibilities when the context window expands to 10 million tokens: 🔘 All of the emails you've sent in the past year 🔘All of the text messages you’ll send in a lifetime 🔘About 500K lines of code 🔘About 11 hours of video 🔘About 80 average-length English novels 🔘Full text of all US federal laws and regulations 🔘Main text of all papers published at NeurIPS this year 🔘Transcripts of over 2K podcast episodes 🔘About 20K news articles 🔘Audio for over 2K songs 🔘Earnings scripts for about 2K companies 🔘All of the calendars at a small to medium-sized startup 🌐 The Future of Infinite Context Windows Google CEO Sundar Pichai envisions a future with infinite context windows, where AI can handle vast amounts of information seamlessly, making it possible to perform incredibly complex tasks with unparalleled precision and efficiency. ✨Future says - Here I come ! #AI #ArtificialIntelligence #MachineLearning #GoogleGemini #NaturalLanguageProcessing #NLP #LLM

6

📚 A new benchmark for AI model performance tl;dr: A new benchmark called GSM1k is challenging the way we evaluate the performance of Large Language Models (LLMs). By focusing on grade school arithmetic, it highlights a significant gap in accuracy and dataset contamination across various LLMs. * * * 👩‍🏫 When I was in grade school, my teachers would often say, "You won't always have a calculator in your pocket." They were right; we now have something even better - our smartphones. But as technology leaps forward, there's a growing concern: are our digital companions getting a little too smart for their own good? A new benchmark called Grade School Math 1000 (GSM1k), designed to assess the mathematical reasoning abilities of large language models is shaking the foundations of what we thought we knew about LLMs, revealing a troubling truth - our algorithms might be getting a bit too clever for their own good. A study, which evaluated various LLMs on the GSM1k benchmark, uncovered insights that challenge the current narrative around these models: 1/ Up to a 13% drop in accuracy: Compared to traditional benchmarks, some LLMs exhibited a significant decrease in performance on GSM1k compared to other commonly used benchmarks, with an average drop of 6%. 2/ Systematic overfitting: Several models showed signs of overfitting on GSM1k, with Gemini/GPT/Claude displaying minimal overfitting. Overfitting is a common issue in machine learning and statistical modeling where a model is trained too well on its training data, to the point where it captures noise or random fluctuations in the data rather than the actual underlying patterns 3/ Data Contamination: There was a direct correlation between a model's exposure to the larger GSM8k data and its performance gap on GSM1k, suggesting partial memorization of benchmark problems by many models. So LLMs "learn" the responses to ace specific tests, but don't really have the reasoning to back those answers. The authors plan to run recurring evaluations and update results, and will eventually open source the GSM1k benchmark. 🚀 It’s important, when looking at benchmarks (like the leaderboard in HuggingFace) to remember that benchmarks and tests can be gamed (anyone remembers the dieselgate?). I guess with the race to be at the top, we've just discovered another area where AI Ethics will come in handy. #AI #LLM #GSM1k #MachineLearning #DataContamination Image credit: Hugh Zhang et al.

9

1 Comment

Here's a calm, considered opinion about the the humble level of machine "understanding" we're able to achieve now: "Nvidia chief executive Jensen Huang was asked during an interview with CBS News’ 60 Minutes what he thinks will happen to workers when AI and robots can do more human jobs. “'The workers work for companies. And so companies, when they become more productive, earnings increase,' Huang said in the interview that aired Sunday. 'I’ve never seen one company that had earnings increase and not hire more people.' “'I believe that you still want human in the loop, because we have good judgment, because there are circumstances that the machines are not — just not going to understand,' Huang added." Workplace situations have lots of complexity and nuance. These weekly declarations in the media that companies won't need to hire doesn't make sense. But what is changing is how companies hire, who's hiring, what the roles are, how much the pay is, how much is demanded of workers, and how much a dollar buys. Here companies who are well-positioned have considerable leverage. And the main consequence of digitization is a massive power shift, away from rank-and-file workers, away from traditional companies, away from powerful governments, and away from the middle class and income equality, away from equitable labor arrangements. Most workers will have to fight for everything they get from scratch. They'll have to be incredibly agile and adept at organizing and bargaining, and they'll have to find new ways to hunt for new opportunities. The opportunity is out there, and capital is plentiful. But the capital isn't savvy or patient...or fair. And society at large is in some ways suffering a reset. In some ways, we're back to a form of digital feudalism. In another, it's a second, much more chaotic and ungovernable Gilded Age.

9

1 Comment

Apple Mac Studio M2 Ultra approx. 40% as fast as Nvidia 4090 on TRAINING using MLX. Very cool, as MLX feels very fast and the Macs have HUGE unified memory with M series chips. vast.ai had an article benchmarking the 4090 at 1 Sample/sec with Batch Size = 8 on Llama-2-7B-hf quant 4 bit, on a particular guanaco data set. I was able to reproduce the test with exact data set, model, 4bit, and batch size. Nvidia GPUs are great too, but I'm really love my mac. Like my mac pro is quiet compared to my friend's alienware with a 3090 - night and day! This Ultra Studio is the beefiest one, approx. $7K 196GB unified ram, 76 GPU cores, M2 chip. I recommend the 2TB for any data science machine these days. Contrast that to my Macbook Pro Max with 96GB and 38 GPU cores, M2, which seems to run around 1/2 the speed of the Ultra, which puts this LAPTOP at approx. $4K and 20% the speed of a 4090 DESKTOP but portable. And my Macbook is approx 55% the speed vs a 4090 laptop - but a lot quieter! One big tip - you need to get "Macs Fan Control" - it's free and just set it to "Full Blast" to keep your machine cool - still quiet on that setting. Tip: You can edit the source code of the MLX lora.py and put a sleep(.01) or sleep(.001) just to give your system a few milliseconds on each iteration to chill. I tweak the sleep just enough to lower from full 100% blue bars to like 95% and all is well. No matter if I trained quatized 4bit or full 16 float, the model used around 145 GB. I had probably 30 go of other stuff running But I do find it interesting that the MLX didn't seem to use less memory in QLoRA vs LoRA. So, I'd say they have some room for improving that, but with all this ram, you're just better off training in Float 16 full precision. You can always quantize it later, and you can even use MLX_LM to convert to GGUF, which can be ran outside the MLX on normal servers. One last thought, even if you train your final model on Nvidia cuda cores, having this machine locally accelerates my learning and experiments a lot. Remember, I'm able to generate 1K to 10K synthetic data points, and then fine tune in a matter of a couple of hours. I can even try things in less than 15 minutes and see results. My laptop feels fast to me, and is so convenient, and recommend the M series silicon for all devs. I'd trust my Macbook Pro to run all night as long as I put a sleep in the lora.py to chill it out to 80%. The Mac Studio Ultra feels super fast, and I really recommend one "mothership" Mac Studio in your corporate office for people to do their bigger experiments on - as the fan cooling is sufficient to run continuously at 90%. Remember this is training and it’s doing great - but I love running Mac’s for Inference (generating locally) and running any size model Hope these tips helped someone decide if MLX training on mac is what they want. I'm looking forward to what Apple does with MLX!

10

Interested in #GenAI and its impact on #softwaredevelopment? Read EQengineered's Principal Technical Consultant/Architect Ed Lyons' recent essay. #techinsights #softwareengineering #digitaltransformation #GenAI #LLMs #EQengineered

7