Vishal M.

The 7th workshop on Neural Scaling Laws is about to start: https://lnkd.in/gQSB5k8v Scaling laws, emergency, alignment, transfer learning, with a focused track on multilingual models. Looking forward to kick it off with Irina Rish and discuss all these topics with Yuhai Tu, Paul Bogdan, Hailey Schoelkopf, Darshil Doshi, Rio Yokota, Neha Sengupta, Preslav Nakov, Tatiana Shavrina, PhD, Yishi Xu. Join in person or virtually! #icml2024 #ai

37

🔍 Key Papers to Demystify How GPT-o1 Works 1) Training Language Models to Self-Correct via Reinforcement Learning: https://lnkd.in/gwgdE5fb 2) STaR: Bootstrapping Reasoning With Reasoning: https://lnkd.in/gRdir7PP 3) Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking: https://lnkd.in/g47fqB7p 4) V-STaR: Training Verifiers for Self-Taught Reasoners https://lnkd.in/gGRRyBrF 5) Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters: https://lnkd.in/g9atmpvC 6) Chain of Thought Empowers Transformers to Solve Inherently Serial Problems: https://lnkd.in/gCyMncsf 7) Large Language Monkeys: Scaling Inference Compute with Repeated Sampling https://lnkd.in/ghy2r5nH 8) An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models: https://lnkd.in/gRHU2xyU 💡 Stay Tuned! I will continually update this post with more insights and a special focus on the "how" aspects of these papers.

18

2 Comments

RAG Foundry, an open-source framework, tackles the complexities of Retrieval-Augmented Generation (RAG) systems by effectively merging data creation, training, inference, and evaluation. Discover more about this integrated solution here: https://lnkd.in/g2Z2FDj3

19

1 Comment

Semantic chunking vs. fixed-size chunking: Which delivers better results in Retrieval-Augmented Generation (#RAG) systems? 🤔 In our latest blog, Renyi Qu and Forrest Sheng Bao, engineers at Vectara, challenge the hype around semantic chunking. Through systematic evaluation across RAG tasks, they uncover surprising insights into performance, computational costs, and real-world practicality. 🔍 Key Takeaways: ✅ Fixed-size chunking often outperforms semantic chunking in real-world scenarios. ✅ Computational costs of semantic chunking don’t always justify the benefits. ✅ Focus on embedding quality for bigger gains in RAG performance. 👉Check out the full blog here: https://lnkd.in/gZ9M7eYD

15

Get a deeper look into the results submitted by NVIDIA in #MLPerf Inference v4.1, highlighted by the first ever submission using the NVIDIA Blackwell architecture, increased performance using the NVIDIA H200 Tensor Core GPU, and more.

1

Organizations have insufficient technical capabilities for the speed and scale of their ML model implementations. My colleague Sumit Agarwal just published a very helpful research with a capability view and sample vendors for defining a modular, flexible and scalable MLOps solution. https://lnkd.in/gnB_T9bq #MLOps #LLMOps #AI

149

17 Comments

Interesting analysis from Fundrise about potential value of AI Revolution, based on historical precedents of previous disruptive technology waves (PC, Internet, Mobile and Cloud). https://lnkd.in/gAHRv9eK Per Benjamin Miller, his analysis revealed a consistent "Rule of 3" pattern of ~3X-3.5X increase in value creation from one technology wave to another, which leads to $20 trillion estimated value creation from current AI wave. The analysis emphasizes that "Nvidia or GPUs are producing more compute faster than Moore's law" to highlight that their estimates might be conservative. In my opinion, one of the main bottlenecks in AI scaling is GARGANTUAN POWER NEEDS (and not only physical compute units) required to operate enormous expected scale of AI applications. It takes around 10 years to fully grasp the impact of major technology wave - exciting milestone to watch for! #GenerativeAI #largelanguagemodels #artificialintelligence #airevolution

62

8 Comments

The game of large language models (LLMs) got a major plot twist! Meta's unveiling of Llama, a family of AI models pushing the boundaries of parameter size and processing power, is a game-changer. Why it matters? Llama boasts a staggering 405 billion parameter model, dwarfing previous heavyweights like Google's LaMDA and OpenAI's Jurassic-1 Jumbo. This sheer size unlocks unprecedented capabilities in: -> From scientific papers to philosophical treatises, Llama can digest vast amounts of data, leading to deeper comprehension. -> Imagine writing styles that perfectly mimic different authors or create compelling narratives on the fly. -> From machine translation to chatbot interactions, Llama holds the potential to transform how humans and machines communicate. -> Llama isn't just about brawn. Meta stresses its commitment to open-source access and responsible AI development. This fosters collaboration and helps mitigate potential biases often associated with large datasets. Meta's ambitious project is a harbinger of the future, where AI can not only process information but truly understand it. Imagine personalized education tailored to individual learning styles, or AI assistants that anticipate your needs before you even ask. https://lnkd.in/dYn_cTjB #MetaLlama #ai #LLMs #futureoftechnology #naturallanguageprocessing #opensource

6

"A New Age of Intelligence: India’s Path to AI Leadership" The recent discussion between Mukesh Ambani and Jensen Huang on India’s AI future marks a transformative step for the country’s tech landscape. Their conversation outlined a vision for India as a global AI leader, with Reliance and NVIDIA collaborating to build robust AI infrastructure that democratizes access to intelligence. A key highlight was the role of open-source AI in creating an AI flywheel—a self-sustaining ecosystem fueled by domestic data, AI models, and a vast population of users. As Ambani noted, open-source frameworks like Meta’s LLaMA 3 empower developers to build on foundational models, reducing entry barriers and making AI accessible for all. This approach aligns with Ambani’s mission to replicate Jio’s success by delivering affordable, impactful AI solutions to the 1.4 billion people of India. Reflecting on Ambani’s vision, I see both opportunity and challenge for enterprises. As companies look to manage run costs, open-source AI appears promising. Yet the definition of “open source” in AI is still evolving and, unlike traditional software, requires much more than just source code. To truly “tinker” with AI models, developers need access to a complex ecosystem: trained models, preprocessing code, training data, model architecture, and more. This lack of standardization in open-source AI is both a strength and a hurdle, as enterprises must weigh flexibility against the practical costs of managing these resources. Despite these challenges, open-source AI and enterprise-grade AI together create powerful potential. With NVIDIA’s infrastructure and Reliance’s commitment to accessibility, India’s AI flywheel can drive sustainable growth across the economy, creating an ecosystem that not only supports enterprise innovation but elevates everyday lives. The ambition Ambani and Huang outlined is more than a blueprint for India; it’s a roadmap for making AI scalable, cost-effective, and inclusive on a global scale. Exciting times ahead for India's tech ecosystem! https://lnkd.in/gBBHjq6A #GenerativeAI #OpenSourceAI #AIFlywheel #DigitalTransformation #NVIDIA #Reliance #AIIndia

32

7 Comments

Hallucination is Inevitable in LLMs An important new paper out that shows that even with perfect data and an idealized architecture “LLMs will always hallucinate”. I suspect that this will over time be linked to Gödel’s Incompleteness Theorem [1] and Turing’s Halting Problem [2]. Every technology has trade-offs. We love LLMs but they need to be used in the right way. [1] Good pop science depiction in Oppenheimer (Gödel was Einstein’s walking companion) or also see Rebecca Goldstein’s book Incompleteness [2] Turing is depicted in The Imitation Game and also the progenitor of the famed Turing Test.

36

NVIDIA NIM™ — inference microservices that provide models as optimized containers — to deploy on clouds, data centers or workstations, giving them the ability to easily build generative AI applications for copilots, chatbots and more, in minutes rather than weeks. Available to download today! #TDSYNNEX #NVIDIA #DESTINATIONAI

14

1 Comment

Just published my latest article: "Transforming Business Operations with LLMs: A Path to a Production Ecosystem." a breakdown of the key components necessary for integrating LLMs into core business operations, including Human In The Loop (HITL), Feedback, Evaluation, and Decision Executor subsystems. Learn how these elements work together to start building an AI platform.

37

13 Comments

Using generative AI, NVIDIA operations built an AI Planner agent, developed on NVIDIA Inference Microservices (NIM). The agent leverages LLM, NeMo Retriever and CuOpt NIM to reduce re-planning time from hours to just seconds. https://nvda.ws/4bIfS2T

18

Avoid taking leadership advice from extremely lucky companies.

7

1 Comment

If you think compute scaling will plateau, think again. We are in the baby states of inference time compute. VLLMs coupled with computer tools, reward learning and RL can leverage orders of magnitude more inference time compute than what we have seen with foundation models. Here’s a concrete training architecture (replace \pi with a large pre-trained vLLM and add it to the reward learner too, \R is the space of all computer tools)

21

1 Comment

Good detailed overview of Retrieval-Augmented Generation (RAG), one of leading techniques to improve accuracy of LLMs. When developing a production-grade LLM-based system, it's essential to understand the advantages and limitations of different techniques (RAG, prompt engineering, fine-tuning etc), across training and serving/inference phases, to improve LLMs for specific applications (see my previous post https://lnkd.in/giuJdJJt) A balanced approach to LLM applications that takes into account accuracy, latency, and cost is crucial for successful productization at scale. #artificialintelligence #largelanguagemodels #GenerativeAI #airevolution

10

#TodayinVC Spotlight: EdgeRunner AI Pioneering Generative AI for the Edge EdgeRunner AI was founded in 2024 by Tyler Xuan Saltsman and Colton Malkerson, with a mission "To Build Generative AI for the edge that is safe, secure, and transparent." EdgeRunner AI operates in one of the most popular areas of technology today: generative AI. According to a report from Bloomberg Intelligence, generative AI had a market size of $40 billion in 2022, with the potential for the AI industry to reach $1.3 trillion by 2032. This rapidly growing market presents opportunities for many competitors to enter, allowing companies to specialize and carve out niches within the industry. EdgeRunner AI has made a swift start, securing a successful $5.5 million seed round of investment from Four Rivers Group and Madrona Ventures. The company is focusing on several key areas: - Defense: Enhancing operational comprehension for those engaged in warfare. - Manufacturing: Aiding with maintenance and improving performance quality. - Transportation: Using data to overcome logistical difficulties while ensuring safety and security. - Other Sectors: Providing services in Oil & Gas, Healthcare, and the Public Sector. Venture Capital Backing: One of EdgeRunner AI’s investors, Four Rivers Group, is a global venture capital firm based in San Francisco, California, with a focus on technology. Their strategy of investing at the early stages of a company’s life aligns perfectly with their seed fund support of EdgeRunner AI. Madrona Ventures, founded in Seattle, WA in 1995, also contributed to the seed round. Similar to Four Rivers Group, Madrona Ventures focuses on early investments and supports the long-term success of tech companies, guiding them toward M&A or IPO. Looking Ahead: EdgeRunner AI is entering the generative AI market at an exciting time. With the backing of their investors and a specialization in security and efficiency across various industries, it will be a company to keep an eye on. Best of luck to EdgeRunner AI! If you would like your company to be featured in our VC company spotlight, make sure to like and comment on Today in VC daily post! For more information on #startups and #venturecapital news follow Today in VC!

20