Jerry Held

AI leaders have a crucial chance to address risks now and strengthen governance practices before playing catch-up to regulations, notes Kjell Carlsson, Ph.D. of Domino Data Lab. In a recent VentureBeat article, he shares his perspective on California's AI bill veto and why proactive governance is key to the future of AI innovation. Learn more about this addressing this opportunity: https://gag.gl/4WCTkP #AI #AIGovernance #AIInnovation #DataScience

1

Organizations have insufficient technical capabilities for the speed and scale of their ML model implementations. My colleague Sumit Agarwal just published a very helpful research with a capability view and sample vendors for defining a modular, flexible and scalable MLOps solution. https://lnkd.in/gnB_T9bq #MLOps #LLMOps #AI

149

17 Comments

Some freshness in the LLMs arena! LLM can be accurate if you follow SOTA practices!

2

Using generative AI, NVIDIA operations built an AI Planner agent, developed on NVIDIA Inference Microservices (NIM). The agent leverages LLM, NeMo Retriever and CuOpt NIM to reduce re-planning time from hours to just seconds. https://nvda.ws/4bIfS2T

18

Just published my latest article: "Transforming Business Operations with LLMs: A Path to a Production Ecosystem." a breakdown of the key components necessary for integrating LLMs into core business operations, including Human In The Loop (HITL), Feedback, Evaluation, and Decision Executor subsystems. Learn how these elements work together to start building an AI platform.

37

13 Comments

AI next frontier is possibly developing and scaling the reasoning layer, where AI systems possess hard problem-solving and higher order cognitive intelligence. Came across this good article from Sequoia.

20

Had the privilege to join a Fireside Chat with Sudivya from the Google Cloud team at IIMA Ventures. Key takeaways: LLM democratization: Making custom LLMs accessible to non-tech users. AI in action: Impactful use cases like RAG and domain-specific automation. Build vs. leverage: Pre-trained models offer speed and flexibility; fine-tuning ensures precision. Tech challenges: Managing data at scale, ensuring compliance, and balancing the economics of GPU-intensive workloads remain complex challenges. Balancing data efficiency with cost-effective GPU scaling via distilled LLMs. Why GCP?: Powerful AI tools, stellar startup support, and global scalability. #DataNeuron #AI #FiresideChat #LLM #GoogleCloud #Innovation #Startups

41

Success with generative AI can only be unlocked by strategically using an organization’s unstructured data. This is giving rise to an emerging unstructured data stack, which consists of three crucial components: 1️⃣ data extraction and ingestion 2️⃣ data processing 3️⃣ data management In this post, I share more context about the solutions in these spaces and how they are used: LlamaIndex Reducto Tensorlake unstructured.io Vectorize Eventual Databricks Polars Anyscale Pandas Onehouse Tabular (now part of Databricks) Chroma Pinecone Supabase Spiral LanceDB MongoDB Stately Neo4j FalkorDB TigerGraph https://lnkd.in/dT74Yj3C

117

11 Comments

Researching the ROI of LLM-based applications? Here is an excellent article by Prof. Rama Ramakrishnan at MIT Sloan School of Management. He wrote: "Gartner analyst Philip Walsh has pointed out that with tools costing roughly $20 per month per user, even a 5% productivity gain means that companies are “effectively adding another developer to the team for $400 a month.” "LLMs are a mixed blessing. They have immense potential to increase the efficiency of business processes, but they are subject to numerous weaknesses as well." cautions Prof Ramakrishnan. "It’s vital to have a robust and efficient way to evaluate an LLM-powered application’s effectiveness, identify failure modes, and formulate strategies for iterative improvement, both during development and after launch." I attended the GenAI Summit (Nov 1 - 3) last weekend at the Santa Clara Convention Center. I met RagaAI Inc at their booth, and I was impressed with how they are addressing the challenges mentioned by Professor Ramakrishnan in this article, using hallucination detection, latency and cost monitoring, prompt and response monitoring, and support for model selection (e.g., Auto-reroute of prompt to best model at the backend that can be OpenAI's API or on- prep deployed llama) and fallback strategy implementation. https://lnkd.in/gS5CUMBU #HallucinationDetection #LLMChallenges #GenAISummit

22

1 Comment

AI leaders have a crucial chance to address risks now and strengthen governance practices before playing catch-up to regulations, notes Kjell Carlsson, Ph.D. of Domino Data Lab. In a recent VentureBeat article, he shares his perspective on California's AI bill veto and why proactive governance is key to the future of AI innovation. Learn more about this addressing this opportunity: https://gag.gl/4WCTkP #AI #AIGovernance #AIInnovation #DataScience

1

Large language models are wonderful at ingesting large amounts of content & summarizing. Uploading an academic paper and I can pester it with an infinite list of questions & it will respond with equally infinite patience. In comparing the two most recent Microsoft earnings calls, Claude highlighted: - faster than expected Azure growth (29% vs 27%) - AI contributing 8% of Azure revenue up from 7% - higher CapEx spending & greater capacity constraints for data centers - much better commercial bookings growth : 17% vs 13% Excellent analysis. Benn Stancil described LLMs as great averagers of information. But what if I don’t want the average? If I seek the data point two standard deviations out? If I’m channeling my inner Anthony Bourdain & I’m seeking the fermented shark or cobra heart within my query? At Google, we ranked web pages & ads with many signals. An engineer taught me that of them found rarity of a word across a set of documents. For example, if there’s a document containing 10 instances of the word ghotiacross a collection of documents, where the average document has 0 ghotis, that document is likely the best to answer a search about fish. I haven’t found a way to goad an LLM to produce the rare result. Why did one user decide to use a particular piece of software when 10 others did not? Maybe I haven’t learned how to prompt an LLM well. Ideogram launched a feature called Magic Prompt that expands a basic prompt into richer instructions that elicit better results, marrying the language of a user & a computer. A user might write : A boy with a dog in a park. Magic Prompt replies : A heartwarming scene of a young boy playing with his happy, wagging dog in a lush, green park. The boy has a playful smile on his face while he tosses a bright yellow ball for his furry friend to fetch. The park is filled with beautiful flowers and trees, creating a serene atmosphere for this bonding moment between boy and dog. Magic Prompt surprised me in three ways. First, unlike Google search where terse queries tend to work very well, LLMs relish verbosity. Less Hemingway & more Faulkner. Second, maybe there’s a prompt for rarity. If you know what it is, please share with the rarity shibboleth. Third, I wonder what other prompt techniques are out there that have yet to be discovered to wrest the best from an LLM. This technique is called TF-IDF. It means Term Frequency Inverse Document Frequency.

21

6 Comments

Another great article by Pete Flint, Anna Piñol and the NFX team: https://lnkd.in/g8XKjGdX. One big takeaway is AI-first B2B software will necessarily expand and fuse with knowledge workers, which not only creates tremendous opportunities for new AI-first companies (“AI Colleagues” or “AI Vendors”), but also necessitates every organization to adopt AI to remain competitive (and more frankly, to remain alive). Favorite quote of the article, “In the long term, software with a soul may endure far beyond software that seeks to automate repetitive tasks." At Glowing.io, we strongly encourage (and moving towards requiring) everyone on our team to leverage AI tools in a growing percentage of their work in order to free up more time on creating impactful and delightful solutions for our customers.

8

The Stanford Human-Centered AI Report is out. This is the 7th year that Stanford has published this report and it is a must-read if you're trying to keep track of the developments in this field. https://lnkd.in/gwsHtzsq

2

The term “AI Hallucination” describes instances where LLMs generate incorrect, misleading, or nonsensical results. However, recent discussions in psychology suggest that “confabulation” is a more accurate term (https://lnkd.in/gvYwBGCk). While hallucination implies a false sensory experience, confabulation refers to the creation of false memories—more akin to what AI does. Understanding these nuances helps us harness AI’s potential while mitigating its shortcomings. At GumGum, we thoughtfully evaluate state-of-the-art technology with rigorous quantitative testing and meticulous manual spot-checking before incorporating it into our existing workflows. This ensures our datasets are curated to remove anomalies and yield top-performing models.

5

1 Comment

Turns out #GenAI takes some basic common sense: (1) use the right Model for the right Use-Case (2) use Open models to leverage the Open eco-system (3) run models - where they should run - based on data availability, proximity, latency, operational efficiency, etc... On Tuesday, July 23, #Meta announced the launch of the Llama 3.1 collection of multilingual large language models (#LLMs). The instruction-tuned Llama 3.1-405B, which figures to be the largest and most powerful open-source language model available today and competitive with the best proprietary models on the market. So, if you have a use-case that benefits from this large scale OPEN model - it is now available on IBM® #watsonx.ai - where it can be deployed on the IBM cloud, in a hybrid cloud environment or on-premises. #watsonx provides a consistent common platform for use with your models, open models, and our models.... Read the announcement here: https://ibm.biz/BdKqta

18

Registration for #APISummit24 by Kong Inc. is about to close! RSVP now so you don’t miss out on lessons from some of the world’s most notable API practitioners and AI innovators. This global event will explore the convergence of AI and API tech and cut through the noise to provide real-world lessons for your organization. ⏰ The clock is ticking! 🦍 #KongPin #AI #API #TechEvent

30

"AI will eat software." This bold claim from Ali Ghodsi, co-founder & CEO of Databricks, takes Marc Andreessen's famous truism one step further. AI won't be a mere add-on—it will be native to all data platforms. Intelligence will creep in wherever you have data. That’s going to happen in every profession, in every industry, in every organization. The result will further software’s transition from a static, hard-coded tool into a self-driving, decision-making engine. Ali draws a parallel to Oracle's game-changing relational database in the late ‘70s. Just as Oracle revolutionized data organization, "AI databases" are the natural next step. Databricks calls it a Lakehouse—a database with ML and AI abilities built in. Wherever data resides, machine learning will sit right next to it. Increasingly, intelligence and automation will be bundled into the data architecture that underlies enterprise software. The integration of AI into core data platforms mirrors tech giants like Google and Twitter—data analytics and AI companies at the core. Search, targeted advertising, and social media are outgrowths of these fundamental capabilities. These companies disrupted their markets with data analytics and AI. How do we enable every company on the planet to do that?

116

11 Comments