Philip Black’s Post

The title should have been: "In AI systems, smaller is almost always better". Good to see this article on small language models at the WSJ, which is the optimal method for internal chatbots run on enterprise data. Unfortunately, it still misses the bigger issue that language models have limited use, and doesn't mention the efficiency, accuracy and productivity in providing relevant data to begin with -- tailored to each entity. Even if limiting reporting to language models, which shouldn't be done when attempting to cover all of AI systems, please go beyond LLM firms and big techs as they have natural conflicts -- they are scale dependent. Mentioning big tech and LLM firms is like citing fast food giants for stories on good nutrition. Yes, one can find an occasional story, but that's not where most of the value is. It gives readers the wrong impression. There is an entire health food industry out there. The same is true for responsible AI. That said, it's an improvement over the LLM hype-storm. ~~~~~ “It shouldn’t take quadrillions of operations to compute 2 + 2,” said Illia Polosukhin. “If you’re doing hundreds of thousands or millions of answers, the economics don’t work” to use a large model, Shoham said. “You end up overpaying and have latency issues” with large models, Shih said. “It’s overkill.”

For AI Giants, Smaller Is Sometimes Better EXCERPT: The start of the artificial-intelligence arms race was all about going big: Giant models trained on mountains of data, attempting to mimic human-level intelligence. Now, tech giants and startups are thinking smaller as they slim down AI software to make it cheaper, faster and more specialized. This category of AI software—called small or medium language models—is trained on less data and often designed for specific tasks. The largest models, like OpenAI’s GPT-4, cost more than $100 million to develop and use more than one trillion parameters, a measurement of their size. Smaller models are often trained on narrower data sets—just on legal issues, for example—and can cost less than $10 million to train, using fewer than 10 billion parameters. The smaller models also use less computing power, and thus cost less, to respond to each query.... #techgiantsthinksmaller #smallercheaperaimodels #aimodelstentimescheaper

LET'S GET SMALL (with #AI): It's obvious in my mind, but I am always focused on operationalization and meeting model / AI #governance requirements; small models are much more congruent with that, particularly since one of my Zoldi-isms is "explainability > predictive power." Perhaps other organizations will back into that belief as a byproduct of adopting smaller AI models; then, perhaps the realization will be that when we no longer need the AI giants, small language models will be within the grasp of internal AI teams. Edited to add: This was one of my #AIpredicitons2024, in fact. 2. Small will be beautiful "Large AI models, including large language models (LLMs) like ChatGPT, Google Bard AI, et. al., are incredibly unwieldy, difficult and expensive to build and operate. Furthermore, they are not easily retrained without cannibalistic entropy. "I have long held an unorthodox belief in “explainability first, predictive power second,” a core tenet of Responsible AI. In 2024, small, highly explainable AI models will be accepted and embraced as being more effective than outsized large models, as organizations focus on practical, reliable, net-positive business outcomes." My full predictions blog is here: https://lnkd.in/gbuVMVfN

Yay! 🎉 Snorkel AI got mention in recent The Wall Street Journal article. Our unique programmatic approach is helping companies customize their AI models 10-100 times faster by embedding their business expertise. As we explore the practical applications of large language models (LLMs), we’re turning our attention to specialized models and smaller models. These not only reduce latency but also provide immense value. To understand the state of GenAI… Read more here 👇

For AI Giants, Smaller Is Sometimes Better https://lnkd.in/eUZTt4em

This article highlights what our Data & AI practice has been applying for a while. We can build more accurate and efficient models. To build more accurate and efficient AI models, incorporating foundational statistical models can be highly beneficial. These models offer a robust framework for understanding and manipulating the underlying data distributions, ensuring that the AI models are grounded in solid statistical principles. LLM benchmarking is key for selecting the right GAI model but incorporating it with foundational methods is a true differentiator. Companies are turning their attention to less powerful models, hoping lower costs and solid performance will win more customers.

In the latest Wall Street Journal article, Tom Dotan and Deepa Seetharaman explore how tech giants are moving from giant AI models to smaller, cost-effective ones.   Smaller models, like Microsoft’s Phi, are cheaper and specialized, performing many tasks almost as well as their larger counterparts.  #wsj #AI #hpaipc

With a vote approaching in California on the landmark AI bill SB 1047, the tech industry is getting closer to the first real piece of legislation that will attempt to ensure responsible development and safety standards for AI platforms. Arun K. Subramaniyan, CEO of client Articul8 AI, weighs in for this Business Insider story from Lakshmi Varanasi, discussing how startups may be impacted by the bill and the benefits of regulation for general-purpose AI models. #AI #GenAI

AI has a data problem. The rule of thumb is that technology will double every 8 months (that's the current metric) But with LLMs this is not happening, we've already started slowing down, even with the addition of GPT-4o (which is great BTW) we're running into the issue of recycled data. LLMs only work well when it has access to new and rich data, but what happens when all of the new data is generated by LLMs? Diminishing returns. So what can you do? Start leveraging your personal expertise with AI. If you have a tech product you're going to need to build that expertise into the non AI parts of your application. What do you think, have you started seeing the data problem in practice?

this week in artificial intelligence (though, really, most of this editorial essay from YouTube channel Modern MBA explores why so few of the data-centric startups from the 2010s like Doordash, SoFi, Allbirds, or Wayfair can brag about their operating income; turns out, parsing through all that collected data is prohibitively expensive) ... "The people that worked on Big Data had a vested interest in keeping the technology trendy despite the lack of results, and are now doing the same with AI. Whether it's bottoms-up or top-down adoption, money talks, even for those who are simply implementing the technology itself. This is why there's never any end to the online debates between React versus Angular, Kubernetes versus ECS, and so on. You can see that same tribalism in AI where there's no consensus on what tool is best, Tensorflow or PyTorch, or which model is most accurate. The premise of Big Data was that you could unearth hidden innovation, business value, customer insights, and market patterns from data that was simply too overwhelming for a human to digest and analyze. These days everyone has seemingly forgotten about Big Data's failed promises. But the current premise of AI is even more confusing. The new narrative is that data is inherently too complex. Instead what we're supposed to believe is that all that promised innovation, buried insights, and hidden business value is still in this data, it's just that humans can't pull it out. Instead, our only solution is to trust these artificial models—where only a few people have a true understanding of what's really happening under the hood—for perspective and insights; we should be told the answers and not seek it ourselves, and we should value the digestibility and presentation above the accuracy of information itself. For Big Data, every company had its own data, timelines, politics, and priorities, which made execution completely unique—every implementation was a Snowflake deployment. Since no one has achieved any value with Big Data, no one really knows, even now, nearly a decade later, how to actually implement and derive value. No one can authoritatively say: This is how you execute, and this is how you can replicate step for step what we did for your own business. Engineers essentially are reinventing the wheel but never finishing."