Observer’s Post

🚀Building Free, Building Reliably, Building Proud: The Road to LLM Observability🚀 In the ever-evolving world of AI, ensuring the reliability and performance of large language models (LLMs) is crucial. How significantly does RAG (e.g., embedding technique, new pre-trained models) enhance our product, or does it potentially hinder its performance? Namira Suniaprita ’s latest article on Medium dives into the importance of observability in LLMs, using Phoenix from Arize AI, LLM-as-a-judge concept and provides a comprehensive framework to achieve it in state-of-the-art models like OpenAI’s GPT and Google’s Gemini models. Here are the main takeaways from our work: Model-Agnostic Engine Architecture Overview which uses AI Agents (ReAct) leveraging Lang Graph, LangChain, Google Search, facilitates model-agnostic operations, choosing between different models based on the task at hand. Vector Embeddings with Vertex, Elasticsearch and Redis Cache. Phoenix providing insights into model performance, helping us monitor and refine RAG continuously. 🔍 Why Observability Matters: Understanding, debugging, and improving LLMs require robust observability. It allows for real-time monitoring and performance tracking. ⚙️ Challenges: The complexity of LLMs, the need for real-time insights, and the difficulty in interpreting outputs are significant hurdles. 🛠️ Framework for Success: Metrics and Logging: Define key metrics and implement logging to track behavior. 📚 Real-World Examples: The article showcase the practical implementation of observability in LLMs. By following this framework, developers can build more reliable and robust LLM applications, ensuring they can monitor and respond to issues effectively. Check out the full article : https://lnkd.in/eCvpSVZz #AI #MachineLearning #Observability #LLM #TechInnovation

I feel like in the past few years, my vocabulary has increased exponentially as I have attempted to get my arms around the terms used in #datascience and #machinelearning. Just when I think I am up to speed, new words pop up. #AI, #neuralnetworks, #LLMs, #genAI, and now hello to #SLMs - the acronym for small language models. Imagine the power of language leveraged in LLMs, but for a focused use case not requiring the full training data or prompt complexity of an LLM. Hello, SLM. SLMs can run on a fraction of the compute required by an LLM and are trained on specific domain data. Microsoft recently announced #Phi-3, an SLM available in #AzureAIStudio. "Phi-3 models significantly outperform language models of the same and larger sizes on key benchmarks (see benchmark numbers below, higher is better). Phi-3-mini does better than models twice its size, and Phi-3-small and Phi-3-medium outperform much larger models, including GPT-3.5T." https://lnkd.in/gxETb__5

"Generative AI and large language models (LLMs) like GPT-4, Llama, and Claude have pathed a new era of AI-driven applications and use cases. However, evaluating LLMs can often feel daunting or confusing with many complex libraries and methodologies, It can easily get overwhelming. LLM Evaluation doesn't need to be complicated. You don't need complex pipelines, databases or infrastructure components to get started building an effective evaluation pipeline." https://lnkd.in/dk9FiAVH.

How good is small? 2024 does have a Small Language Model quality about it. Microsoft's new Phi-3 is one of the smallest AI models available. While SenseTime 商汤科技 announced their own Generative AI model SenseNova 5.0 that focuses on knowledge, mathematics, reasoning, and coding capabilities, don't expect it to get any coverage. Phi-3 learned from ‘bedtime stories’ created by other LLMs. Despite its tiny size Phi-3 mini has already performed as well as Llama 2 on some benchmarks with Microsoft saying it is as responsive as a model 10 times its size. Sebastien Bubeck even made a quick demo about it. Not Sparks of AGI this time, but efficient. Phi-3 Mini measures 3.8 billion parameters and is trained on a data set that is smaller relative to large language models. While Apple is likely to showcase their efforts in on-device models, the open-weight ecosystem is thriving. Microsoft for all their efforts, didn't really move the needle in Search yet. They still like to brag: "Phi-3 is not slightly cheaper, it's dramatically cheaper, we're talking about a 10x cost difference compared to the other models out there with similar capabilities," said Sébastien Bubeck, Microsoft's vice president of GenAI research. Some like to compare: Phi-3 Mini is as capable as LLMs like GPT-3.5 “just in a smaller form factor.” Microsoft Phi-3 was trained with a "curriculum", according to VP of Azure Eric Boyd. Phi-3 comes in three sizes; mini which is just 3.8 billion parameters, a 7 billion parameter small and the 14 billion parameter medium model SLMs are designed to perform more simple tasks, making it easier for use by companies with limited resources, the company said. Read the paper: https://lnkd.in/grYtx7nU Read the blog: https://lnkd.in/gM2tebpr On Azure: https://lnkd.in/gpm7pTqz

AI is becoming 10x cost-efficient y-o-y. A game-changer for innovation and accessibility? The cost of running large language models (LLMs) has been dropping at an astonishing rate, a trend that Andreessen Horowitz has coined "LLMflation." According to their analysis, LLM inference costs are dropping tenfold each year—bringing the same level of performance at a fraction of the cost. 💡 Key Insights: 1,000x Cost Reduction in 3 Years In November 2021, running GPT-3 cost $60 per million tokens. Now, similar models cost around $0.06 per million tokens. What’s Driving This? Advances in GPU performance, model quantization, software optimizations, and competition from open-source models have all contributed. More Accessible AI Applications As costs drop, more businesses can deploy LLMs in real-world applications, opening up transformative possibilities. My Take: This trend is game-changing. Lower inference costs are enabling businesses and developers to work with advanced AI without needing massive budgets. As the ecosystem grows, we're likely to see AI applications in areas that were previously cost-prohibitive. #DigitalTransformation #LLM #AI #withRanjan

A cool behind the scenes look at how we created DBRX, an open source Large Language Model (LLM) that outperforms all established open source models, as well as GPT-3.5 on standard benchmarks. In addition to high quality, the model is also extremely fast and cost-effective for inference. https://lnkd.in/g4NAJt6n

There's been a lot of debate in recent weeks about whether foundation model scaling has hit diminishing returns i.e. will GPT-5 (or whatever it's called) be the same kind of step change improvement that we saw between GPT-3 and GPT-4? While many leading researchers have indicated that this may somewhat be the case it also misses the point. There are many ways to make Large Language Models smarter and researchers are only just skimming the surface of what's possible. It's true that GPT-4 has already been trained on most of the available data on the internet and so it's going to be hard to find significantly more data to train on but data quality is just as important as quantity. If you were able to replace the bad data in your training set with better data you'd get a 'smarter' model without it being any larger. One route to this is licensing high quality data from reliable sources which most of the AI companies are doing but synthetic data may be far more powerful. Just a few months ago people were worried that training AI models on AI generated text would lead to the models getting worse but then the o1 model came along and changed the game. o1, for the uninitiated, is a newish model from OpenAI which introduced a new way to get better answers by giving the model more time to 'think'. For those of you who are familiar with prompt engineering terms this is basically a form of Chain of Thought (similar to the Tree of Thoughts technique we teach in our classes). Using the o1 model they can produce much higher quality synthetic data than the median content in the existing data set and then use that to train a new model. As far as I can see there's no reason this approach couldn't be applied repeatedly, incrementally improving each version of the foundation model until we get something much 'smarter' than the original dataset would have allowed for. I'm guessing that when Sam Altman says that they know what they need to do to build AGI this is what he's referring to but what do you think?

Large language models (LLMs) have transformed, and continue to transform, the AI and machine learning landscape, offering powerful tools to improve workflows and boost productivity for a wide array of domains. I work wi...

Notable for his substantial contributions to OpenAI and former leader of Tesla's autonomous team, Andrej Karpathy recently declared his intention to transition towards personal projects, and this development has already generated considerable attention in the software community. Karpathy's contributions at OpenAI encompassed the creation of a conversational AI assistant that drew inspiration from the fictitious J.A.R.V.I.S. system popularized in films. His objective was to develop an AI tool that was authentic and practical in nature. His most recent publication, implementing the Byte Pair Encoding (BPE) algorithm, was made public after his departure. Byte Pair Encoding (BPE) is an algorithm for tokenization in large language models (LLMs) such as GPT. By functioning at the byte level and processing UTF-8-encoded strings, it can efficiently process an extensive array of human languages and symbols. The GPT-2 paper popularized the LLM approach, and contemporary models, including GPT, Llama, and Mistral, have since adopted it. #ai #bpe #Karpathy https://lnkd.in/edehgADR

Remember when Mistral AI promised open source? Their latest move tells a different story... Their new closed-source LLM is undeniably powerful. But this shift raises concerns about their open-source commitment. Is this a sign they're following OpenAI's path? Read the full Blog here: https://lnkd.in/d8azRmVE 𝗪𝗮𝗻𝘁 𝘁𝗼 𝘀𝘁𝗮𝘆 𝗳𝘂𝘁𝘂𝗿𝗲-𝗽𝗿𝗼𝗼𝗳 𝗶𝗻 𝘁𝗵𝗲 𝗰𝗵𝗮𝗻𝗴𝗶𝗻𝗴 𝗔𝗜 𝗹𝗮𝗻𝗱𝘀𝗰𝗮𝗽𝗲? At Katonic AI, we're LLM agnostic. Experiment with commercial giants (OpenAI GPT-4, Cohere Command) and open-source leaders (Meta Llama 2, and more) on the Katonic AI GenAI Platform. Don't let vendor restrictions hold you back! Let's talk about the future of your business! #LLM #MistralAI #opensource #AI