Sri Satish Ambati

Just having data is not enough in building a AI model, you have to make lot of modifications to the data and change its structure. In this article I discuss about some most fundamental ways of changing data shape which become very necessary when working with tensors. More on this comming soon. #DeepLearning #ML #MachineLearning #Pytorch

3

Organizations have insufficient technical capabilities for the speed and scale of their ML model implementations. My colleague Sumit Agarwal just published a very helpful research with a capability view and sample vendors for defining a modular, flexible and scalable MLOps solution. https://lnkd.in/gnB_T9bq #MLOps #LLMOps #AI

149

17 Comments

*Mixture of Agents (MoA)* is an innovative way to boost Large Language Models’ performance by leveraging the strengths of multiple agents as a team. Layered Architecture: MoA arranges many LLMs in layered architecture, each layer composed of numerous individual agents(LLMs). Collaboration: These agents generate responses that are based on outputs from the previous layer’s agents. They repeatedly refine and enhance the last output. Diverse Insights: By integrating different capabilities and insights from various models, MoA achieves a more powerful combined model that is secure against diverse risks. Performance Boost: MoA outperforms standalone LLMs by a wide margin. For instance, it set a new benchmark with 65.1% on AlpacaEval 2.0 as against GPT-4 Omni at 57.5%, using only open-source models. In short, if we combine language modelers’ strengths then it results in increased efficiency and versatility. Here is a nice implementation of MOA by Together AI Github source: https://lnkd.in/dMCVRS3D #AI #LLM #LargeLanguageModels #MachineLearning

58

3 Comments

LLMs and Optimization: An Unbeatable Combo!

19

If you think compute scaling will plateau, think again. We are in the baby states of inference time compute. VLLMs coupled with computer tools, reward learning and RL can leverage orders of magnitude more inference time compute than what we have seen with foundation models. Here’s a concrete training architecture (replace \pi with a large pre-trained vLLM and add it to the reward learner too, \R is the space of all computer tools)

21

1 Comment

Very cool article by Christoph Molnar on the notion of inductive bias (here for Random Forests). The concept of inductive bias is often overlooked by practitioners, who assume this is a purely theoretical notion. On the contrary! This is one of the most fundamental aspects to consider when picking a model, interpreting its predictions or troubleshooting an underperforming system. #inductivebias #cart #trees #machinelearning #troubleshooting

1

The current trend in LLMs is small models. This easy-to-understand article discusses the various ways to achieve model compression and provides key references.

5

Very interesting details in the Gemma 2 paper now up on arXiV including e.g. ablation results on some of the new techniques introduced in this version of the model. https://lnkd.in/giX7Uwmj

42

Had the privilege to join a Fireside Chat with Sudivya from the Google Cloud team at IIMA Ventures. Key takeaways: LLM democratization: Making custom LLMs accessible to non-tech users. AI in action: Impactful use cases like RAG and domain-specific automation. Build vs. leverage: Pre-trained models offer speed and flexibility; fine-tuning ensures precision. Tech challenges: Managing data at scale, ensuring compliance, and balancing the economics of GPU-intensive workloads remain complex challenges. Balancing data efficiency with cost-effective GPU scaling via distilled LLMs. Why GCP?: Powerful AI tools, stellar startup support, and global scalability. #DataNeuron #AI #FiresideChat #LLM #GoogleCloud #Innovation #Startups

41

Good detailed overview of Retrieval-Augmented Generation (RAG), one of leading techniques to improve accuracy of LLMs. When developing a production-grade LLM-based system, it's essential to understand the advantages and limitations of different techniques (RAG, prompt engineering, fine-tuning etc), across training and serving/inference phases, to improve LLMs for specific applications (see my previous post https://lnkd.in/giuJdJJt) A balanced approach to LLM applications that takes into account accuracy, latency, and cost is crucial for successful productization at scale. #artificialintelligence #largelanguagemodels #GenerativeAI #airevolution

10

Semantic chunking vs. fixed-size chunking: Which delivers better results in Retrieval-Augmented Generation (#RAG) systems? 🤔 In our latest blog, Renyi Qu and Forrest Sheng Bao, engineers at Vectara, challenge the hype around semantic chunking. Through systematic evaluation across RAG tasks, they uncover surprising insights into performance, computational costs, and real-world practicality. 🔍 Key Takeaways: ✅ Fixed-size chunking often outperforms semantic chunking in real-world scenarios. ✅ Computational costs of semantic chunking don’t always justify the benefits. ✅ Focus on embedding quality for bigger gains in RAG performance. 👉Check out the full blog here: https://lnkd.in/gZ9M7eYD

15

#ObsessivelyEnthusiasticInvestor #UnreasonablyEnthusiasticInvestor #IrrationallyEnthusiasticInvestor at REFASHIOND Ventures: The Industrial Transformation Fund. We look for the people we believe are doing the best work in Industrial Transformation & Supply Chain Technology; Attempting the impossible. Then we go all-in on them. With obsessive and unreasonable enthusiasm. Join us. #IndustrialTransformation #SupplyChain #Innovation #Technology #Startups #VentureCapital #DemandTheImpossible #UnreasonableEnthusiasm #ObsessiveEnthusiasm #IrrationalEnthusiasm 

6

This is a win for #sustainability. In effect, ML applications may better leverage GPU capabilities, saving more time and energy 🚀

1

Human in the loop evals 🧑‍💻👀 → custom, automated evaluation models 🤖📈? https://lnkd.in/gruPtC8d Human in the loop (HITL) is a really popular technique for teams just getting started with LLMs and their LLMOps functions. Given your biggest concern is your model acting ‘not like a human’, it’s easy to see how sticking a human in the loop, to review and edit any potential mistakes or weirdness, can solve that problem. But it’s equally easy to see how challenging this can be to scale, and how it can eat at a lot of the cost-savings promised by automations. So how should teams approach balancing the security and certainty from human-in-the-loop, with an eye towards scaling? Over the weekend I got the chance to experiment with a few approaches to turning HITL datasets into their own custom evaluation models. I also think about how I would balance my human-agent evaluations alongside my model-driven evaluations, to set me up for long-term success. https://lnkd.in/gruPtC8d

17

1 Comment

The most common feedback I get from customers is "we have so much data, but don't know how to get value out of it." Current tools and product analytics models have failed to fill that void. Love how the team at Motif explains the difference between what leaders think a "causal driver" is, what it means in practice, and how you figure out what those are. Incredibly complex concepts explained extremely well! No, it won't give you a (probably BS, but conveniently packaged) narrative like "a Facebook user is hooked after 7 friends in 10 days". But it can give you the most points to build your intuition and experiments around. Tools like Motif (and we've been working on 😉 ) are reducing the domain, technical expertise, and time required to get value out of your data. We won't replace analysts in the near future, but I see it as an upskilling tool. If every junior analyst or data scientist is effectively more experienced, the strategic questions to answer and business context - that are nearly impossible for LLMs to capture today - become the differentiators between project success & failure. If this is a space you're interested in, would love to connect!

7

Causal AI is an emerging field within AI, but there is a distinction between causal inference and causal tracking.

1

Large language models are wonderful at ingesting large amounts of content & summarizing. Uploading an academic paper and I can pester it with an infinite list of questions & it will respond with equally infinite patience. In comparing the two most recent Microsoft earnings calls, Claude highlighted: - faster than expected Azure growth (29% vs 27%) - AI contributing 8% of Azure revenue up from 7% - higher CapEx spending & greater capacity constraints for data centers - much better commercial bookings growth : 17% vs 13% Excellent analysis. Benn Stancil described LLMs as great averagers of information. But what if I don’t want the average? If I seek the data point two standard deviations out? If I’m channeling my inner Anthony Bourdain & I’m seeking the fermented shark or cobra heart within my query? At Google, we ranked web pages & ads with many signals. An engineer taught me that of them found rarity of a word across a set of documents. For example, if there’s a document containing 10 instances of the word ghotiacross a collection of documents, where the average document has 0 ghotis, that document is likely the best to answer a search about fish. I haven’t found a way to goad an LLM to produce the rare result. Why did one user decide to use a particular piece of software when 10 others did not? Maybe I haven’t learned how to prompt an LLM well. Ideogram launched a feature called Magic Prompt that expands a basic prompt into richer instructions that elicit better results, marrying the language of a user & a computer. A user might write : A boy with a dog in a park. Magic Prompt replies : A heartwarming scene of a young boy playing with his happy, wagging dog in a lush, green park. The boy has a playful smile on his face while he tosses a bright yellow ball for his furry friend to fetch. The park is filled with beautiful flowers and trees, creating a serene atmosphere for this bonding moment between boy and dog. Magic Prompt surprised me in three ways. First, unlike Google search where terse queries tend to work very well, LLMs relish verbosity. Less Hemingway & more Faulkner. Second, maybe there’s a prompt for rarity. If you know what it is, please share with the rarity shibboleth. Third, I wonder what other prompt techniques are out there that have yet to be discovered to wrest the best from an LLM. This technique is called TF-IDF. It means Term Frequency Inverse Document Frequency.

21

6 Comments