Rok Fortuna

Regression-Testing algorithms are outdated! My recent projects have me working on various LLM optimisations, moving away from larger LMs to smaller and more task focused LMs. An interesting problem we run into during this migration is regression testing (primarily for generative and IR problems). No customer likes getting an update and all your models being worse than they were a day ago. So to tackle this, we've employed various novel methods to automatically test updates without QA involvement. One of the most effective techniques has been using an LLM-as-a-Judge. It's a one time setup initiative, fairly intuitive and future-proof. Using grading notes with the judge adds significant consistency and allows the judge to handle a broader ranger of judgement scenarios. These judges can be tweaked and expanded to a variety of tasks and automate the work of entire QA teams! If you want to chat more about how to design these judges, drop me a DM and I'd be happy to talk. You can also check out the cool stuff we at PremAI are doing on SLMs here : https://lnkd.in/gW-4pQ53

22

Having an ML-based optimisation model is great but a huge part of the success is nailing down what the model should optimise for - especially in domains like pricing where the goal is not that straightforward. And what if the answer is different per product, season of the year or temporary agreements with stakeholders and your current business strategy? How do you translate that into numbers that a model can digest? Our answer is a separate target variable outside of the model. Decoupling the target variable has many advantages: it means the strategy can change and you don't have to do anything to adjust the model. It also means you are forced to agree with others on what the ideal outcome should be and hand over the control what the model does to people who are the most suitable instead of the ones who built it. Read more about how we did it in Otrium here: https://lnkd.in/eR7sMWRe Thanks Nikki Bueno de Mesquita for being the best person to bounce ideas off and helping to shape this :) And Jorne, Francesca, Diana and Robert for thoroughly beta-reading the article.

47

5 Kommentare

Humans would not be needed anymore for data cleaning and labelling? 🤯 Meta released "Self-taught evaluator" that evaluates generative systems and also improves itself when needed. But why you should care? 1. Reduces project timeline by ~70% 2. Reduces cost by ~50% 3. Automates ~90% of the most tedious task This post walks you through - 1. The current challenges 2. Introduction to Meta's bad boy 3. How does it work? 4. Real-world applications and its USP 5. Challenges If you've liked the crisp content then repost it to uplift your network ♻️ And follow Bhavishya Pandit to stay ahead in Generative AI! 🚀 #llm #opensource #llama #huggingface #largelanguagemodels #google #ibm #meta #openai #rag #python #datascience #machinelearning #ml #ai #artificialintelligence #nvidia #machinelearning #genai #generativeai

360

29 Kommentare

It's been a while since I've worked with RAG, I heard about Anthropic's latest release of their Contextual Retrieval technique, and I knew I had to take a closer look. They're aiming to improve RAG systems by addressing a common challenge: when documents are broken down into small pieces, important context is often lost. I took some time to understand how Contextual Retrieval works. Here's what I learned: - Contextual Retrieval adds extra information to each small piece of the document, helping the system understand the information in the right context. Anthropic claims this has significantly improved the accuracy of their RAG systems in finding the most relevant information. - They also introduced a "reranking" method, which prioritizes the most relevant pieces of information at the top of the list. Similar ideas have already been used in RAG systems for some time. From what I've seen, Anthropic's approach of adding context to each small piece can be computationally expensive for large documents, with costs increasing rapidly as the document grows. There are alternative methods, such as adding headers with titles and summaries of the document and sections to each piece, which can be more efficient and cost-effective while still delivering good results. I believe a holistic approach is needed to optimize a RAG system for real-world applications, focusing on improving the entire system from start to finish, rather than just one component. https://lnkd.in/dTgERi8Z #RAG #ContextualRetrieval #NLPResearch #MachineLearning

5

Meta Launches LLaMA 3: The Next Frontier in AI Innovation Key Points: Open Source Advantage: Meta introduces LLaMA 3, an open-source large language model (LLM), positioning it as a direct competitor to OpenAI's ChatGPT. The model aims to democratize AI development by being accessible to developers, researchers, and businesses worldwide. Enhanced Capabilities: LLaMA 3 is engineered to excel in complex reasoning, follow instructions precisely, and generate creative content. It marks a significant advancement over its predecessors by integrating capabilities to handle a broader range of inquiries, including more nuanced and complex questions. Multimodal Integration: Planned future enhancements include multimodal functionalities that will allow LLaMA 3 to interact through various inputs like text, voice, and images, broadening the use-case scenarios. Strategic Deployment: Meta plans to embed LLaMA 3 across its extensive suite of applications and upcoming smart devices, including Ray-Ban Meta smart glasses and Quest 3, aiming to create a more integrated user experience. #MetaAI #LLaMA3 #AI #OpenSource #TechInnovation #ArtificialIntelligence #MachineLearning #SmartTechnology #FutureOfAI https://lnkd.in/ded9kwjg

3

🤖 Unearthing AI Tool Usage at Celebrate Company 💡 Who utilizes AI resources in today's dynamic workplace? To find out more about the AI Tool Usage at celebrate company, my colleague Martin Schmitt and me recently conducted a survey among our co-workers. Key Findings: 👥 Gathered data from 70 participants in total 🛠️ Approximately ¾ already use AI Tools for work 🏆 most popular tools are LLM Chats (like ChatGPT), followed by image generation tools ❌ 2 participants don't want to use AI tools at all 🔍 Desires for deeper knowledge: Prompt Engineering, General AI intuition and Domain-specific tools We gained a lot of insights and are very thankful for the great participation in the company. Now we are excited to translate findings into actionable strategies, fostering a thriving AI environment at our organization! Onward and upward! 🚀💻💡 #AIatWorkplace If you're curious about the form we used, you can have a look here: https://lnkd.in/dVEU5v82

9

The Rising Costs of Frontier AI Models: Training cutting-edge AI models has become significantly more expensive due to increased model complexity, higher data requirements, and the need for advanced hardware. For instance, OpenAI GPT-4 required an estimated $78 million for compute, while Google's Gemini Ultra cost a staggering $191 million. These investments underscore the immense resources needed for state-of-the-art AI advancements Any guesses for why the Gemini Ultra costs more than 2x compared to GPT-4 ? Source : https://lnkd.in/davKygGX #GenerativeAI #MachineLearning #GPT4 #OpenAI

13

OpenAI's New Feature: Predicted Outputs Accelerates Response Times: OpenAI's latest API feature, "Predicted Outputs," promises to reduce response times by allowing developers to include "anticipated completions" as part of their queries. Here’s how it works: When users provide a predicted output that closely aligns with what the model would naturally generate, it minimizes latency and enhances the interaction experience at no additional cost. However, if the predicted output doesn’t match the model's response, the system must sample additional tokens, which can increase costs. This tool could be a powerful solution for applications requiring fast responses, such as customer service, live chat, or time-sensitive decision-making. · Accelerated Response Times: The feature optimizes latency by skipping token generation if the prediction matches, delivering quicker outputs—ideal for real-time interactions. · Cost Efficiency with Accuracy: When anticipated outputs align, responses incur no extra charge, offering a cost-effective way to maintain speed. Balancing Speed and Precision: In scenarios where precision is critical, developers can weigh response times against accuracy, adjusting predictions to control costs. · Applications in High-Interaction Environments: This feature is particularly suited for use cases where fast, accurate responses are essential, like live customer support or real-time applications. https://lnkd.in/dFCWWxK7 #AI #OpenAI #Innovation #TechNews #MachineLearning #CustomerService #RealTime

2

Need robustness and scalability in your Data Product? Let's include the data experts in your pizza team, not outside. Below the point of view of Data Science Product through Bastille Collective. Julien Vong Mathilde Brousse Thomas Belahi

8

FlexAttention: Simplifying Attention in PyTorch _________________________________________________________________ FlexAttention is a new PyTorch API that combines the flexibility of PyTorch with the performance of optimized attention methods. It allows researchers to implement various attention mechanisms easily without needing to write custom kernels. Key Points: - Flexibility with Performance: -- Traditional attention methods offer high performance but limited flexibility. -- FlexAttention provides a flexible API that supports many attention variants with minimal effort. - How It Works: -- Standard Attention: Calculates attention scores using queries, keys, and values, normalizes them, and computes the weighted sum of values. -- FlexAttention: Allows for modifications to the attention scores through a flexible scoring function, enabling customization for different types of attention mechanisms. - Examples of Attention Variants: -- Relative Position Encoding: Adds positional information to attention scores based on relative positions. -- ALiBi Bias: Applies a bias to attention scores to account for position differences, enhancing model performance. -- Causal Mask: Masks future positions in the sequence to ensure the model only attends to previous positions, useful for autoregressive models. Handling Sparsity: -- BlockMask: Efficiently manages sparse attention masks, such as those used in causal attention, by creating masks that prevent unnecessary computations. Learn More: https://lnkd.in/g58pxB4A

8

Are GPTs/LLMs overfit models? The scientific answer has to be a yes! What does it actually mean, though? Does it "impact" users in any way? Typically, statistical models (advanced or otherwise) that are predictive (incl. generative) in nature tend to "memorise" the data they see during the training phase so well that they can perform their tasks with unbelievable accuracy, but only for the scenarios represented by data that was a part of the training set. They often lack generalizability, i.e., they cannot deal with unknown or unseen scenarios with the same level of confidence. GPTs/LLMs are mostly trained on a significantly high volume of data from the topic of interest and quite obviously tend to overfit. Hence, any interaction around a scenario that wasn't well represented in the training data would end up with an unexpected outcome. But since the data used for training represents the universe in that context, the chances of something not being "memorised" are fairly negligible. Hence, users would not "feel" the impact as such. Counterintuitive? Yes. How can a model that overfits be generalised? This is where relativity comes into play. The expanse of training data w.r.t. the topic or domain is such that it almost negates the constraints imposed by the model being overfit. So, despite being overfit models, GPTs/LLMs create the notion of being generalised. Alright, so can any pre-trained model serve any use case(s)? 

47

5 Kommentare

Here is a video creation tool that you need to try: Runway ML to me is the OG of text to video generation. While it didn't have the natural hype that OpenAI Sora has (plus their demo is way more convincing as well), it has remain one of the most accessible text to video product out there. The launch of Runway ML Gen 3 had put them back onto the race of the generated video category. I gave Runway a try to see if it's good enough for most use cases (and also because Gen 3 is not in the free plan yet, I am trying Gen 2 instead, which has up to 25 sec of video per month) Here is the same prompt that I used again from my previous post: A Paddington bear in sunglasses eating honey, dancing and headbanging in Kings Cross Station in front of a crowded passenger atrium And the results look much better than Luma with the first try! You can at least tell it is a Paddington bear from a distance! What would you do think? What prompt would try at https://runwayml.com/?

1

Learnings from Generative AI in the Real World: Andrew Ng on where AI is headed. It’s about agents. 🌟 AI Agents are more than useful tools RPA as Agents could decide to plan,solve and executes a task without human intervention.This might leading to "Agentic moments" where the Agents might do it differently then how humans do it 🌟 Agentic workflows let AI work iteratively, which yields a huge improvement in performance. 🌟 When is an Gen AI app is ready to release? What are best practices for enterprise use?-->Still to early. Need more work on Evals 🌟 Meta is the major open source LLMs. Should we be worried?The cost of training is falling by approx 75% annually, so we can expect more players, more open source providers

9

2 Kommentare

Anthropic's newest AI agent #Claude 3.5 is out and your feed is undoubtedly buzzing it's impressive technical specs and benchmark scores (and rightfully so). But I'd like to draw your attention to a fascinating aspect that often gets overlooked: the #AI's personality. Beyond processing power or token limits – it's about creating an AI assistant with a distinct moral framework and interpersonal style. This video introduced me to new dimensions to evaluate an AI's personality, such as - Charitability: How does the AI interpret a prompt assuming the user's best intentions? - Honesty: The importance of Claude being direct and truthful, and balance between creativity and hallucinations - Self-awareness: Claude's understanding of its own limitations and the ethical implications of its responses Like the format of the video, I believe this topic is an ongoing conversation but I appreciate how there seems to be genuine thought and effort placed in its discourse.

5

RAG Trick: Cosine similarity is the same as dot product Most embedding models normalize their output to 1.0, e.g. all models from OpenAI, Cohere, etc., check the documentation. In other words, the length of the vector is always 1. All the same: - Dot product - Cosine similarity - Euclidean distance (range 0-2 instead of -1-1) Just do dot product. There’s a lot fewer mathematical operations to get the same result. 💡 It’s a (hyper)sphere! Have a hard time wrapping your brain around distances in embedding space? Think about it as a sphere. All embedding points lie on the surface of that sphere. Not sure about you, but that simplifies a lot for me. All the same: - Distance from centroid (average of vectors) - Logistic regression (scikit-learn) The difference between these is the paradigm. With the centroid, you find the center by averaging. With a logistic regression you find the outside edge by training a logistic regression model to draw a hyperplane. Since all points are on a hypersphere, they get you the same result. Choose whichever makes more sense to you. IMO, - centroids are easier to update & delete from, but you have to come up with the distance - logistic regression is more obviously a classifier and so easier to wrap your head around and makes code clearer #RAG #LLMs #LLM #AI #embeddings #vectordb #vectordatabase

5

6 Kommentare

Awesome example of how reproducible dev environments with Jetify Devbox speeds up developer onboarding: "Linus demonstrated how he used Devbox from Jetify to significantly reduce the onboarding time for his students – from days to just a few hours – by setting up his computing environment in a re-runnable way."

4

The idea that data is what matters, not model architecture is an old piece of knowledge that is having a moment right now in LLM world. More and higher-quality data matters where you don't have synthesized knowledge. One hard-earned law of nature can substitute for an awful lot of data. It can also ground the data that you have and give you scaffolding for checking, verifying, cleaning, and policing your data. Mind the bones.

20

Agents are hyped for good reason: Most people think that the LLM is the limiting factor for high accuracy GenAI services, this is wrong: It's context. More specifically, it's how you architect the context (data). Large language models respond poorly to large amounts of context. They like bit sized pieces of context that allow them to stay laser focused on a task. At least, this is what is required if they're to stand up in a corporate environment that requires at least 90% accuracy. This is where agents come in. Imagine if we could create systems made up of many small components that are interacting with our digital environment and powered by a super smart machine that guides decision making a long the way? Well, you don't need to imagine, because we can create these systems by employing the emerging theory of agents. Important note: this document is a synthesis of many research papers and articles by AI greats such as Andrew Ng and is not my original work. Having used agentic architectures personally and for clients, it is truly a game changer. Please provide feedback to this document because it will evolve as more theory emerges on the topic of agents! Crdit- Mateios What questions are left in your head when you think about agents?

25

1 Kommentar