Temi Afeye

The TLDR - Reinforcement Learning from Human Feedback is not really Reinforcement Learning and, although of some use, is partially responsible for the fact that LLMs want to please us rather than be right. Very odd - we are creating the exact inverse of the AI dystopias we have been so scared of - our AIs are possibly..... too nice! But Karpathy points out that this probably a temporary situation. Someone will figure out how to use RL more effectively for LLMs and unlock a step change in accuracy. We are really only at the beginning of this AI renaissance.

9

It's been incredible to witness the impact that AI is having on various industries. A striking example is the journey of Carrie Rich, whose daughter, affected by a rare genetic condition, showed improvement after starting an AI-identified repurposed drug. Researchers have used AI to analyse existing medications that could potentially modify disease progression. One such drug, originally designed for diabetes, has shown promise in early trials to improve cognitive functions in Alzheimer's patients. These cases underscore how AI and data analysis not only accelerate the drug discovery process but also enhance it by uncovering unexpected therapeutic potentials. This approach not only saves time and resources but also opens new avenues for personalised medicine, offering hope where traditional methods have faltered. I've summarise below some of the ways AI is being used: [1] Data Mining and Integration: AI algorithms can process and analyse vast amounts of data from diverse sources such as scientific literature, clinical trial databases, and genetic databases. This capability allows AI to identify potential links between drugs and diseases that are not immediately obvious. [2] Predictive Modelling: AI can build models that predict how a drug might interact with specific biological pathways or disease mechanisms. This involves understanding the drug's molecular structure, target profiles, and possible side effects to suggest new therapeutic uses. [3] Molecular Docking Simulations: AI can simulate how a drug molecule binds to different targets in the body. This is crucial for understanding if an existing drug could be effective against a different disease by interacting with new biological targets. [4] Machine Learning Algorithms: These are used to sift through the data and identify patterns or correlations that might suggest a drug’s efficacy for a new use. Machine learning can also refine its predictions based on new data, improving its recommendations over time. [5] Natural Language Processing (NLP): AI uses NLP to scour through unstructured data (e.g., text from scientific articles and patents) to extract useful information that can lead to drug repurposing opportunities. Another great example of AI making a positive impact, and the importance of continued investment and innovation in AI technologies. #HealthTech #AI

10

My one consistent messaging during this hype (business hype but still scientific progress) has resounded with ‘data-centric’ AI. Understanding how noise propagated and explodes across higher dimensions requires a good foundation in understanding how models work, how weights are learnt, transfer learning, gradients etc. Happy to see the shift once again change from ‘throwing everything at it’ to ‘The game is now about one thing: developing the data to tune and evaluate these LLMs for real business use cases’. This is the approach I have always taken and inspired in others and my teams(standing on the shoulders of giants like Andrew NG, Fei Fei Li and Yann LeCun to guide us) GenAI is simple a step in an interesting and fascinating direction but to move to reliable and scalable models we need a lot more research where we incorporate planning, world agents potentially through deep RL. Excited to see where we head and thankful for the companies who have trusted the science and kept moving forward! also excited for the new era of robotics this is going to lead to! #dataCentricAI #genAI #responsibleAI

8

How do you scale your AI Agent experiment into a real solution for production-level usage? The AI conversation reminds me a lot of the Big Data conversation from a decade ago. The initial concept works great but scaling it beyond just a few use cases falls flat. This doesn’t mean that AI can’t be useful, it just needs supporting technology to be truly effective. Many of these same AI problems existed in Big Data and were solved by clever and robust technologies. You can move more quickly and build better by leveraging tried and tested technologies! #ai #agents #scale #efficiency #bigdata

21

2 Comments

🚀 Microsoft has just released GraphRAG, a groundbreaking approach to Retrieval Augmented Generation (RAG) that leverages knowledge graphs to enhance LLMs' reasoning capabilities. ## Key Features of GraphRAG: - Structured Hierarchical Approach: GraphRAG extracts a knowledge graph from raw text, builds a community hierarchy, generates summaries for these communities, and leverages these structures for RAG-based tasks. - Improved Question-Answering Performance: GraphRAG shows substantial improvement in answering questions that require traversing disparate pieces of information or understanding summarized semantic concepts over large data collections. - Enhancing LLMs with Private Datasets: GraphRAG helps LLMs reason about private datasets, such as enterprise proprietary research, business documents, or communications, which the LLMs have never seen before. Getting Started with GraphRAG: - Solution Accelerator: Microsoft provides a user-friendly end-to-end experience with Azure resources to quickstart the GraphRAG system. - GitHub Repository: Check out the GraphRAG GitHub repository to access the code and documentation. - Prompt Tuning: Fine-tune your prompts following the Prompt Tuning Guide in the GraphRAG documentation for optimal results. ## Explore GraphRAG: - Microsoft Research Blog Post: Visit the Microsoft Research Blog Post to learn more about GraphRAG and how it can enhance your LLMs' abilities to reason about your private data. - Indexer and Query Packages: Dive deeper into the main sub-systems of GraphRAG by exploring the docpages for the Indexer and Query packages. Don't miss out on this exciting opportunity to leverage GraphRAG and take your LLM-based tools to the next level! 🌟 Important links in the comment. #GraphRAG #LLMs #RetrievalAugmentedGeneration #MicrosoftResearch #ArtificialIntelligence

21

1 Comment

Is Human in the Loop the Answer for Building Confidence in AI Systems? The Real Story is More Like a 'Hapless-in-the-Loop' Situation In the high-stakes AI world, where SOTA tech and high-octane innovation are the norm, the debate rages on about the future of AI. Businesses and users swear by "human in the loop" systems, believing it is the secret sauce for safe and exceptional performance. But does this hold water? Enter Feckless Effem, an FM/ML solution with a reputation for fast decisions and great accuracy. Alongside Effem is Hapless Human, a diligent but error-prone human involved in making decisions. Both are given the same high-pressure classification task. As the task unfolded, the results were a surprise. Despite a few huge blunders, Feckless's performance left Hapless in the dust, as seen in stubbornly higher cumulative logloss figures for Hapless, versus lower and consistent logloss for Effem. Delving deeper into the results, we saw a pattern emerge. Hapless' errors were frequent and fell within a narrow range, but Feckless Effem’s were infrequent, and sometimes catastrophic. This throws a wrench into the prevailing wisdom that human-in-the-loop invariably improve AI performance. The Points We Make: 1. The Reality Check:   - Feckless Effem outperformed Hapless Human on average in classification tasks, despite occasional high-impact errors.   - Humans consistently showed higher cumulative log loss, indicating a higher rate of mistakes over time. 2. Understanding the Classification ML Problem:   - The goal is to minimize both the size and variability of errors.   - Human errors were frequent and moderate, while machine errors, though less frequent, are sometimes significant. 3. Rethinking "Human in the Loop":   - The assumption that humans always enhance machine performance might be flawed.   - Systems should focus on minimizing both the absolute size and variability of errors, leveraging the strengths of both humans and machines. 4. Strategic Human Oversight:   - Humans should be strategically deployed to catch rare but significant machine errors, rather than overseeing every decision.   - This approach leverages the reliability of machines and the intuition of humans to mitigate catastrophic errors. 5. Creating Synergy:   - Continuously track decisions and compare them to ground truth as they become available.   - This feedback loop allows models to learn and adjust, refining them to reduce both the size as well as the variability of errors over time.   - By adopting this data-driven approach, Feckless Effem and Hapless Human can support a robust, synergistic decision-making process. Conclusion: It's not about choosing humans over machines or vice versa—it's about creating a synergy that makes our systems truly exceptional. 🌟 #ml #ai #GenAI4FS #inc81starch

42

3 Comments

what should pretraining compute be spent on? 4 years ago: mostly parameters 2 years ago: an even split of parameters and data now: mostly on data you might be wondering why it has changed so much; here's a brief intellectual history of pretraining recipes and why it has evolved over the past 4 years https://lnkd.in/gSSQse8X

14

2 Comments

Training embedding models. Colab Notebook. #LLMs #RAG #ml #python #bookmark

2

It's been a busy few days in the open finance space, as developments with FIDA and the announcement of the UK's Smart Data Roadmap all occurred almost simultaneously (at least that's how it felt). I've put down my quick take in this blog post (link below) but the main message is that this opens up significant new product development opportunities for banks active in these territories. The low hanging fruit will be in areas like personal financial engagement (PFE) propositions, but there will be potentially significant impacts on areas such as SME lending and mortgages too. #FIDA #SmartData #openfinance #opendata #banking #payments Celent

22

1 Comment

Not a surprise since fine tuning benefits just static data. In general, I don't see great use case for fine-tuning, to justify ROI, the fine tuning needs to apply to the use case where you are dealing with massive, massive amount of static data. Typically I see that good prompt engineering work, regular RAG, and agentic RAG are effective. 🌟 Let me know your thoughts? #AI #GenAI #LLM #RAG Thanks for sharing it Andrew Jardine

4

Value capture has always fluctuated between applications and infrastructure. What will we see getting commiditized first in the ML world - compute infra or applications? Read my thoughts on this here : https://lnkd.in/gD3N7dHX #strategy #hyperscale #scaleups #bizgrowth #foundersadvice #bizmodel #innovation #fintech #insurtech #tokenization #productmanagement #web3  #TMinsights

5

𝐄𝐯𝐚𝐥𝐮𝐚𝐭𝐢𝐧𝐠 𝐋𝐚𝐫𝐠𝐞 𝐋𝐚𝐧𝐠𝐮𝐚𝐠𝐞 𝐌𝐨𝐝𝐞𝐥 𝐀𝐩𝐩𝐥𝐢𝐜𝐚𝐭𝐢𝐨𝐧𝐬 Unlike traditional software, which often focuses on functional testing for specific predetermined outcomes, evaluating LLM applications involves a broader and more nuanced spectrum of considerations 𝐇𝐞𝐫𝐞'𝐬 𝐰𝐡𝐲 𝐞𝐯𝐚𝐥𝐮𝐚𝐭𝐢𝐧𝐠 𝐋𝐋𝐌𝐬 𝐢𝐬 𝐚 𝐝𝐢𝐟𝐟𝐞𝐫𝐞𝐧𝐭: 𝐁𝐥𝐚𝐜𝐤 𝐁𝐨𝐱: Unlike traditional models, LLMs are complex models. We can't pinpoint why they generate a specific output, making debugging trickier. 𝐍𝐮𝐚𝐧𝐜𝐞𝐬 𝐨𝐟 𝐋𝐚𝐧𝐠𝐮𝐚𝐠𝐞: Unlike a program spitting out a single answer, LLMs can be creative! This means evaluating factors like factual accuracy, coherence, and even user preference become important 𝐌𝐨𝐯𝐢𝐧𝐠 𝐓𝐚𝐫𝐠𝐞𝐭𝐬: LLMs are constantly learning and evolving. An evaluation benchmark today might not be relevant tomorrow. LLMs offer incredible potential, but effective evaluation is key to unlocking their true value.  #AI #MachineLearning #LLMs #Evaluation

39

1 Comment

A great course for anyone wanting to learn in detail about LLMs, key prompting techniques, fine tuning methodologies for LLMs - be it Instruction Based or Parameter Efficient fine tuning (PEFT) like LoRA, Reinforcement Learning based on Human Feedback, Retrieval-Augmented Generation with LLMs, Evaluation Methodologies of LLMs, all this with well crafted lab sessions including mostly all listed concepts

6

A trillion-token datastore for retrieval-based models: https://lnkd.in/enEtKfet "A research team from the University of Washington and the Allen Institute for AI constructed a new datastore called MassiveDS, which comprises 1.4 trillion tokens. This open-source datastore is the largest and most diverse available for retrieval-based LMs. It includes data from eight domains: books, scientific papers, Wikipedia articles, GitHub repositories, and mathematical texts. MassiveDS was specifically designed to facilitate large-scale retrieval during inference, enabling language models to access and utilize more information than ever before." #NLP #LLMs

5

How can we inject our domain knowledge to the traditional ML models? In machine learning, achieving model interpretability while adhering to domain knowledge is crucial. My latest blog explores how implementing monotonic constraints can enhance both aspects. These constraints allow practitioners to embed domain expertise directly into their models, improving reliability and alignment with expert insights. This is especially important in regulated industries like finance and healthcare, where understanding the impact of input variable changes is essential. By enforcing monotonicity, models become less susceptible to overfitting, ensuring they behave consistently with prior expectations and maintain strong generalization performance on unseen data. If you have any comments or would like to engage in a detailed discussion, please feel free to reach out. Your opinion is greatly appreciated. #datascience #machinelearning #CodeX #mediumblog #gboost #dataanalysis

34

1 Comment

🧠 Inspiring talk from Jonathan Frankle at Databricks (prev Mosaic ML until aquired for $1.3B). In his own words, Jonathan "trains neural networks and spends a lot of money on GPUs"! The talk went through best practices of training LLMs which they learnt whilst training DBRX, their open-source LLM that was state of the art for "just 2 weeks"! We also went through the mind boggling maths on the costs behind training a foundation model ($100s of millions in some cases), and why it costs so much. What was potentially even more inspiring than the power of trained neural networks and the opportunities they open up, was just how clearly Jonathan loves what he does! LLMs have clearly kick-started a tidal wave of extremely valuable AI innovation & extreme investment 💰 Link to DBRX blog post: https://lnkd.in/ey3yhA3x

1

At Simpl, we use sequence models in a widespread way - from approving new users to determining which users are not likely to pay us back. A few weeks ago I sat down with Yash Vardhan Gupta and Nikita Gupta and got an understanding of sequence models - specifically RNNs and LSTMs and followed it up with readings of my own. In this article I explain these concepts to the audience who are decent with Math (read - comfortable with matrix multiplication), but not professional data scientists. There are very simple core ideas behind RNNs which lead to so powerful models. https://lnkd.in/gmq5AJDv

63

A Fully Automated Learning Algorithm Applicable to Board Game Playing Programs By Shree Ganesh Keith Ragoonaden, 1995. This UK SERC-funded research project was initiated in 1987 with the idea of implementing an unsupervised learning algorithm that would self-correct its assessment (if deemed necessary) of its response to a given situation based on the opponent's counter-response. In other words, the algorithm would self-learn from the knowledge of its opponent without any human intervention/supervision. A semantic network of multi-layered interconnected knowledge representation of the environment in which the self-learning algorithm would evolve was imperative for its application. The game of GO was the environment used to demonstrate how this unsupervised learning algorithm would evolve with time. The document describing this research work together with the fully implemented unsupervised learning algorithm was completed in 1994.

2