Abigail Haddad

Foundational models, or large language models (LLMs), train at the scale of the entire internet to acquire the vocabulary, syntax, and narrative structure of human language. Fine-tuning involves using a smaller, domain-specific dataset to grant additional specificity to a pre-existing foundational model that has already acquired patterns and features from massive underlying training datasets. By utilizing the existing knowledge embedded in a pre-trained language model, we unlock high-performance language understanding, but fine-tuning helps to get the right answer - not just an answer that sounds right - with a minimum of hallucinations or incorrect info. In healthcare, out-of-the-box, foundational models are often too generic to sufficiently answer the precision of a healthcare query. Foundational language models are trained on a massive scale of data, which includes jokes, incorrect information, misunderstandings, and misinformation, along with the sum total of human knowledge. The problem is, if you ask a large language model to give you factually correct information when it has been trained on BOTH accurate and inaccurate information - you’re playing a bit of a dangerous game. Enter fine-tuning language models. In the old days, we used to build a single model for a single task - for example, classifying procedures into evidence-based or not as laid out by clinical practice guidelines, for patients with particular disease markers. Each disease would have its own model, which took a long time to build and didn’t scale particularly well, and we struggled with the sparsity and inherent high-dimensionality of healthcare data. By fine-tuning LLMs, however, we're able to specialize those foundational models to achieve greater domain specificity and specialization, leverage the unique predictive and planning capabilities of LLMs, and cut down on the incorrect information that such models inevitably produce, while still preserving the underlying language prowess of the LLM. If improved accuracy and specificity wasn't enough of a selling point, there's another notable upside to fine-tuning foundational models for use in a clinical context. In terms of computational efficiency, using LLMs out of the box on specific tasks can require massive compute (and spend) that amounts to computational overkill for the problem. Some LLMs charge per query, some by compute load or time - either way, as a business running LLMs in a production environment, minimizing the number of LLM calls means optimizing your compute expenditures, too. When LLMs fail at scale in healthcare, it's usually along the fault line of failure to specify adequately. Fine-tuning large language models along domain-specific lines is one way to effectively productize genAI for healthcare use cases. #genAI #LLM #ai #healthcareAI #healthtech #largelanguagemodels #generativeAI #chatGPT

61

2 Comments

Great piece on how the public sector can turn sensible risk management and pragmatism into obstructive tick-box processes - and why that's a real risk in how we adopt and use AI for public services and social good. The key line for me: 'Well-meaning and well-written legislation originates at the top of a very tall hierarchy, and as it descends, the flexibility that its authors intended degrades. Laws often have an effect entirely different from what lawmakers intended because of this cascade of rigidity.' I've seen it happen and it risks widening the divide between public and private sector. Government has done impressive things in the field of AI safety in the past year, but the community must be mindful of context and capacity as we collectively move from analysis and guidelines towards regulation. And a great point worth bearing in mind, too, that we must consider the risk not only of 'irresponsible use, but [also] irresponsible lack of use'. #aiingoverment #aisafety #riskmanagement

4

I'm exploring how Splink can be used for address matching. Does anyone know of any labelled datasets of messy address data that could be used to measure performance? (i.e. labelled to indicate which addresses really match) FWIW there are some rough demos of matching some open address data (EPC to price paid data, and food hygiene score to price paid data) here: https://lnkd.in/e_wHRDcm But without labels, difficult to quantify how good it is

19

6 Comments

This post digs deeper into DCN V3 architecture and intuition behind math. This is written for folks who are technical but not necessarily familiar with Deep Learning or CTR prediction literature. 🤖

25

Metrics are gameable and quite amusingly at times. A friend once told me what happened at a large tech giant. The bonus of Team-1 depended on improving the time complexity of the recommendation engine. The bonus of team-2 was dependent on improving the space complexity of the data fetching service. In the appraisal month, Team-1 started allocating a larger cache space to improve the time complexity and increase bonuses. Team-2 focussed on the brute force search approaches to improve the space complexity required by the algorithm. Both metrics improved for respective teams. The time and space complexity of the entire pipeline was preserved. Things went back to being as they were after bonuses were given out. Join me for a webinar on Tuesday, 26th November to understand how to select metrics that are not gameable in experimentation: https://lnkd.in/gszag6HK #metrics #experimentation #gameability

17

2 Comments

- SwiftKV is a novel model transformation and distillation procedure (by folks at Snowflake) designed to optimize large language model (LLM) inference. Very cool findings in the paper (as presented in the vLLM meetup on 11/13). - SingleInputKV: Prefills later layers' KV cache using earlier layer outputs, reducing computation on prompt tokens. The key point is that the first few layers of attention in LLMs are similar (nice find), allowing SwiftKV's SingleInputKV mechanism to skip computations in later layers during inference. By using a single earlier layer's output to compute the KV cache for subsequent layers, it significantly reduces both computation (by 50%) and memory usage (by 62.5%) without sacrificing model performance. - AcrossKV: Merges KV caches of neighboring layers to minimize memory usage and increase batch size for higher throughput. - For models like Llama-3.1-8B and 70B, SwiftKV reduces prefill compute by 50%, memory usage by 62.5%, and achieves up to 2× higher throughput and 60% lower time per token. https://lnkd.in/guhEBW5G

62

Low-code and no-code ELT tools have been around for a while, making the ELT process easy for non-technical folks. This article explores how low-code and no-code platforms are revolutionizing the Extract, Load, and Transform (ELT) process by enabling quicker data integration and transformation with minimal coding. It emphasizes the benefits of these tools, such as increased efficiency, cost reduction, and ease of use, making them accessible to both technical and non-technical users in various industries. https://lnkd.in/deUdgsYV Airbyte Fivetran Integrate.io Matillion Qlik

3

Kolmogorov–Arnold Networks seem like a paradigm shift in ML architectures. Curious to see how they evolve. "While MLPs have fixed activation functions on nodes ("neurons"), KANs have learnable activation functions on edges ("weights"). KANs have no linear weights at all -- every weight parameter is replaced by a univariate function parametrized as a spline. " https://lnkd.in/gdPk_qCQ

29

I often find myself wanting to reference a "greedy algorithm" but know I will be misunderstood by the anti-market commentariat that makes up a surprisingly large part of LinkedIn. Greedy algorithms are those that select locally optimal paths without regard to the big picture. They are useful because they are low complexity and require very little data. The classic example is making change. Subtraction is hard, so it isn't too critical to say that most cashiers couldn't make change if they had to use it. Instead, you make change by adding as many quarters as you can, then dimes, nickels, and finally pennies. 82 cents takes 3 quarters, 1 nickel, and 2 pennies. In general, finding the smallest number of coins to make change would be equivalent to the knapsack problem and therefore is NP-Complete, so the greedy algorithm is a big win. Not all greedy algorithms guarantee global optimization. In fact, lets look at the above 82 cent example when we run out of nickels. Using our original algorithm, we'd grab 3 quarters and 7 pennies, for 10 total coins. But we can make change with 2 quarters, 3 dimes, and 2 pennies, totaling 7 coins. The coin denominations are selected to make the greedy algorithm find the global optimization. I'm amazed at how many of life's problems are caused by the mismatch between the locally optimal solution and the global one. I recently posted about Boeing (and most other companies) eschewing vertical integration to focus on only the places they add unique value. Every step down this path makes sense, but some day you wake up and find out you don't actually do anything anymore, because you've shed the whole business one step at a time. This post is actually about investment advice, though. Or more specifically how index funds are going to crash the stock market. Index funds are the best choice for each individual investor. Low fees and high diversity are shown to win time and again. But most money is being passively invested this way. And that means when NVIDIA goes up, we are all holding more of it to reflect the larger share of total market capitalization. Prices are supposed to be market signals, but the end state of indexed investing is that stock market price only indicates stock market price. What should you do about this? Well, the best move for you is to be in index funds, because the alternative is sub optimal. But the global outcome after the aggregation of all these locally-optimal decisions is sub optimal. Very suboptimal. We're out of nickels and it isn't going to end any better for the stock market than it did for Boeing.

35

14 Comments

Interesting #nytimes opinion piece that describes the #datascience of a key problem in america, i.e. educational inequality. Yes, the system is rigged for the rich. This is big, big problem. A real question to ask any elite degree holder is “how much money do daddy and mommy have?”, vs what did you study. As long as that is the most important question, there is deep trouble in the system at the foundation. `According to Daniel Markovits of Yale, on the SAT, “students from families earning over $200,000 per year (roughly the top 5 percent) score 388 points higher than students from families earning less than $20,000 per year (roughly the bottom 20 percent).” According to a 2017 studyled by Raj Chetty of Harvard, students from families in the top 1 percent of earners were 77 times as likely to get into the Ivy League as students from families making less than $30,000 a year. In that year students from the top income quintile were about 16 times as numerous at the University of North Carolina at Chapel Hill as students from the bottom quintile.` https://lnkd.in/dDcsK__U

3

Fine-Tuning in Large Language Models: A Transformative Approach to Task-Specific Adaptation Abstract The rise of large language models (LLMs) has revolutionized natural language processing (NLP) by offering impressive generalization across a variety of tasks. However, their generalized nature requires adaptation to specific tasks through fine-tuning techniques. This paper explores the evolution of fine-tuning, focusing on instruction tuning, label-supervised fine-tuning, and Low-Rank Adaptation (LoRA). While these techniques provide significant performance improvements, they each face limitations in certain applications. We discuss the shortcomings of current fine-tuning methods and propose directions for future research to enhance the efficiency, adaptability, and precision of LLM fine-tuning for diverse NLP tasks. 1. Introduction The ability of large language models to process and generate human-like text has opened new avenues for various NLP applications, from chatbots to content generation. While these models are trained on vast and diverse datasets, they often require fine-tuning to perform optimally on specific tasks, such as text classification, sentiment analysis, or named entity recognition (NER). Fine-tuning involves refining the model's parameters using task-specific data, allowing it to specialize in new domains without needing to train from scratch. This adaptability has been key to the success of LLMs in real-world applications. However, as more sophisticated NLP tasks emerge, traditional fine-tuning methods reveal certain limitations that need addressing. This paper will explore fine-tuning techniques—instruction tuning, label-supervised fine-tuning, and Low-Rank Adaptation (LoRA)—to better understand their strengths and weaknesses. 2. Related Work Instruction Tuning Instruction tuning is one of the most popular methods for fine-tuning LLMs. It involves training models to follow human instructions across a range of tasks, allowing the model to interpret prompts and generate responses based on task-specific instructions. Instruction tuning has been particularly effective in zero-shot and few-shot learning contexts, where models are tasked with generalizing to unseen tasks using minimal examples. This method allows LLMs to tackle creative tasks such as generating stories, writing essays, and engaging in dialogue generation with minimal additional training. However, while instruction tuning is ideal for tasks involving natural language generation, its performance is limited when it comes to structured tasks like text classification and NER. In such cases, where accuracy and precision are paramount, instruction tuning often underperforms. This is because it tends to focus on generating fluent and diverse text, rather than making accurate predictions based on specific labels . Label-Supervised Fine-Tuning

4

We just released a blog on whether the use of multimodal feature transfer can detect deception (phenomena like sarcasm, irony, and condescension) in conversation. The short answer... Yes. This particular write-up profiles a subset of our work at Symbl.ai on multimodal large language models, and is research led by one of our summer interns, Benjamin R. (from Prof. Larry Heck's AVA Lab at Georgia Institute of Technology). We have a paper currently under submission on this work, and we can't wait to share the full details with all of you soon. Blog Link: https://lnkd.in/g6m_wFDi #multimodal #featuretransfer #LLMs #deception #conversation #sarcasm #irony #language #AI #ML #NLP #LargeLanguageModels #SummerInternship #ResearchPapers #ACL #Llama2 #SymblDotAI

16

The big plan all along was to replace human beings right? Otherwise why spend hundreds of billions on neural nets trained with effectively stolen scraped data exposing you to all kinds of liability? Here we see the plans are now more like, can it do summaries of notes if we modify long-standing human workflows?

4

Something new came out and it's not a movie (yes there are some new movies that just came out but that's not the point), it's called xLSTM. If you don't remember what LSTM is, initially it's for neural network especially Recurrent Neural Network. Here's what happen in RNN, we have an input, a processing network, and an output. The output gets feedback into the input along with new input again and goes through processing and then another new output. As time goes by, it has become harder to remember everything, hence the LSTM (Long Short Term Memory) comes to rescue, we can now use LSTM to choose what to forget what to remember. So in the processing network, LSTM added a state. A state consist of forget state, input state, and output state. These states then go through processing and then we have new output again, and so on and so forth. Clear as mud? xLSTM derived from LSTM, if you're interested see here: xLSTM info: https://lnkd.in/gsbw4iAZ https://lnkd.in/gxJbcBjd Original paper here: https://lnkd.in/gqvcuJwq

1

Wrote a blog on "time-limiting" metrics, fixing the length of the window over which we compute metrics (or, alternatively, making sure the window-length is "small" relative to experiment duration). It turns out if we don't, a lot of things we take for granted in experiments aren't true. https://lnkd.in/gA-63-Ye

52

13 Comments

I've been a bit dismissive of late regarding the use of semi-open community licensing with GenAI LLMs of late that feel a bit open ended (I'm looking at you, Meta Llama 3.1). I totally understand the value behind these licenses in protecting the model maker's intellectual property (or shall we say "Materials"). But when it comes to generating, not just protecting value, "community" licensing falls well short of free and open source software (FOSS), especially in terms of building trust among the broader AI community. Interestingly, with this week's introduction of a new text to image model from Black Forest Labs (FLUX.1), perhaps we're now starting to see a middle way for vendors to walk between the two worlds of proprietary and FOSS. With this release, we've basically go three models: - FLUX.1 pro, which is their base model, available only via API under a commercial license just like API services from OpenAI, Anthropic, and Cohere. - FLUX.1 dev, a guidance-distilled variant of pro that's available via a non-commercial license that looks a bit like Meta's Community License in terms of trying to control associated model materials. - FLUX.1 schnell, which is a step-distilled variant of pro that's available under the well-regarded and highly permissive Apache 2.0 FOSS license. Both dev and schnell models use the same weights, which are nicely available under the Apache 2.0 license. Certainly, we're talking about a tiered solution that becomes increasingly prohibitive as it increases in capability, but at least with this kind of approach, the model license more closely matches the use case. For example, if I'm looking to create a derivative product that competes with pro, I'm not going to be able to do that with FLUX.1 dev or FLUX.1 schnell. But at least I'll be able to perform useful research or build a viable commercial product using the same fundamental technology as FLUX.1 pro. https://lnkd.in/e7fkbMJX #AI Omdia Informa Tech #GenAI #Llama #FLUX

6

Just read about Contextual Retrieval on Anthropic's blog—it's fascinating how much precision it adds to chunked document retrieval! The idea is simple yet powerful: adding context to individual document chunks before embedding them or creating a BM25 index. For example, instead of a standalone chunk like "The company's revenue grew by 3% over the previous quarter," the contextualized version might be: "This chunk is from an SEC filing on ACME Corp’s Q2 2023 performance; the previous quarter's revenue was $314 million. The company's revenue grew by 3% over the previous quarter." The added context frames the information, making it more relevant and easier for the retrieval model to understand the chunk’s significance within the larger document. 🔥 Key takeaways for implementation: - Automatic Context Generation: Anthropic uses Claude with prompts tailored to condense document-wide insights into chunk-specific context, ensuring high-quality retrieval without manual annotation. - Flexible Tuning: Chunk size, embedding models, and custom prompts tailored to the domain all impact performance—so testing different setups is crucial. - Reranking for Precision: Retrieving the top-150 chunks, then reranking and narrowing to the top-20, helps reduce processing load and enhances relevance, combining accuracy with efficiency. Ultimately, this approach integrates methods like contextual embeddings, BM25, and reranking for stacked performance gains, transforming standard document retrieval into a highly precise tool. Highly recommend checking out Anthropic's post if you're interested in advanced retrieval techniques! #ContextualRetrieval #AI #MachineLearning #InformationRetrieval #NaturalLanguageProcessing #NLP #Embeddings #RAG #BM25 #Claude #DataScience #DocumentSearch

24

I like some things about this year's NeurIPS checklist, especially the emphasis on statistics, but I think that should be expanded beyond just saying people should provide error bars or standard deviations at a minimum. That's a low bar. That said, another checklist item requires providing the license and terms of use for every dataset, which feels a bit excessive, especially since I'm using 11 datasets in one of my papers. I went ahead and added the licenses to my appendix for the datasets that have them, but as best I can tell, many of them don't have them (e.g., CIFAR-10, CIFAR-100, some of the Oxford datasets, etc.).

29

3 Comments