David Foster

In context of debate if large context #LLM models will eventually negate the need for #RAG, an article from Databricks has some good insights - • Longer contexts boost RAG accuracy... to a point • Performance often peaks at 32k-64k tokens, then declines • Top models (GPT-4, Claude 3.5) maintain consistency at scale • Others show unique failure modes (e.g., copyright concerns, summarization instead of Q&A) • Optimal context size varies by model and task • Lack of long-context training data may explain some issues Key takeaway: Long context and RAG are synergistic, but LLMs need refinement to fully leverage extended contexts. https://lnkd.in/eehdKtCm

17

Apparently, someone (maybe KPMG) is running a supply chain due diligence on pydantic on behalf of NASA. The thread is interesting. I link in the comment, but I summarise in a couple of points: - "This is so dumb. Do they know what OSS is?" Dumbness is not an issue in regulations. There was literally a supply chain attack recently on Linux, and if successful, a state actor would have gained access to half the internet (which makes you wonder what else is happening out there). You can read more about this if you search "hacker news xz," which is a fascinating story; I might write about it. - "KPMG ripping of NASA and the taxpayers" Maybe the astronauts and the rocket scientists don't want to spend their time on this? Outsourcing these are pretty standard to those poor people whose entire job is just this. (Shed a tear for their souls) - "Can you do it for free for NASA?" Why would he? They have an enormous budget. Pydantic must have a good place for that money. - "Did not OSS maintainers suffer enough?" They can ignore the request, and NASA will have no other choice but to either fork pydantic or recreate it from scratch. They have no other choice. They are not doing this because they have nothing better to do. - "AI will replace these people, can't wait for it." That's just mean. - "This is so dumb." They plan to send people to the Moon, not exactly a "move fast and break things" mission. Look at what happened when this principle was ignored at Boeing. NASA wants to ensure that if there is a bug or issue with pydantic, someone will be there to fix it. They also want to ensure that no nefarious actors have a chance to take down anything serious or at least have a contained blast radius. This is a negotiation. How much the maintainer can charge for this? What is the alternative? Recreate pydantic from scratch? How many lines of code? How many engineers and how much time to recreate? How much time does it take to review a fork and maintain compatibility? This will provide a BATNA ("Best Alternative to Negotiated Agreement) that you can use as a baseline. On the other hand, you have your costs to actually fulfil the above agreement. Some quoted lowball numbers, which indicate that the entire deal can be done in 1-2 weeks. It won't, and you will regret accepting the deal. There is a huge crisis in OSS, especially with single-person maintainers. Huge companies rely on these softwares, but because there is no business behind them, the individual probably can't or doesn't want to deal with the commercial side. They started doing it in their spare time and now things go out of hand. [continued in comments]

89

40 Comments

SYNTHETIC DATA On many occasions synthetic data is being discussed as a potential solution. Any sensible conversation about the use of synthetic data starts off by clarifying what we are using that data for: a) Training AI models b) Building Data Pipelines In the case of b) it seems that many do not understand that a hard part of building data pipelines especially on older data sources comes from data not being as expected and that many corner cases must be handled. That makes generating synthetic data for building data pipelines missing a major aspect of what is hard about building data pipelines. Of course it works well for scrambling to ensure privacy and copies to test volume resilience or real-time capabilities. In the case of a) it is a different discussion - still, however, with both pros and cons. There are some obvious invariances which are naturally handled by manipulated data (like rotation, scaling, and translation), while others like representativity is far less obvious - if interested in that part you might like the article linked to below .. #laerdalCPHlife #helpingsavelives #ai https://lnkd.in/dAM5VEUZ

12

One of the best blogs I've come across on quantization, with outstanding visualizations. Highly recommend a read for anyone looking to understand LLM quantization in more depth! 📐 - https://lnkd.in/e_WTgq6x At Felafax AI (YC S24), we are building Infra to make full precision multi-GPU fine-tuning easier. #AI #Infra #LLM

16

3 Comments

Text embeddings have become crucial for transforming textual data into dense vector representations (with context), enabling efficient processing across various modalities like text, images, and audio. This article provides a good summary on some of these (not all) embeddings available for text https://lnkd.in/gKp4ta9a 1. Gensim Word Embeddings (Free) Why Use It: Specializes in NLP with open-source models like Word2Vec and FastText, promoting transparency and ease of custom training. 2. Facebook Embeddings (Free) Why Use It: Trained on extensive corpora and supports over 100 languages, making it versatile for global applications. 3. AllenNLP Embeddings (Free) Why Use It: Provides NLP-focused embeddings like BERT and ELMo, with tools for fine-tuning and visualization. 4. OpenAI Embeddings (Paid) Why Use It: Comprehensive training on massive datasets allows for effective semantic understanding, making it suitable for advanced NLP tasks. Ideal for tasks requiring zero-shot learning. 5. MultiLingual BERT (Free) Why Use It: Handles multilingual datasets effectively, offering robust support for multiple languages. Perfect for applications requiring consistent performance across different languages, such as global customer support systems. 6. DistilBERT Base Uncased (Free) Why Use It: Smaller and faster than BERT, maintaining near-comparable performance with reduced resource usage. 7. MPNet V2 (Free) Why Use It: Excels in text similarity tasks with a Siamese architecture, providing accurate similarity measurements. 8. HuggingFace Embeddings (Free/Paid) Why Use It: Offers versatility with a wide range of embeddings (text, image, audio) and the ability to fine-tune on custom data. HF also provides wrappers to almost all open sourced models. 9. Longformer Base 4096 (Free) Why Use It: Supports long texts up to 4096 tokens, making it unique in handling lengthy documents without truncation. 10. RoBERTa (2022) (Free) Why Use It: Offers updated text representations with comprehensive training on recent data. 11. GTE-Base (Free) Why Use It: Offers a good balance of performance and resource requirements with a 768-dimensional embedding. 12. GTE-Large (Free) Why Use It: Provides high-quality embeddings with more detailed representations, useful for nuanced text analysis. Best for applications requiring in-depth text understanding, such as detailed content recommendations. 13. GTE-Small (Free) Why Use It: Optimized for speed with a smaller model size, making it efficient for applications needing quick processing. Useful for real-time systems where fast text similarity searches are critical. 14. E5-Small (Free) Why Use It: Combines compact size with good performance, making it suitable for fast, general-purpose text embedding tasks.

11

👉🏼 Evaluating large language models in theory of mind tasks 🤓 Michal Kosinski 👇🏻 https://lnkd.in/eB2YATpm 🔍 Focus on data insights: - 📊 Eleven large language models

2

A fairly short article with which I agree on one thing: pure #LLMs are too expensive to set up. There's no point in each entity creating everything 'from scratch'. HOWEVER the LLM craze is making possible for some companies that are reluctant to use 'classic' AI to get started !! An OLS is nice and mastered by everyone, and the same goes for LogitProbit, but we're talking about SVMs and Bayesian models. To conclude, the integration of predictive analytics into major sectors will most likely streamline operations thus it will also drive economic #growth, making it a win-win for businesses and the broader community ! Eric Siegel #Data #PredictiveAI #DigitalTransformation #LuxembourgTech

11

2 Comments

👉🏼 Dual Causes Generation Assisted Model for Multimodal Aspect-Based Sentiment Classification 🤓 Rui Fan 👇🏻 https://lnkd.in/eC5m_4qv 🔍 Focus on data insights: - The proposed MDCA method tracks underlying causes behind expressed sentiments, providing reasoning cause (RC) and direct cause (DC) to explain emotions. - Utilizes large language models (LLMs) and visual-language models to construct MABSC datasets with RC and DC. - Implements a multitask learning framework leveraging cause data to train a generative model for sentiment prediction. 💡 Main outcomes and implications: - MDCA model achieves state-of-the-art performance in MABSC benchmark datasets. - Small fine-tuned model shows superior adaptability to MABSC compared to larger models like ChatGPT and BLIP-2. 📚 Field significance: - Advances in understanding sentiment analysis in multimodal data. - Enhances accuracy and interpretability of sentiment prediction models. 🗄️: #Multimodal #SentimentAnalysis #DataInsights #GenerativeModel

1

Language models do not reason as they are not designed for it. They rigidly pull together statistical relationships in a feed-forward manner. While some language models, like BERT, are trained bidirectionally, most language models are not. BERT is trained to learn bidirectional representations from unlabeled corpora by joint conditioning on the left and right context in all layers. In contrast, OpenAI’s models adopt a left-to-right architecture, where each token can only attend to previous tokens in the self-attention layer. This process relies on patterns learned from the training dataset, utilizing a sophisticated data lookup rather than conventional logical reasoning. These unique training methods explain the Reversal Curse, where language models learn different mappings for the same factual association in opposite directions. In the figure below, GPT-4 correctly identifies the name of Tom Cruise’s mother (left). However, when prompted with the mother’s name, it fails to retrieve “Tom Cruise” (right). The model's knowledge is one-dimensional and accessed in specific ways. For instance, Mary Lee Pfeiffer's son is underrepresented in the training data, but Tom Cruise's mother is not. GPT-4 will correctly answer “Who is Mary Lee Pfeiffer’s son?” if you first prompt the model with “Tom Cruise’s mother is Mary Lee Pfeiffer.” This demonstrates the model's limited abilities for in-context learning. It has not encoded the response to “Who is Mary Lee Pfeiffer’s son” in its parameters but can use information in the context to find the answer. This is precisely why chain-of-thought prompting strategies work: these models can discover answers within the context but struggle in a zero-shot environment.

50

7 Comments

👉🏼 Potential roles of large language models in production of systematic reviews and meta-analyses 🤓 Xufei Luo 👇🏻 https://lnkd.in/eVTxQ6Ce 🔍 Focus on data insights: - Large language models (LLMs) like ChatGPT are increasingly utilized in medical research. - LLMs can expedite steps in systematic reviews such as defining clinical questions, literature search, and information extraction. - Attention to transparent reporting and avoiding academic misconduct is crucial when using LLMs. 💡 Main outcomes and implications: - Enhanced efficiency and resource conservation in systematic reviews with the use of LLMs. - Need for clear differentiation between genuine and false content generated by LLMs. - Future research directions to maximize the benefits of LLMs in systematic reviews. 📚 Field significance: - Advancements in utilizing LLMs for systematic reviews and meta-analyses. - Importance of ethical considerations and quality control when integrating LLMs in research processes. 🗄️: [#medicalresearch #systematicreviews #LLMs #efficiency #ethics]

1

1 Comment