Shishir Bashyal

Text embeddings have become crucial for transforming textual data into dense vector representations (with context), enabling efficient processing across various modalities like text, images, and audio. This article provides a good summary on some of these (not all) embeddings available for text https://lnkd.in/gKp4ta9a 1. Gensim Word Embeddings (Free) Why Use It: Specializes in NLP with open-source models like Word2Vec and FastText, promoting transparency and ease of custom training. 2. Facebook Embeddings (Free) Why Use It: Trained on extensive corpora and supports over 100 languages, making it versatile for global applications. 3. AllenNLP Embeddings (Free) Why Use It: Provides NLP-focused embeddings like BERT and ELMo, with tools for fine-tuning and visualization. 4. OpenAI Embeddings (Paid) Why Use It: Comprehensive training on massive datasets allows for effective semantic understanding, making it suitable for advanced NLP tasks. Ideal for tasks requiring zero-shot learning. 5. MultiLingual BERT (Free) Why Use It: Handles multilingual datasets effectively, offering robust support for multiple languages. Perfect for applications requiring consistent performance across different languages, such as global customer support systems. 6. DistilBERT Base Uncased (Free) Why Use It: Smaller and faster than BERT, maintaining near-comparable performance with reduced resource usage. 7. MPNet V2 (Free) Why Use It: Excels in text similarity tasks with a Siamese architecture, providing accurate similarity measurements. 8. HuggingFace Embeddings (Free/Paid) Why Use It: Offers versatility with a wide range of embeddings (text, image, audio) and the ability to fine-tune on custom data. HF also provides wrappers to almost all open sourced models. 9. Longformer Base 4096 (Free) Why Use It: Supports long texts up to 4096 tokens, making it unique in handling lengthy documents without truncation. 10. RoBERTa (2022) (Free) Why Use It: Offers updated text representations with comprehensive training on recent data. 11. GTE-Base (Free) Why Use It: Offers a good balance of performance and resource requirements with a 768-dimensional embedding. 12. GTE-Large (Free) Why Use It: Provides high-quality embeddings with more detailed representations, useful for nuanced text analysis. Best for applications requiring in-depth text understanding, such as detailed content recommendations. 13. GTE-Small (Free) Why Use It: Optimized for speed with a smaller model size, making it efficient for applications needing quick processing. Useful for real-time systems where fast text similarity searches are critical. 14. E5-Small (Free) Why Use It: Combines compact size with good performance, making it suitable for fast, general-purpose text embedding tasks.

11

Foundational models, or large language models (LLMs), train at the scale of the entire internet to acquire the vocabulary, syntax, and narrative structure of human language. Fine-tuning involves using a smaller, domain-specific dataset to grant additional specificity to a pre-existing foundational model that has already acquired patterns and features from massive underlying training datasets. By utilizing the existing knowledge embedded in a pre-trained language model, we unlock high-performance language understanding, but fine-tuning helps to get the right answer - not just an answer that sounds right - with a minimum of hallucinations or incorrect info. In healthcare, out-of-the-box, foundational models are often too generic to sufficiently answer the precision of a healthcare query. Foundational language models are trained on a massive scale of data, which includes jokes, incorrect information, misunderstandings, and misinformation, along with the sum total of human knowledge. The problem is, if you ask a large language model to give you factually correct information when it has been trained on BOTH accurate and inaccurate information - you’re playing a bit of a dangerous game. Enter fine-tuning language models. In the old days, we used to build a single model for a single task - for example, classifying procedures into evidence-based or not as laid out by clinical practice guidelines, for patients with particular disease markers. Each disease would have its own model, which took a long time to build and didn’t scale particularly well, and we struggled with the sparsity and inherent high-dimensionality of healthcare data. By fine-tuning LLMs, however, we're able to specialize those foundational models to achieve greater domain specificity and specialization, leverage the unique predictive and planning capabilities of LLMs, and cut down on the incorrect information that such models inevitably produce, while still preserving the underlying language prowess of the LLM. If improved accuracy and specificity wasn't enough of a selling point, there's another notable upside to fine-tuning foundational models for use in a clinical context. In terms of computational efficiency, using LLMs out of the box on specific tasks can require massive compute (and spend) that amounts to computational overkill for the problem. Some LLMs charge per query, some by compute load or time - either way, as a business running LLMs in a production environment, minimizing the number of LLM calls means optimizing your compute expenditures, too. When LLMs fail at scale in healthcare, it's usually along the fault line of failure to specify adequately. Fine-tuning large language models along domain-specific lines is one way to effectively productize genAI for healthcare use cases. #genAI #LLM #ai #healthcareAI #healthtech #largelanguagemodels #generativeAI #chatGPT

61

2 Comments

👉🏼 Reliability of large language models for advanced head and neck malignancies management: a comparison between ChatGPT 4 and Gemini Advanced 🤓 Andrea Lorenzi 👇🏻 https://lnkd.in/eSW4Admy 🔍 Focus on data insights: - ChatGPT 4 generally outperformed Gemini Advanced in adherence to guidelines and comprehensive treatment planning. - Higher AIPI scores were observed with ChatGPT 4, indicating better overall performance. 💡 Main outcomes and implications: - Both large language models showed potential in aiding multidisciplinary management of head and neck oncology. - Discrepancies in critical areas emphasize the need for further refinement and validation against current clinical standards. 📚 Field significance: - Growing role of AI in enhancing clinical decision-making. - Necessity for continuous updates and validation to fully integrate AI into healthcare practices. 🗄️: [#largeLanguageModels #headAndNeckMalignancies #AIclinicalDecisionMaking]

1

1 Comment

Some very interesting points from Yossi Matias that worth looking at, and the main one is: "Learn to code" remains crucial, even as AI advances. Here are some additional points: • Understanding tech basics is vital for everyone, not just software engineers • AI will enhance fields like biology, chemistry, and medicine • Healthcare presents a "huge opportunity" for AI transformation • Education could be revolutionized with AI-powered interactive tools • AI can be integrated in life-saving applications like Google's flood forecasting • Junior professionals may face challenges, but opportunities still exist • AI is becoming "both a tool and a science" https://lnkd.in/d5XSWb5Z #artificialintelligence #ai #chatbot #programming #healthcare #aiinhealthcare

1

🚨 AI's Self-Referential Risk: The "Snake Eating Its Own Tail" 🚨 In recent conversations about Responsible AI, discussions often focus on two main concerns: bias and the terminator scenario—i.e., the autonomy of AI systems making well-intentioned but horrific decisions. Admittedly, I am by no means an expert on responsible AI, but these are common themes. However, another crucial risk is emerging: as AI models become more powerful, there's a growing danger of them feeding off their own outputs, leading to a decline in content quality. As we increasingly use AI to generate the content we would typically create ourselves, this issue arises almost automatically. The risk is that we get something akin to a "snake eating its own tail," resulting in an explosion of language quantity that may become increasingly incoherent and repetitive. A recent article, AI trained on AI churns out gibberish garbage | Popular Science https://lnkd.in/gKjxt5y6 by Mack DeGeurin, delves into this issue, highlighting the potential for AI models to degrade over time when trained on their own outputs. This can lead to: Content Homogenization: A risk of content becoming repetitive and losing its diversity. Loss of Nuance: Models might struggle to retain the depth and context of human experiences, resulting in shallow or nonsensical outputs. 🌟 How might we address these issues? Incorporate Diverse Data: Curating diverse data sets and creating more models, especially smaller ones with their own strengths and weaknesses, could help reduce gibberish. Each model having its own focus might cut down on repetitive and incoherent content. Encourage Human Creativity: It's crucial to continue valuing and integrating original human contributions alongside AI advancements. This seems like the real solution and probably the biggest risk if done incorrectly. While LLMs can feel more creative in some ways (in fact, the #1 use case for LLMs is to use them to ideate), they can also become unoriginal very quickly. I don't have a definitive answer here, but as I think about how to introduce my kids to AI tools, these concerns are top of mind. #AI #MachineLearning #Creativity #ContentCreation #ResponsibleAI #Innovation

11

3 Comments

👉🏼 RDguru: A Conversational Intelligent Agent for Rare Diseases 🤓 Jian Yang 👇🏻 https://lnkd.in/eSCm_MkQ 🔍 Focus on data insights: - 📈 RDguru outperforms ChatGPT in generating authoritative descriptions, highlighting its enhanced accuracy in knowledge Q&A tasks. - 🔍 A novel multi-source fusion diagnostic model shows a significant 5.47% improvement in diagnostic recall compared to PheLR. - 🧬 The use of advanced algorithms facilitates more effective automated phenotype annotation and differential diagnosis in rare diseases. 💡 Main outcomes and implications: - 🏥 RDguru demonstrates the potential to significantly improve clinical decision-making processes in rare disease diagnostics. - 💬 The conversational agent can enhance patient interactions through evidence-based Q&A, leading to better-informed medical consultations. - ⚙️ Integration of multi-source diagnostic strategies could pave the way for the development of more robust AI tools in healthcare settings. 📚 Field significance: - 🚀 This study positions RDguru as a promising solution for addressing the challenges of diagnosing rare diseases, particularly given the limitations of traditional methods. - 🌍 The findings may inspire further research into the implementation of LLMs in clinical environments, promoting wider acceptance and use of AI technologies in healthcare. - 🔒 Ensuring accuracy and reliability in AI outputs is crucial for safeguarding patient safety and enhancing trust in AI applications within medicine. 🗄️: [#RareDiseases #AIAssistedDiagnosis #ClinicalDecisionSupport #MachineLearning #HealthTech #NaturalLanguageProcessing #DiagnosticAccuracy]

1

👉🏼 The consequences of generative AI for online knowledge communities 🤓 Gordon Burtch 👇🏻 https://lnkd.in/gAqspcch 🔍 Focus on data insights: - Analyzed data from Stack Overflow and Reddit developer communities between October 2021 and March 2023 - Documented ChatGPT's influence on user activity in Stack Overflow - Observed declines in website visits and question volumes at Stack Overflow, particularly in topics where ChatGPT excels 💡 Main outcomes and implications: - Significant impact of generative AI technologies on online knowledge communities - Importance of social fabric in mitigating community-degrading effects of LLMs like ChatGPT - Decline in participation on Stack Overflow concentrated among newer users 📚 Field significance: - Revolutionizing information acquisition and content production - Potential to impact participation and content production in online knowledge communities 🗄️: [#generativeAI #onlinecommunities #datainsights]

1

1 Comment

Quantized embeddings offer significant advantages for NLP models, including reduced storage requirements, faster computations, lower energy consumption, and improved model interpretability. They are becoming increasingly important for enabling efficient and scalable NLP on resource-constrained devices and cloud-based applications. Read my latest article below

6

👉🏼 Predicting glaucoma before onset using a large language model chatbot 🤓 Xiaoqin Huang 👇🏻 https://lnkd.in/eehWZ8E5 🔍 Focus on data insights: - ChatGPT4.0 demonstrated an accuracy of 75% in predicting conversion to glaucoma one year before onset. - AUC of 0.67 and weighted F1 score of 0.77 were achieved by ChatGPT4.0. - ChatGPT3.5 provided an accuracy of 61% and a sensitivity of 64% in predicting glaucoma development. 💡 Main outcomes and implications: - The performance of ChatGPT4.0 was reasonable in forecasting glaucoma development before onset. - Large language models (LLMs) show promise in enhancing glaucoma research capabilities and clinical care. 📚 Field significance: - Future exploration of ophthalmology-specific LLMs utilizing multi-modal data and active learning is crucial for improved integration with clinical practice. 🗄️: [#glaucoma #language #model #data #insights]

1