Dr. David Ng

Interesting paper about the limitations of the generative AI. Actually Yann LeCun talks about this limitation as inherited problem in the existing models for long time. Knowing the limitation helps to align expectations. https://lnkd.in/dCid9B_J #generativeai #llm #artificailintelligence #ai

7

3 Kommentare

Kolmogorov–Arnold Networks seem like a paradigm shift in ML architectures. Curious to see how they evolve. "While MLPs have fixed activation functions on nodes ("neurons"), KANs have learnable activation functions on edges ("weights"). KANs have no linear weights at all -- every weight parameter is replaced by a univariate function parametrized as a spline. " https://lnkd.in/gdPk_qCQ

29

👉🏼 Evaluation of Rhinoplasty Information from ChatGPT, Gemini, and Claude for Readability and Accuracy 🤓 Monica K Rossi Meyer 👇🏻 https://lnkd.in/e-6j2hpj 🔍 Focus on data insights: - 📊 ChatGPT demonstrated superior accuracy and overall quality in responses compared to its counterparts Gemini and Claude. - ✍️ All three chatbots provided responses rated as neutral to incomplete, indicating a gap in exhaustive information dissemination. - 🏷️ Use of medical jargon was prevalent across all platforms, with readability scores assessed at a college level. 💡 Main outcomes and implications: - ⚖️ Surgeons must critically evaluate the information provided by chatbot platforms due to potential incompleteness and accuracy issues. - 📚 There is an opportunity for improving chatbot technology through training on evidence-based medical recommendations to enhance educational effectiveness. - 🧩 The findings highlight a need for continuous monitoring and development of AI tools in healthcare to ensure reliable patient education resources. 📚 Field significance: - 🔬 The study underscores the importance of accurate medical information accessibility in the age of AI and digital resources. - 📋 It prompts further exploration into integrating AI into medical education while maintaining rigorous quality control standard practices. - 🛠️ The technology's potential for contribution in education suggests innovative pathways for enhancing surgeon-patient communication through AI-enabled platforms. 🗄️: [#rhinoplasty #AIinHealthcare #medicaleducation #ChatGPT #Gemini #Claude #readability #accuracy #patienteducation #digitalhealth]

Exciting news for the ones who are interested in quantum, now there is an alternate to virtual qubits, called mechanical qubits that has been recently developed by ETH Zürich, very interesting paper to read. From the way they explained how it is built, it sounds rather complex and expensive process to scale up, but nevertheless solves the problem of virtual qubits to disappear in no time. https://lnkd.in/eZCfUUCX

19

👉🏼 Potential roles of large language models in production of systematic reviews and meta-analyses 🤓 Xufei Luo 👇🏻 https://lnkd.in/eVTxQ6Ce 🔍 Focus on data insights: - Large language models (LLMs) like ChatGPT are increasingly utilized in medical research. - LLMs can expedite steps in systematic reviews such as defining clinical questions, literature search, and information extraction. - Attention to transparent reporting and avoiding academic misconduct is crucial when using LLMs. 💡 Main outcomes and implications: - Enhanced efficiency and resource conservation in systematic reviews with the use of LLMs. - Need for clear differentiation between genuine and false content generated by LLMs. - Future research directions to maximize the benefits of LLMs in systematic reviews. 📚 Field significance: - Advancements in utilizing LLMs for systematic reviews and meta-analyses. - Importance of ethical considerations and quality control when integrating LLMs in research processes. 🗄️: [#medicalresearch #systematicreviews #LLMs #efficiency #ethics]

1

1 Kommentar

👉🏼 RDguru: A Conversational Intelligent Agent for Rare Diseases 🤓 Jian Yang 👇🏻 https://lnkd.in/eSCm_MkQ 🔍 Focus on data insights: - 📈 RDguru outperforms ChatGPT in generating authoritative descriptions, highlighting its enhanced accuracy in knowledge Q&A tasks. - 🔍 A novel multi-source fusion diagnostic model shows a significant 5.47% improvement in diagnostic recall compared to PheLR. - 🧬 The use of advanced algorithms facilitates more effective automated phenotype annotation and differential diagnosis in rare diseases. 💡 Main outcomes and implications: - 🏥 RDguru demonstrates the potential to significantly improve clinical decision-making processes in rare disease diagnostics. - 💬 The conversational agent can enhance patient interactions through evidence-based Q&A, leading to better-informed medical consultations. - ⚙️ Integration of multi-source diagnostic strategies could pave the way for the development of more robust AI tools in healthcare settings. 📚 Field significance: - 🚀 This study positions RDguru as a promising solution for addressing the challenges of diagnosing rare diseases, particularly given the limitations of traditional methods. - 🌍 The findings may inspire further research into the implementation of LLMs in clinical environments, promoting wider acceptance and use of AI technologies in healthcare. - 🔒 Ensuring accuracy and reliability in AI outputs is crucial for safeguarding patient safety and enhancing trust in AI applications within medicine. 🗄️: [#RareDiseases #AIAssistedDiagnosis #ClinicalDecisionSupport #MachineLearning #HealthTech #NaturalLanguageProcessing #DiagnosticAccuracy]

1

We just posted on arXiv a new preprint of a BRKGA titled “The connected Grundy coloring problem: Formulations and a local-search enhanced biased random-key genetic algorithm.” Here is a summary of the paper: The paper introduces the connected Grundy coloring problem, which involves finding a proper vertex coloring for a graph using a first-fit heuristic on a connected vertex sequence, aiming to maximize the number of colors used. The authors propose two integer programming (IP) formulations and a local-search-enhanced biased random-key genetic algorithm (BRKGA) to solve the problem. The first IP formulation partitions vertices into color classes, while the second leverages representative vertices to reduce symmetry. The BRKGA includes a novel neighborhood-based local search and a reset mechanism. Computational experiments reveal that the problem is computationally challenging for the IP formulations, but the representative-based formulation performs better. The BRKGA demonstrates the ability to generate high-quality solutions efficiently, even for large instances, and can also be extended to solve the general (non-connected) Grundy coloring problem effectively. https://lnkd.in/eM_yAq2p

20

Conditional Computation in LLMs (MoEs): Understanding the Concept of MoEs The Mixture of Experts (MoE) works on the idea of conditional computation, allowing the model to only run through some parts (an Expert) for a given input compared to a dense model where all parameters are used. It consists of a gating mechanism that selects the experts for the given input, enabling the model to scale without increasing the required computation. Popular LLM architectures like Mixtral and Grok use the same concept. What is MoEs? The Mixture of Experts (MoE) makes a simple modification to the usual architecture by replacing all or some of the Feed Forward layers with an MoE layer. The MoE layer consists of: 1. Router: A parameter gating mechanism that selects the experts used to process the given input by producing a probability distribution over experts. 2. Experts: Standalone NN modules with independent sets of parameters trained jointly with routers. It can use more complex architecture to create hierarchical MoE modules by implementing each expert as another MoE. - Implementation in Switch Transformers : Training was found to be 7x faster than training the base T5, keeping computation and training time constant. - Gshard and MeshTransformer : These frameworks propose efficient methods to train the MoE architectures using the parallelisation of models across GPUs/TPUs. Challenges with MoEs and Solutions 1. Inconsistent Batch Size for Experts: Can be overcome with the use of "soft" constraints in training loss. 2. Load Balancing Across Experts: The issue of repeatedly using the same few experts while training can be overcome by including better balancing mechanisms. 3. Model Instability Issues: Use of router-z Loss as an auxiliary loss to encourage smaller values within the router mechanism. 4. Model Overfitting: Requires different hyperparameter settings for effective fine-tuning. It was found that models benefit from high learning rates and smaller batch sizes. Recommended Reading/References 1. Hugging Face Blog on MoE - https://lnkd.in/g36gQJTx 2. Cameron Wolfe's Substack on Conditional Computation - https://lnkd.in/gtpjMHnJ

60

1 Kommentar

🎉 Semantic Data Specifications: A Practical Framework for Consistent, Interoperable, and Reusable Data in Healthcare. This primer has something for everyone. If you're looking for an introduction to Semantic Data Specifications, which are fundamental to #SemanticDataQuality, you're in the right place! I have been focusing on Semantic Data Quality research and data harmonization in healthcare for over a decade. I have come to recognize its significance in enhancing care delivery and improving patient outcomes. The Unified Modeling Language course I took in graduate school was, without a doubt, the most valuable skill I gained. If you want to learn more about semantic data quality and why semantic data specifications are crucial for trustworthy AI, follow me. I will start addressing this topic more consistently in 2025. The future is bright! #UML

8

1 Kommentar

Whether Large Language Model is the Right Path to AGI? MIT Paper in Nature Shows Language is Primarily a Tool for Communication, Not Thought... The pursuit of Artificial General Intelligence (AGI) often hinges on advancements in language models. However, a groundbreaking paper by Evelina Fedorenko and colleagues published in Nature argues that language's primary function is communication rather than thought. This raises crucial questions about the trajectory we're on with AGI development. Key Insights from the Paper: 1. Neuroscientific Evidence:   - Recent findings reveal a double dissociation between language and thought in the brain.    - Language impairments (like aphasia) do not necessarily impair thinking or reasoning, and intellectual impairments do not always affect linguistic functions. This suggests that complex thought processes do not rely on language. 2. Optimization for Communication:   - Linguistic analyses show features across languages (spoken and signed) optimized for efficient communication.    - Common words are shorter, related words cluster in sentences, and languages have mechanisms to convey meaning despite distortions, highlighting a primary role in communication rather than thought. 3. Cultural Transmission:   - Language serves as a powerful tool for transmitting cultural knowledge, transforming human culture without being a prerequisite for complex thought. 4. Co-evolution:   - Language likely co-evolved with human thinking and reasoning, reflecting but not creating cognitive sophistication. 5. Challenging Common Assumptions:   - Despite the intuitive link between language and thought, the paper argues that inner voices and similar experiences do not indicate that language is used for thinking. 6. Interdisciplinary Approach:   - The paper combines evidence from neurobiology, cognitive science, and corpus linguistics, providing a comprehensive view of the topic. 7. Implications:   - By positioning language primarily as a communication tool, the paper challenges long-held beliefs about language and cognition. This perspective could significantly impact our understanding of human cognitive evolution and the nature of thought. What Does This Mean for AGI? If language is not the medium of thought but a tool for communication, are we on the right path by focusing on large language models for achieving AGI? This perspective might suggest that we need to explore other cognitive processes and mechanisms beyond language to reach true artificial general intelligence. Follow Leon Zheng to join the discussion! Do you think the current path of language-focused AI research is sufficient for AGI, or should we pivot our approach? How should this new understanding shape the future of AI development? 🔗 Image source: (https://lnkd.in/g9-TyTsn) #AI #AGI #ArtificialIntelligence #LanguageModels #CognitiveScience #Neuroscience #MIT #NaturePublication #Innovation #FutureOfAI #TechInsights

1

Fine-tuning LLMs for domain adaptation requires more than just specialized data. 🔬 This new study dives deep into CPT, SFT, DPO, ORPO and Model Merging, revealing their unique impact on model specialization. Explored various fine-tuning techniques: • Continued Pre-Training (CPT) • Supervised Fine-Tuning (SFT) • Direct Preference Optimization (DPO) • Odds Ratio Preference Optimization (ORPO) • Model Merging using Spherical Linear Interpolation (SLERP) Key insights: 🚀 Merging Models: Combining multiple fine-tuned models using Spherical Linear Interpolation (SLERP) yields synergistic, emergent capabilities, exceeding individual model performance. 💪 Model Size Matters: The study observes a threshold effect, where the benefits of advanced fine-tuning techniques, including SLERP, are more pronounced in larger models (7B-8B parameters) compared to smaller ones (1.7B parameters). Paper: https://lnkd.in/gBPseyja #LLM #AI #FineTuning #DomainAdaptation #ModelMerging #MaterialsScience #Engineering #DeepLearning #AIResearch #SphericalLinearInterpolation #SLERP #llama #mistral #SmoLLM

2