Ka Chung Ng’s Post

Don't miss our upcoming webinar featuring insights from Harpreet Singh Riat, our Director of Technical Sales. Join us as he discusses the crucial aspects of applications, the significance of Large Language Models (LLMs), and the role of ontologies in improving accuracy and context comprehension. Harpreet will shed light on SciBite's strategic decisions in incorporating LLMs and ontologies into a Retrieval Augmented Generation (RAG) system. Learn about establishing a strong information retrieval system, semantic parsing of natural language questions, and the importance of explainability in answer generation. #TechnicalSales #Webinar #SciBite #LLMs #Ontologies #RAGSystem

⚡We are excited to share that our paper, entitled: “Metaheuristics and Large Language Models Join Forces: Towards an Integrated Optimization Approach” is now available on arXiv 🤘 👉 This paper introduces a novel integration of Metaheuristics (MHs) and Large Language Models (LLMs) to tackle complex combinatorial optimization problems. Our research reveals that, in specific contexts, LLMs can uncover non-obvious patterns and relationships, which can then be leveraged to guide MHs towards more effective or high-quality solutions, leading to improved outcomes. 📄 Preprint (PDF): https://lnkd.in/ebAmir6F #LLMs #metaheuristic #optimization #AI

💡Small Language Models (SLMs) are becoming increasingly visible. They offer a more energy-efficient alternative to the powerful but resource-hungry LLMs. SLMs are designed for efficiency, perfect for on-device or edge use where computational resources are limited. 📚 A recent survey dives into the techniques behind these compact models—like model compression, pruning, and quantization—that make them lightweight without compromising too much on performance. 👉🏻 https://lnkd.in/eh6VErtj 🤖 I believe, while LLMs aren’t going to vanish soon, SLMs will play a big role in powering Agentic AI, bringing scalable intelligence into more resource-constrained applications. #AI #SML #LLM #AgenticAI

👉🏼 Testing theory of mind in large language models and humans 🤓 James W A Strachan 👇🏻 https://lnkd.in/e5sQwzcb 🔍 Focus on data insights: - Comparison of human and LLM performance on theory of mind abilities - GPT-4 models performing at or above human levels in identifying indirect requests, false beliefs, and misdirection - LLaMA2 outperforming humans in detecting faux pas 💡 Main outcomes and implications: - LLMs exhibit behavior consistent with human mentalistic inference - Importance of systematic testing for meaningful comparison between human and artificial intelligences 📚 Field significance: - Understanding the capabilities and limitations of large language models in theory of mind tasks 🗄️: [#theoryofmind #largeLanguageModels #dataInsights]

Artificial intelligence made by yourself! From "Decision Making" to "Recommendation Systems" to "prediction models"; from "image classification" to "large language models (LLM) to "neural networks"; as 20 years ago at the IT-school of Bond University, it was an exciting journey through the universe of artificial intelligence. The course was structured in such a way that you could build and train artificial intelligence models with your own hands. Experience artificial intelligence as a craft.

🔍📖 ***Key Insights: Empirical Study of "Catastrophic Forgetting" in LLMs!***🤖 The issue of "Catastrophic Interference" was originally coined by McCloskey and Cohen (1989), concluding that: 1) At least some interference will occur whenever new learning alters the weights involved in representation. 2) The greater the amount of new learning, the greater the disruption in old knowledge. 3) Interference is catastrophic in backpropagation networks when learning is sequential but not concurrent. Now, we commonly know this phenomenon as "Catastrophic Forgetting," where a model forgets previously learned information as it acquires new knowledge. I recently explored the insightful paper titled "An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning" published this April. 👉 https://lnkd.in/gNPZFgNU 📌 Key Contributions of the Study: Empirical Analysis of CF: An initial empirical study analyzing CF during continual instruction tuning, focusing on general knowledge areas such as domain knowledge, reasoning, reading comprehension, and bias. Evidence of CF Across Models: The research provides preliminary evidence that the CF problem is prevalent in the continual instruction tuning process across various models, including BLOOMZ, mT0, LLAMA, and ALPACA. The study highlights how model architecture, scale, and general instruction tuning influence the severity of the CF problem. 🔧 Effect of Model Architecture The findings show that decoder-only models can retain more knowledge during continual instruction tuning. Specifically, as model scale increases, decoder-only models like BLOOMZ exhibit less CF compared to encoder-decoder models like mT0. 🪂 Conclusion The findings reveal that CF is prevalent in the continual fine-tuning of various LLMs, with an increasing severity of forgetting in domain knowledge, reasoning abilities, and reading comprehension skills as model scale increases. The paper also suggests that employing general instruction tuning techniques may help alleviate the CF problem in LLMs, indicating that exploring effective methods to mitigate CF during continual fine-tuning is a promising research direction. #AI #CatastrophicForgetting #LanguageModels #ContinualLearning #LLMs #DataScience #ArtificialIntelligence #MachineLearning

Bridging the Gap: Resonance RoPE's Novel Approach in AI Modeling -Suyuchen Wang, Ivan Kobyzev The paper introduces Resonance RoPE, a method enhancing Large Language Models' performance in train-short-test-long scenarios, where models struggle with longer sequences not seen during training. This approach refines Rotary Position Embedding to improve handling of out-of-distribution token positions, enhancing model robustness without extra computational cost. Additionally, a new benchmark, PosGen, is presented for analyzing model behavior in these scenarios. The method demonstrated improved performance in both language modeling and long-text applications, outperforming existing RoPE methods in extensive tests. #ResonanceRoPE #AIModeling #LargeLanguageModels #TrainShortTestLong #RotaryPositionEmbedding #ModelRobustness #PosGenBenchmark #LanguageModeling #LongTextApplications #Research #Innovation

Ever wondered how the next generation of AI agents will make decisions, learn, and interact with the world more intelligently? 🤖💡 The paper "Cognitive Architectures for Language Agents" introduces CoALA, a groundbreaking framework that combines the reasoning power of Large Language Models (LLMs) with structured cognitive processes inspired by cognitive science. CoALA positions LLMs at the core of modular systems, enabling language agents to: 🧠 Manage memory (working, episodic, semantic, procedural). 🔄 Plan and execute actions in structured loops. 🌐 Interact with complex environments, from robotics to digital simulations. What’s exciting is how this framework draws from decades of AI research to organize and enhance modern language agents, setting the stage for more adaptable, general-purpose intelligence. The future of AI agents isn’t just about solving tasks, it’s about building systems that think, learn, and grow like humans. This paper offers both a theoretical foundation and actionable insights to design smarter agents. Read more here: https://lnkd.in/egWj5RCf #ArtificialIntelligence #LanguageModels #CognitiveScience #MachineLearning #Innovation #AIResearch #LLMs #TechForGood #AIAgents #AgenticAI #AgenticAgents #AgenticWorkflows

Cognitive Architectures for Language Agents (CoALA) is a framework that integrates principles from historical AI concepts like production systems and cognitive architectures with modern large language models (LLMs). It organizes agents around memory (working, semantic, episodic, and procedural), actions (internal reasoning/retrieval and external grounding), and decision-making cycles. CoALA leverages LLMs to manage reasoning, planning, and learning, enabling structured interaction with environments and adaptive behaviors. Drawing parallels with symbolic AI, CoALA addresses LLM challenges like opacity while utilizing their scalability and flexibility, offering a structured approach to designing advanced language agents for diverse applications. https://lnkd.in/gCvP2SKM

Researchers have proposed "TokenWorld," an innovative neural architecture that represents diverse information like text, images, and audio as "tokens" and processes them using a flexible attention mechanism. This breakthrough enables improved handling of complex multimodal tasks like text-to-image generation and audio-visual understanding. The system opens up exciting possibilities for unifying multimodal data and scaling across various AI applications. Read more at https://lnkd.in/gkbCHvN7 #AI #DeepLearning #MultimodalLearning #NeuralNetworks #MachineLearning #TokenWorld #AIResearch #Innovation #TechResearch