Jyh-Herng Chow

The BayLearn 2024 abstract submission deadline has been extended to Aug 5, 2024. If you are working on cutting-edge ML research, please consider submitting an abstract. BayLearn 2024 will be an in-person event, held on Oct 10, 2024: https://baylearn.org This is a great opportunity for student researchers to showcase their latest research work and get good feedback. Submissions are non-archival and the review process is quite informal. #machinelearning #research

69

Thanks Hamdi Amroun, PhD - this is a great paper to share. In the real world business context we often need AI to reason over large pieces of text, and often you need/want multi-document summarization (MDS). 'When testing five LLMs with benchmarks using news and conversation datasets, they found up to 75% of content in MDS summaries was hallucinated, with notable increases towards summaries' end. Alarmingly, even for non-existent topics, models like GPT-3.5-turbo and GPT-4 generated fabricated content 79% and 44% of the time, respectively. Analysis of 700+ generated insights showed that hallucinations often arose from failures to follow instructions or overly generic content.' I agree with Hamdi's sentiments that despite improvements in post-processing techniques, robust solutions are urgently needed to reduce hallucinations, as inaccurate summaries can lead to business risks (notably misinformation and misrepresentation). I would add that there is still a lot of utility in LLMs/Multimodal models but we need to be vigilant - this is not about AI malevolence or intentional deception. LLMs do not possess consciousness or intent; they generate content based on patterns in data they were trained on.

6

1 Comment

Using generative AI, NVIDIA operations built an AI Planner agent, developed on NVIDIA Inference Microservices (NIM). The agent leverages LLM, NeMo Retriever and CuOpt NIM to reduce re-planning time from hours to just seconds. https://nvda.ws/4bIfS2T

20

1 Comment

Just did a keynote in PyTorch conference, on LLM deployment. Thanks for the great work of PyTorch team, our partners, and the interests from the audience! Please try out our deployment ecosystem of torchchat, torch.compile, torch.export, AOT Inductor, torchao and ExecuTorch, and checkout all the amazing talks, posters and demos in the conference!

50

3 Comments

Can LLM preference tuning degrade Multi-Agent system performance? In a recent tweet, Andrej Karpathy questioned why all LLMs sound similar, prompting Sebastian Rascha to speculate about the uniformity of alignment datasets. Aligning user preferences is a last-mile step in training large language models. Regardless of the pre-training datasets used, could the preference datasets used during alignment compromise the effectiveness of LLMs? This issue could adversely affect multi-agent systems incorporating multiple LLMs in their design.

7

1 Comment

MLCommons just released an AI Safety Benchmark POC (https://lnkd.in/gyAG7z2U). This is a great first step in the long road of ensuring AI is safe.

4

1 Comment

Using generative AI, NVIDIA operations built an AI Planner agent, developed on NVIDIA Inference Microservices (NIM). The agent leverages LLM, NeMo Retriever and CuOpt NIM to reduce re-planning time from hours to just seconds. https://nvda.ws/4bIfS2T

2

There is often an unspoken assumption by ML researchers that all we need to make progress and model every aspect of our world is data. We do not ascribe to this in my lab. In fact, we believe that data will never be enough. Our experiences with the nuances and complexities of scientific problems have informed us to the insufficiency of data to capture continuous physical processes, which afterall govern our biological and physical world. An example of this is this series of two papers led by my wonderful PhD student, Anowarul Kabir and advanced by a precious multi-year collaboration of my lab with Los Alamos National Lab: Anowarul Kabir, Manish Bhattarai, Kim Rasmussen, Amarda Shehu, Anny Usheva, Alan R Bishop, and Boian S Alexandrov. Examining DNA Breathing with pyDNA-EPBD. Bioinformatics 39(11):btad699, 2023. https://lnkd.in/gZaHE4hS Anowarul Kabir, Manish Bhattarai, Selma Peterson, Yonatan Najman-Licht, Kim Ø Rasmussen, Amarda Shehu, Alan R Bishop, Boian Alexandrov, Anny Usheva. DNA Breathing Integration with Deep learning Foundational Model Advances Genome-wide Binding Prediction of Human Transcription Factors. Nucleic Acids Research: gkae783, 2024. https://lnkd.in/gfTTqcPy Our goal: advance an exceptionally challenging problem in molecular biology, prediction of transcription factor binding sites. Our first step: capture the underlying physics that is missing in the data. Our second step: integrate that now with the data we have in a foundation model for predicting transcription factor binding sites. Performance improves. Most importantly, when we look at the sequence motifs that constitute "signatures" of what makes a transcription factor binding site, we obtain answers. All in the open, nothing opaque.

33

1 Comment

How do you make large-scale knowledge graphs actually usable for AI retrieval? We’ve been tackling this question with memory fragment projection using cognee. The idea is simple: 💡 Instead of analyzing the entire graph, we extract focused subgraphs (memory fragments) that are small enough for in-memory exploration but rich enough to support AI queries. In the blog, Hajdu László, Ph.D. walks us through: 🚶‍➡️ Subgraph filtering methods (community-based, neighborhood-based, etc.) and their tradeoffs 🚶‍♀️‍➡️ How cognee integrates with Neo4j to detect communities and project graph fragments 🚶‍♂️‍➡️ A practical example of projecting memory fragments from Neo4j’s knowledge graph This approach lets us: ✅ Optimize AI retrieval pipelines with GraphRAG ✅ Maintain flexibility while focusing on relevant graph layers ✅ Scale down the complexity without losing context If you’ve worked with knowledge graphs or retrieval pipelines, you know the pain of balancing scale and precision. This solution makes that process much more approachable. Here’s the blog 👇 https://lnkd.in/d5js2psf Would love to hear your feedback or challenges you’ve faced in this space!

21

A time-series foundation model is valuable because time-series data is ubiquitous across industries, yet traditional modeling approaches often fail to fully exploit the complex temporal dependencies and cross-domain similarities present in such data. We introduce Moirai-MoE: the first and state-of-the-art time series foundation model in universal forecasting. Takeaways: 1. Autonomous Specialization: The model autonomously achieves token-level specialization, enhancing efficiency and performance. 2. Performance Boost: Delivers a remarkable 17% improvement over its predecessor, Moirai, without increasing the model size. 3. Two model variants: 117M parameters with 11M activated parameters and 935M parameters with 86M activated parameters 4. Limitations of Existing LLMs: Current large language models (LLMs) struggle with time-series forecasting tasks. Moirai-MoE outperforms GPT4-o, a model 1000+ times its size in time-series tasks. 5. We need more time-series foundation models!!! Paper: https://bit.ly/3O1yiRQ Code: https://bit.ly/48FAF6i Models: https://bit.ly/3YNozDY

246

17 Comments

The performance of LLM run on HPC platform is stuck on the memory of HPC. The memory of an HPC platform is certainly a factor in the performance of LLMs running on it, but it's not the only one. Here's a breakdown: Factors affecting LLM performance on HPC: Memory: Large language models (LLMs) can be very memory-intensive. If the HPC platform doesn't have enough memory to store the LLM's data structures and intermediate computations, performance will suffer. Compute power: LLMs also require significant computational power. The number and types of processors on the HPC platform will significantly impact how fast the LLM can process information. Interconnection speed: The speed at which different parts of the HPC platform can communicate with each other is crucial. If data transfer between processors or memory is slow, it can bottleneck the LLM's performance.

5

💥💥💥 Stronger Models are NOT Stronger Teachers for Instruction Tuning Zhangchen Xu, Fengqing Jiang, Luyao Niu, Bill Yuchen Lin, Radha Poovendran Abstract Instruction tuning has been widely adopted to ensure large language models (LLMs) follow user instructions effectively. The resulting instruction-following capabilities of LLMs heavily rely on the instruction datasets used for tuning. Recently, synthetic instruction datasets have emerged as an economically viable solution to provide LLMs diverse and high-quality instructions. However, existing approaches typically assume that larger or stronger models are stronger teachers for instruction tuning, and hence simply adopt these models as response generators to the synthetic instructions. In this paper, we challenge this commonly-adopted assumption. Our extensive experiments across five base models and twenty response generators reveal that larger and stronger models are not necessarily stronger teachers of smaller models. We refer to this phenomenon as the Larger Models' Paradox. We observe that existing metrics cannot precisely predict the effectiveness of response generators since they ignore the compatibility between teachers and base models being fine-tuned. We thus develop a novel metric, named as Compatibility-Adjusted Reward (CAR) to measure the effectiveness of response generators. Our experiments across five base models demonstrate that CAR outperforms almost all baselines. 👉 https://lnkd.in/dUyZe8au #machinelearning

3

1 Comment

Organizations have insufficient technical capabilities for the speed and scale of their ML model implementations. My colleague Sumit Agarwal just published a very helpful research with a capability view and sample vendors for defining a modular, flexible and scalable MLOps solution. https://lnkd.in/gnB_T9bq #MLOps #LLMOps #AI

149

17 Comments

Puma taps Google Cloud’s Imagen on Vertex AI solution to create immersive digital shopping experiences- https://lnkd.in/g_PRKrkR AI continues to be a game changer. It's a blessing and a curse, so learn how to use it and learn how to make it work for your business. Being afraid of it is not the solution. Those fears will alleviate once you get familiar with it and learn to use it properly.

1

🚀 Does Size Always Matter? 🧐 When it comes to Large Language Models (LLMs), the answer isn’t always straightforward. While larger models often top leaderboards with impressive average benchmark scores, real-world applications tell a different story. A few considerations: • 📊 Leaderboards favor larger models, but these models might not always be the best for specific tasks. • 🎯 For business applications, we often use LLMs for single tasks or a narrow subset of tasks, which align with specific benchmarks, not the whole portfolio. • 💡 A great example is Bielik 2 - a Polish LLM with just 11B parameters, performing well in SpeakLeash.org Arena, where it ranks between LLaMA 3.1 405B and 70B (this ranking reflects how well the responses of different models meet the subjective expectations of Polish users). • 💸 This becomes especially important when the models under consideration differ significantly in terms of cost. Key Takeaway: When choosing an LLM, prioritize benchmarks or rankings that closely match your use case over general leaderboards, or even create your own benchmarks closely aligned with your needs. #AI #MachineLearning #LLM #BusinessInsights #TechTrends #AIModels

52

5 Comments

Meta has just released Llama-3. Pretrained and instruction-fine-tuned language models with 8B and 70B parameters: https://lnkd.in/gb3Aqppj They are also launching https://www.meta.ai

11

Large language model on a microcontroller? That won't be happening any time soon—but what we CAN do is build a pipeline that distils a subset of knowledge from an LLM (or other foundation models) and squeezes it into a tiny model that can run on any device. And you can do the same with Edge Impulse! This post shows how easy it is to integrate tools like OpenAI GPT-4o with Edge Impulse. We use GPT-4o to rapidly label data and use it to train an NVIDIA TAO object detection model—which we've squeezed down to fit on an Arduino! https://lnkd.in/eSSHpq_i Follow Edge Impulse for more features like this that you can use right now!

87

6 Comments

This is a great first step in understanding how scaling dynamics extend beyond language to other types of sequential data. Time is a universal axis, understanding how models learn optimally over data collected over time is a crucial step in the broader AI research agenda.

26