Jakub Nowacki

Struggling to wrap your head around exchange-correlation functionals? 🤔 📢My latest blog post tries to cover the basics without the heavy math.👩‍🔬🧑‍💻 🔗https://lnkd.in/dAuXmRwy

20

LlamaIndex + MLflow RAG contains a lot of parameters to tune, from chunking to indexing - everything affects downstream answer accuracy. It’s important to have a systematic approach to track and tune these parameters, and define the right eval metrics/dataset to evaluate changes. Check out Jino Rohit's video on experimenting with a LlamaIndex pipeline with MLflow for experiment tracking. https://lnkd.in/gyYXfyyb

554

14 komentarzy

Learn the basics of nonlinear dimensionality reduction, kernel PCA, and multidimensional scaling by following along Biman Chakraborty's clear, hands-on Python tutorial.

18

Today we invite you to watch a recording of Contributed Talks from the ML in PL 2023 conference, where Błażej Osiński talked about: Toward Efficient Machine Learning For Sequential Decision Making. https://lnkd.in/datBvHb8 In this talk, I will present my personal journey toward developing efficient machine learning methods for sequential decision making. This journey spans both academia and industry, tackling diverse challenges that range from Atari games to autonomous driving and robotics. Why should we care about efficient methods for sequential decision making? If we observe the progress of artificial intelligence over the last few decades, it has managed to ascend what were perceived as pinnacles of human intelligence. From chess and Go to StarCraft and No Limit Texas Hold'em, machines have toppled human champions. The emergence of large language models extends this progress to more open-ended challenges. Despite these successes, when it comes to physical interactions with the real world, the achievements of AI are less remarkable. For instance, despite three decades of research, multi-fingered robotic hands are still far less dexterous than those of an average human child. The comparatively slower advancements in the realm of physical interactions can largely be attributed to the massive data requirements of contemporary machine learning methods. Advancements in efficiency could pave the way for truly practical robots capable of alleviating our household chores, rather than just beating us in games or eyeing our jobs. Błażej is a machine learning researcher with experience in sequential decision-making problems (reinforcement learning, imitation learning). He is about to graduate with a Ph.D. from the Uniwersytet Warszawski. He has divided his time between academia and industry, having affiliations with institutions such as Lyft Level 5, Google Brain, and the University of California, Berkeley. He is now pivoting to the ambitious endeavor of explaining cellular biology with machine learning working for Altos Labs.

4

1 komentarz

This article will explore how leveraging lower-precision formats can enhance training and inference speeds up to 3x without compromising model accuracy. Mixed-precision training is a powerful technique that significantly enhances the speed and efficiency of training deep neural networks on modern GPUs. Traditionally, 64-bit precision has been the standard in scientific computing due to its high accuracy, but it's computationally expensive and not optimized for GPUs. In deep learning, 32-bit precision, known as single-precision, strikes a balance by reducing memory usage and speeding up computation without greatly impacting model accuracy. Mixed-precision training takes this a step further by strategically alternating between 32-bit and 16-bit precision during training. Just using 16-bit for training might mean the model's loss will not converge. What does mixed-precision means? Mixed-precision training involves converting neural network weights to 16-bit precision for faster computation, then computing gradients in 16-bit before converting them back to 32-bit for numerical stability. Finally, these gradients are used to update the original 32-bit weights. This method not only cuts down training time but also reduces memory consumption, allowing larger models and batch sizes to be processed efficiently. Interestingly, the slight imprecision introduced by lower precision can act as a regularizer, potentially improving the model's generalization and accuracy. New floating-point formats like bfloat16 offer extended dynamic ranges, making them particularly suitable for deep learning tasks on modern hardware, such as NVIDIA's A100 Tensor Core GPUs. For even greater performance improvements, quantization can be employed (during inference, or even while training). This process converts model weights to low-bit integer representations, like 8-bit or 4-bit integers, offering substantial gains in efficiency and speed during inference. The author,  Sebastian Raschka, explains all this using experiments on a model. https://lnkd.in/eEu3Rfg6

9

Graph-based Euclidean clustering is reshaping point cloud segmentation by automating processes, reducing noise, and offering robust scalability for complex scenes. Dive deeper in this article by Dr. Florent POUX. #MachineLearning #Clustering

11

Where Do You Still Need Machine Learning? (Guest: Srujana Kaddevarmuth) https://buff.ly/4eWRj4a via Andreas Welsch of Intelligence Briefing on Thinkers360 #AI #DigitalTransformation #GenerativeAI

3

Machine Learning Methods as Robust Quantum Noise Estimators by Jon Gardeazabal-Gutierrez, Erik B. Terres-Escudero, Pablo Garc\'ia Bringas via quant-ph updates on arXiv.org \(\[Global\] Quantum Computing\) URL: https://ift.tt/84PMw5L arXiv:2409.14831v1 Announce Type: new Abstract: Access to quantum computing is steadily increasing each year as the speed advantage of quantum computers solidifies with the growing number of usable qubits. However, the inherent noise encountered when running these systems can lead to measurement inaccuracies, especially pronounced when dealing with large or complex circuits. Achieving a balance between the complexity of circuits and the desired degree of output accuracy is a nontrivial yet necessary task for the creation of production-ready quantum software. In this study, we demonstrate how traditional machine learning \(ML\) models can estimate quantum noise by analyzing circuit composition. To accomplish this, we train multiple ML models on random quantum circuits, aiming to learn to estimate the discrepancy between ideal and noisy circuit outputs. By employing various noise models from distinct IBM systems, our results illustrate how this approach can accurately predict the robustness of circuits with a low error rate. By providing metrics on the stability of circuits, these techniques can be used to assess the quality and security of quantum code, leading to more reliable quantum products. https://ift.tt/ZaWoNcy

1

Apache Doris user story in AI ⭐ Clusters: 2 Nodes: 20+ (independent deployment of FE/BE nodes) Daily queries: 10,000+ Data storage: 10TB+ Daily data increments: 2TB+

16

In Daniel Klitzke's latest article, he shows how you can use the Huggingface Transformers and Sentence Transformers libraries to boost you RAG pipelines using reranking models. #LLM #MachineLearning

23

Informative post explaining how the attention mechanism works and its importance.

3

The second article in our series, written by Paweł Strzałkowski, is now available on the blog! 📝 In this post, he explores the practical implementation of Vector Search - a technique for finding items based on semantic similarities rather than exact string matches. Paweł covers the theory behind Vector Search, discusses its application in image recognition, and shares a complete photo search application - all developed in Ruby. Enjoy reading! 💫 https://lnkd.in/dT3s6V98

27

Subspaces can be more interpretable than PCA when detecting outliers, especially in large datasets. While PCA reduces dimensionality, subspaces allow you to work with the original features, maintaining clarity in your results. Learn when to use each technique in Brett Kennedy's article.

21

Large Language Models as Markov Chains (October 2024) Paper: https://lnkd.in/gwE6sC73 Abstract: "Large language models (LLMs) have proven to be remarkably efficient, both across a wide range of natural language processing tasks and well beyond them. However, a comprehensive theoretical analysis of the origins of their impressive performance remains elusive. In this paper, we approach this challenging task by drawing an equivalence between generic autoregressive language models with vocabulary of size T and context window of size K and Markov chains defined on a finite state space of size O(TK). We derive several surprising findings related to the existence of a stationary distribution of Markov chains that capture the inference power of LLMs, their speed of convergence to it, and the influence of the temperature on the latter. We then prove pre-training and in-context generalization bounds and show how the drawn equivalence allows us to enrich their interpretation. Finally, we illustrate our theoretical guarantees with experiments on several recent LLMs to highlight how they capture the behavior observed in practice."

1

Join Julia Belyakova's talk at #LambdaConf2024! Explore the remarkable features of Julia, the high-level, dynamic programming language designed for scientific computing. Discover how #Julia excels in code reuse, extensibility, and performance with its optimizing just-in-time compiler. Learn about the intentional restrictions, such as nominal subtyping and the unique semantics of eval, that contribute to predictable compilation and simplified compiler implementation. Join us: https://buff.ly/3wbaG7U #JuliaLanguage #ScientificComputing #DynamicDispatch

16