Zac Policzer’s Post

This week we are presenting our paper "LiNR: Model Based Neural Retrieval on GPUs at LinkedIn" accepted at CIKM 2024 (https://lnkd.in/gUrWqRcD). Please stop by and say hi to Aman Gupta, who will be there in person :) We discuss our experiences and challenges in creating scalable, differentiable search indexes using TensorFlow and PyTorch at production scale. In LiNR, both items and model weights are integrated into the model binary. Viewing index construction as a form of model training, we describe scaling our system for large indexes, incorporating full scans and efficient filtering. We believe LiNR represents one of the industry's first Live-updated model-based retrieval indexes at production scale. Talented co-authors include Fedor Borisyuk, Qingquan Song, Mingzhou Zhou, Ganesh Parameswaran, Madhulekha Arun, Siva P., Tugrul Bingol, Zhoutao Pei, Stanley(Kuang) Lee, Lu Z., Hugh Shao, Syed Ali Naqvi, Sen Zhou, Aman Gupta

In this advanced use case, #VeniceDB is used to merge batch and stream inputs, and to transport the data to serving nodes which load it into GPUs, thus updating their ML model in near real time. Check out the paper for more details!

Introducing Mamba, an innovative selective state space model that addresses the limitations of both Transformers and traditional SSMs. The Mamba architecture integrates selective SSMs into a neural network, removing attention and MLP blocks. Let's explore its key features! 👇 ▪️ Selective SSMs Selective SSMs are a special type of building block used in Mamba neural networks. They allow the network to focus on specific parts of the input sequence, improving efficiency and performance in tasks like language modeling and audio analysis. ▪️ Simplified structure: Unlike traditional models that use a mix of different blocks, Mamba has a single, efficient block called the "Mamba block." These Mamba blocks are all the same and stacked together, making the model easier to understand and faster to run. ▪️ Hardware-aware algorithm: Regular SSMs are powerful but can be slow on hardware like GPUs. Mamba addresses this by using a hardware-aware parallel algorithm. It skips convolutions and uses a scan operation to better manage memory, making Mamba faster and more efficient. For more fascinating information about Mamba, read our article: https://lnkd.in/gKeH5BYB

Why we need a Library like PyTorch at all since Numpy already provides data structures and utilities for working with multi-dimensional numeric data. There are two main reasons: Automatic Differentiation: One of the key features of PyTorch is its automatic differentiation capability through the autograd module. This feature allows PyTorch to automatically compute gradients of the loss function with respect to model parameters, facilitating gradient-based optimization methods like gradient descent. While NumPy provides array operations, it doesn't have built-in support for automatic differentiation, which is essential for training neural networks efficiently. GPU Acceleration: PyTorch seamlessly integrates with CUDA, NVIDIA's parallel computing platform, enabling GPU acceleration for numerical computations. This allows PyTorch to leverage the computational power of GPUs for training deep neural networks, resulting in significant speedups compared to CPU-only implementations. While NumPy can be used with GPU-accelerated libraries like CuPy, PyTorch provides a more streamlined approach to GPU programming, especially for deep learning tasks.

We recently announced a new math library called cuEquivariance. I am particularly happy to see it coming out as this an important set of operations used in equivariant neural network (ENN) such as DiffDock, MACE and NequIP (just the tip of the iceberg - search perplexity.ai to see where they are used). Simply put this technique allows ENN to recognise objects such as molecules in different orientations. The NVIDIA team Mario Geiger, Emine Kucukbenli, Becca Z. release cuEquivariance to facilitate the construction of high-performance ENN. Blogpost: https://lnkd.in/eYUqXuQq Github: https://lnkd.in/e696TwUJ #nvidia #chemistry #neuralnetwork #materialscience #molecularscience #protein #RNA #graphanalysis

🚀 Exciting Keynote at HPCA & PPoPP Conferences! 🚀 The upcoming co-located High-Performance Computer Architecture (HPCA) is co-located with the Symposium on Principles and Practice of Parallel Programming (PPoPP) and will be hosting Nir Shavit (MIT) as a keynote speaker! 🎙 Keynote Spotlight: In his talk, Nir will discuss the need to leverage sparsity in emerging deep neural networks without significant accuracy loss, and will present the challenges that appear in modern CPUs and GPUs, which do not lend themselves easily to the irregular data patterns sparsity introduces. Nir will survey the role of sparsity in neural network computation, and the parallel algorithms and hardware features that nevertheless allow us to make effective use of it. 🌐 Join us to shed light on how sparsity can transform the future of AI, making it more efficient and sustainable. #HPCA #PPoPP #DeepNeuralNetworks #Sparsity #AI #Keynote

* If you are an ML Engineer and you are not using Keras, Here are some incredible benchmarks!✨ Description: * Keras are faster than Pytorch, and produce the output rapidly when compared to Pytorch.⚡ * Keras is a Python-based, deep learning API that runs on top of the TensorFlow machine learning platform, and fully supports GPUs.⚡ *All the benchmarks were done in GPU, I would expect the result in TPU(Tensor Processing Unit).⚡ * To use a TPU, select a TPU runtime (for example, in Colab). Once you’ve connected to the runtime, you need to use a TPU Cluster Resolver to automatically detect the TPU on any supported platform.⚡ * Once you’ve set it up, the TPU workflow will be similar to implementing multi-GPU training on a single machine. The main difference is that the distribution strategy used is TPU Strategy.⚡ * Performance Benchmarks: Benchmarks comparing TPUs and GPUs on similar tasks often show TPUs excelling in tasks optimized for their architecture, offering faster training times and more efficient processing.⚡ 🔗 https://lnkd.in/gcqhGKmF #AI##ML##DeepLearning##DataScience#

Fascinated by the expressiveness of Scala and inspired by Fabio Labella’s talk on using fs2 for control flow, I decided to see if I can apply fs2’s functional streams to a simple Machine Learning workflow. I also used Storch, a Scala library implementing the ever familiar PyTorch API to enable the training of neural networks with GPUs. Blog: https://lnkd.in/gxpV_UDn Repo: https://lnkd.in/gJRDyqPw #scala #functionalprogramming #storch #fs2

Have you ever thought about the untapped power of GPUs in deep learning projects? 🧐 Dive into this insightful article that uncovers the secrets of leveraging GPUs effectively for accelerated computational processes in artificial intelligence. 💡✨ What's your take on incorporating GPUs for deep learning tasks? Let's discuss and unravel the possibilities together! https://lnkd.in/gD-RTjk6

My latest research on "A hybrid convolutional neural network and support vector machine classifier for Amharic character recognition" has just been published with @SpringerNature in Neural Computing & Applications Journal. Read here: https://rdcu.be/dJGl0