Piyush Mittal

Can Redis simplify real-time machine learning? 🤔 Bytewax’s 𝐛𝐲𝐭𝐞𝐰𝐚𝐱-𝐫𝐞𝐝𝐢𝐬 module positions Redis as both a data store and a stream processor, making it well-suited for feature pipelines, live model inference, and real-time analytics. Redis’s in-memory speed combines with Bytewax’s AI-native processing, providing the low latency needed for applications requiring immediate data response and adaptive learning. For data engineering teams, this integration offers practical advantages: 𝐫𝐞𝐝𝐮𝐜𝐞𝐝 𝐜𝐨𝐦𝐩𝐥𝐞𝐱𝐢𝐭𝐲, 𝐢𝐦𝐩𝐫𝐨𝐯𝐞𝐝 𝐥𝐚𝐭𝐞𝐧𝐜𝐲, and 𝐞𝐚𝐬𝐲 𝐬𝐜𝐚𝐥𝐢𝐧𝐠 𝐭𝐨 𝐦𝐞𝐞𝐭 𝐢𝐧𝐜𝐫𝐞𝐚𝐬𝐢𝐧𝐠 𝐝𝐞𝐦𝐚𝐧𝐝𝐬. Redis serves both as a 𝘥𝘢𝘵𝘢 𝘴𝘰𝘶𝘳𝘤𝘦 𝘢𝘯𝘥 𝘢 𝘴𝘪𝘯𝘬 𝘸𝘪𝘵𝘩𝘪𝘯 𝘉𝘺𝘵𝘦𝘸𝘢𝘹 𝘥𝘢𝘵𝘢𝘧𝘭𝘰𝘸𝘴, eliminating the need for separate storage and messaging systems. To see 𝐛𝐲𝐭𝐞𝐰𝐚𝐱-𝐫𝐞𝐝𝐢𝐬 in action, check out our detailed examples on GitHub. https://lnkd.in/ex5zBWAR

16

Discover how to implement an advanced #RAG pipeline using Meta's new Llama-3 quantized model with zero cost and high data security in this in-depth article by Kameshwara Pavan. Go into key advanced concepts including Hypothetical Document Embeddings (HyDE) for superior zero-shot dense retrieval and optimization strategies for peak performance with a fully local setup. 🚀 👨‍🚀 https://lnkd.in/gfMpSnij

35

Category of the week: Procedural Languages Select extension: PL/R 💭 It's not uncommon for analysts and researchers to leverage an IDE (Integrated Development Environment) for incidental statistical or aggregate processing. This tends to involve tremendous manual effort, and is difficult to scale. 💡 PL/R grants the user the power of the R programming language, which allows for example, statistical analysis on the data where it lives; this workflow can be further supported by materialized views, for example. Source: https://lnkd.in/gBrZ7esP Link: https://lnkd.in/er_G7dbY 🧰 Check PL/R out and similar extensions within the Procedural Languages category on Trunk; a Postgres extension registry! https://lnkd.in/g7pvNg2c 📊 Tembo is powered by Trunk's ever-growing collection of Postgres extensions, and Tembo users have the freedom to choose. Try out this extension and more on Tembo Cloud, for free! https://lnkd.in/gDQjj8ur

11

"In this article, I will address the key challenges data engineers may encounter when designing streaming data pipelines. We’ll explore use case scenarios, provide Python code examples, discuss windowed calculations using streaming frameworks, and share best practices related to these topics." by Mike Shakhomirov MBA

34

Using Qdrant with Apache Airflow can revolutionize how you automate and manage complex workflows for semantic querying. It makes handling large-scale vector data much simpler, and Airflow's monitoring features ensure everything runs smoothly. In this tutorial, you'll learn to use the Qdrant Airflow provider to write a DAG that generates embeddings in parallel and performs semantic retrieval based on user input. Explore the guide here: https://lnkd.in/eMpqD2tN P.S. How are you using Airflow in your data workflows? Comment below! 🚀

48

3 Kommentare

The TrueFlow Grid is an important tool in the HVAC sales process imo. The test doesn't take a long time as Ty demonstrates here. Many talk about how companies use scare tactics rather than being honest. Doing actual measurements and showing the results followed by possible solutions is honest. This is one tool to do that when selling a new system. For service measureQuick is a great tool. No need to make stuff up, most systems are pretty screwed up already. I do strongly object to the TrueFlow app's characterization of static pressure up to 1" water column being yellow (kinda ok.) I think that should end at 0.7" and green at 0.5" as most manufacturers specify 0.5" as the max for equipment. Also note that this system needs a bigger return plenum and filter. They pretty much all do. Solve that and a whole lot of things get much better.

44

19 Kommentare

📏 Comparing the Model Sizes of Top LLMs 🚀 When it comes to Large Language Models (LLMs), size is a crucial factor that directly impacts performance. Larger models can capture more complex patterns and nuances, leading to better accuracy and generalization. 🧠 Let's take a look at the model sizes of the top five LLMs: 1️⃣ GPT-4 is estimated to have an extremely large size, potentially surpassing 170 billion parameters. While the exact number is not public, its massive size contributes to its exceptional performance across a wide range of tasks. 🌟 2️⃣ Claude 3 comes in three sizes: Haiku, Sonnet, and Opus. Opus, the largest variant, competes closely with GPT-4 on various benchmarks. Sonnet and Haiku offer a balance between performance and efficiency. 🎼 3️⃣ Llama 3 is available in two sizes: 8 billion and 70 billion parameters. The 70B version demonstrates solid performance, outperforming models like Grok-1. Meta is also working on even larger Llama models, with sizes exceeding 400 billion parameters. 🦙 4️⃣ Mistral Large is a high-capability model with a large context window of 32,000 tokens. While the exact number of parameters is not public, its strong reasoning capabilities and native fluency in several languages suggest a substantial model size. 🌍 5️⃣ Grok is one of the largest open models available, with 314 billion parameters. Its successor, Grok-1.5, builds upon this foundation, incorporating architectural improvements and additional training data to enhance performance. 🚀 When considering model size, it's essential to understand the trade-offs between performance, efficiency, and cost. Larger models generally offer better accuracy but require more computational power, memory, and energy. 💸 The choice of model size depends on the specific needs of the application, available resources, and the balance between accuracy and efficiency. 🎯 #ModelSize #Top5LLMs #GPT4 #Claude3 #Llama3 #MistralLarge #Grok #MVYL

2

Understanding image-based data and evaluating model performance can be time-consuming but is essential for success. With Kolena’s LLM-supported image property extraction, users can access actionable insights in minutes, transforming how you work with complex datasets. In this video Merad Karamblou, Product Manager at Kolena, demonstrates how to use Kolena’s Datasets platform for LLM-supported image property extraction. By leveraging large language models (LLMs), users can quickly and efficiently extract meaningful insights from image-based data to better understand model performance. To try it out yourself with the example in the video, head over to https://lnkd.in/gfhXjv6n

11

1 Kommentar

In our latest blog, our CTO Gang Tao demonstrates how to use a random stream and just one line of #SQL to perform a #MonteCarlo simulation. By generating a large number of random samples, the Monto Carlo method helps estimate complex values - in this example, the value of pi. Curious how? Read on to learn how to calculate pi with Timeplus 🥧 #randomstream #montecarlo #pi https://lnkd.in/gP_NtwaP

4

In this article, the authors focus on a generic #multiterminal (#remote) #source #coding #scenario in which, via a joint design, several intermediate nodes must locally compress their noisy observations from various sets of user / source signals ahead of forwarding them through multiple error-free and rate-limited channels to a (remote) processing unit. Although different local compressors might receive noisy observations from a / several common source signal(s), each local quantizer should also compress noisy observations from its own, i.e., uncommon source signal(s). This, in turn, yields a highly generalized scheme with most flexibility w.r.t. the assignment of users to the serving nodes, compared to the State-of-the-Art techniques designed exclusively for a common source signal. Following the #Information #Bottleneck (#IB) philosophy, we choose the Mutual Information as the fidelity criterion here, and, by taking advantage of the Variational Calculus, they characterize the form of stationary solutions for two different types of processing flow/ strategy. They utilize the derived solutions as the core of our devised algorithmic approach, the #GE #neralized #M #ultivariate #IB (#GEMIB), to (efficiently) address the corresponding design problems. They further provide the respective convergence proofs of GEMIB to a stationary point of the pertinent objective functionals and substantiate its effectiveness by means of numerical investigations over a couple of (typical) digital transmission scenarios. ---- Shayan Hassanpour, Alireza Danaee, Dirk Wübben, Armin Dekorsy More details can be found at this link: https://lnkd.in/erBRh6fF

3

In data partitioning, there are no universal rules. Tailor your approach based on the specific characteristics of your data and the problem at hand for optimal results. Check out Federico Rucci's latest article on creating appropriate test sets. #MachineLearning #DataScience

16

Learn how you can convert a PyTorch model, trained on the web, to a mobile-optimized format. Madhur Zanwar describes the process using a few lines of code. He also discusses his journey and how one approach didn't yield satisfactory results while trying to run it in a React Native app. #machinelearning #pytorch #computervision #edgeml #reactnative #objectdetection #yolov5

14

We've distilled our comprehensive guide on building robust recommender systems into a quick, engaging video. In just one minute, you'll learn: - How to tackle the cold start problem - The power of collaborative filtering - Pro tips for scaling your RecSys 🔗 Check out the video here: https://buff.ly/4dLe1ep 📚 For a deeper dive, read the comprehensive article: https://buff.ly/3X1Cczr #RecommenderSystems #DataScience #ML

9