Vipul Vaibhaw

Shashank Sonkar and colleagues are back with another great paper and matching GitHub repo, "Pedagogical Alignment of Large Language Models". You can read the paper on arXiv here - https://lnkd.in/enZfCJgi, and browse the code here - https://lnkd.in/gyW5hSzp. They demonstrate an interesting approach to generating synthetic data to train open weights models to behave more pedagogically, as well as developing some new evaluation metrics. I think there's lots here for those of us interested in this kind of work to learn from and build on. (If synthetic data generation is new do you, check out this post from answer.ai earlier this week - https://lnkd.in/ecsG7R4b.)

38

4 Comments

Read our new Blog on - "Apache Spark: RDD vs. DataFrame vs. Datasets" https://lnkd.in/gSXPxEhm A popular and effective distributed computing platform for handling massive data is called Apache Spark. Developers using Spark have three primary choices for processing data: DataFrame, Dataset, and RDD (Resilient Distributed Datasets). The variations between these choices, as well as their advantages and disadvantages, will be discussed in this article. For more detailed Blog, please refer to the our Website's Blog Section. https://lnkd.in/gSXPxEhm #TBC #ApacheSpark #BigData #DataScience #SparkRDD #SparkSQL #DataFrame #DataSet #DataEngineering #DataAnalysis #websitetraffic #onlinespeed #Windows11 #Windows10Pro #Angular #GooglePlay #agile #s #socialmedia #CyberThreats #digitalwellbeing #csharp #DotNet #softwaredevelopment #programming #vr #coding #webdevelopment

2

🚀 New Blog Alert: Unlocking the Power of Apache Kafka & Avro for Real-Time Data Processing! 🚀 In the world of real-time data streaming, Apache Kafka is a game-changer, and when you combine it with Avro—a compact, schema-driven binary format—you unlock even more efficiency for handling complex data structures like files. In my latest blog, I break down: - Why Kafka and Avro make the perfect duo for high-performance data pipelines - How to produce and consume Avro files in Kafka - Step-by-step integration with a REST API for seamless interactions - Real-world benefits like efficient serialization, schema evolution, and language-agnostic data exchange Whether you're building modern data architectures or want to explore more about schema-based messaging, this guide is packed with hands-on insights to get you started! 💡 Check it out and let me know your thoughts: https://lnkd.in/d23aWb-e #Kafka #Avro #RealTimeData #DataStreaming #SoftwareEngineering #DataProcessing #SchemaEvolution #TechBlog #KyszTech

13

🚀 Optimizing Vertex Pipelines for Faster Inference 🚀 I recently embarked on optimizing our inference process using Vertex Batch Predict, and the results have been remarkable. Our existing system took around 19 hours for inference, but by leveraging Vertex Batch Predict, I managed to slash the inference time by approximately 60%. After some fine-tuning and research, I made further optimizations that cut the time down to just 55 minutes—a 90% reduction! Here are the key variables I optimized in Vertex Batch Prediction settings: Machine Type Starting Replica Count Max Replica Count Batch Size Insights for Optimization: Batch Size: Ensure your TensorFlow Serving process is configured to utilize multiple threads (equal to the number of CPU cores). If the default settings are used, it will process fewer instances in parallel, underutilizing the VM's CPU cores. A too-large batch size can lead to RAM bottlenecks. Machine Type: A massive machine type might not be fully utilized by your TensorFlow serving process. A medium VM, like n1-highmem-8, is typically sufficient and cost-effective. Replica Count: Set starting_replica_count equal to max_replica_count to avoid the delay in starting new replicas. If your TensorFlow serving isn’t optimized for multi-threading per VM, opt for many small to medium-sized VMs instead. Model Configuration: Consider these variables on the TensorFlow threading side for optimization: https://lnkd.in/gkMxxhng After implementing these optimizations, I achieved a dramatic reduction in inference time from 19 hours to just 55 minutes, achieving a ~90% improvement. Additionally, the cost has decreased as Vertex Batch Predict scales resources on demand. Optimizing your systems not only saves time but also reduces costs. I hope these insights help others in their development journey! #MachineLearning #DataScience #AI #VertexAI #Optimization #TensorFlow #TechTips #batchpredict #vertex‍_batc‍‍‍‍‍h_p‍‍‍‍‍‍re‍‍dic‍‍‍t ‍‍‍

9

Delve into the concept of multi-master PostgreSQL, the key benefits to this architecture, and why it is a critical solution for modern applications offered by pgEdge. Read the blog to learn more! https://hubs.la/Q030b35Z0 #pgedge #postgresql #postgresprofessionals Ibrar Ahmed

16

Build intelligent agents that can process real-time news data with almost no coding required! 📰 🌍 The latest article by Kameshwara Pavan Kumar Mantha explores a powerful RAG pipeline architecture using FlowiseAI (YC S23), Redis, and Qdrant to create custom agentic solutions with the #AskNews API. 🤖 He shows us how to: ➡ Implement a custom JS function to interface with AskNews REST API ➡ Use Redis-backed chat memory for maintaining conversational context ➡ Integrate external APIs into FlowiseAI workflows 🔗 Read the full article: https://lnkd.in/dbfmtpig

70

Check out this blog from postgreSQL expert, Ibrar Ahmed, "The Significance of Multi-Master PostgreSQL: Embracing Active-Active Replication Introduction." See how Ibrar delves into the concept of multi-master PostgreSQL, the key benefits to this architecture, and why it is a critical solution for modern applications offered by pgEdge. Read it here: https://hubs.la/Q02x3Ndl0 #pgedge #postgresql #postgresqlprofessionals #multimaster #activeactive

7

1 Comment

Check out this video from Sudarshan Koirala (Data Science Basics on YT) for a tutorial on integrating the Groq API (including our newly introduce Llava 1.57B model) with the OpenWebUI. https://hubs.la/Q02PVJh-0

32

1 Comment

Can we parallelise model training like transformers yet scale linearly to long sequence length just like RNN? 🤔 Yup, its possible with SSMs, infact they can do inference at computation cost of O(1). 🤯 First of all, how does it even compare with the most advanced Transformer capabilities like, - Self-Attention Mechanisms - Multi-Head Attention - Quadratic Scaling, etc. All of these have been outperformed on the new benchmarks across various domains such as language, audio, and genomics by Mamba. 🏆 Your model might be excelling with the above methods, but does it handle million-length sequences efficiently? Or process long sequences without breaking a sweat? Mamba efficiently integrates a selective state space model. What it effectively does for sequence processing is: ⚡ - Dynamically selects relevant information while discarding irrelevant data. - Operates with linear time complexity, ensuring fast and efficient processing. - Utilizes a hardware-aware parallel algorithm that maximizes GPU efficiency. - Incorporates simplified end-to-end neural network architecture without attention or MLP blocks. And during sequence modeling, - The input is processed through a selective mechanism, allowing the model to focus on the most relevant parts. - The state space parameters (such as Δ, B, and C) are dynamically adjusted based on the input, enhancing content-aware processing. - Efficient recurrent mode computation ensures faster training and inference with lower memory usage. But this approach doesn't just stop at sequence processing; it extends to real-world applications across different domains. For example: - Language modeling with Mamba outperforms Transformers of similar and even larger sizes in both pretraining and downstream evaluations. - Audio modeling with Mamba provides better quality and efficiency, making strides in speech recognition and generation. - Genomic sequence analysis with Mamba achieves state-of-the-art performance, handling long DNA sequences with ease. Mamba is not just an architectural improvement; it's a revolution in sequence modeling. I'll link the research paper down in comments, and you can explore its capabilities further. For more such detailed explanations, follow me. See ya! . . . #Mamba #SequenceModeling #AI #NeuralNetworks #Efficiency #StateSpaceModel #LanguageModeling #AudioModeling #Genomics #DeepLearning

24

𝐁𝐮𝐢𝐥𝐝𝐢𝐧𝐠 𝐘𝐨𝐮𝐫 𝐅𝐢𝐫𝐬𝐭 𝐍𝐞𝐮𝐫𝐚𝐥 𝐍𝐞𝐭𝐰𝐨𝐫𝐤 𝐰𝐢𝐭𝐡 𝐏𝐲𝐓𝐨𝐫𝐜𝐡 In continuation of our 𝐀𝐈 𝐢𝐧 𝐓𝐞𝐬𝐭𝐢𝐧𝐠 blog series, here is today's post : https://lnkd.in/gPNiwyV6 #Testing #TestAutomation #AIInTesting #PyTorch #NeuralNetworks #MachineLearning #DeepLearning #AI #ArtificialIntelligence #DataScience #TechTutorials #Programming #SoftwareDevelopment #Python #ML #DL #AIResearch #TechEducation

1

If you spend 5-10 mins on this post, then you can learn a really important topic in pyspark which a 1 hour video also cannot teach you this effectively. Lets say you have a 20 node spark cluster Each node is of size - 16 cpu cores / 64 gb RAM Let's say each node has 3 executors, with each executor of size - 5 cpu cores / 21 GB RAM => 1. What's the total capacity of cluster? We will have 20 * 3 = 60 executors Total CPU capacity: 60 * 5 = 300 cpu Cores Total Memory capacity: 60 * 21 = 1260 GB RAM => 2. How many parallel tasks can run on this cluster? We have 300 CPU cores, we can run 300 parallel tasks on this cluster. => 3. Let's say you requested for 4 executors then how many parallel tasks can run? so the capacity we got is 20 cpu cores 84 GB RAM so a total of 20 parallel tasks can run. => 4. Let's say we read a csv file of 10.1 GB stored in datalake and have to do some filtering of data, how many tasks will run? if we create a dataframe out of 10.1 GB file we will get 81 partitions in our dataframe. (will cover in my next post on how many partitions are created) so we have 81 partitions each of size 128 mb, the last partition will be a bit smaller. so our job will have 81 total tasks. but we have 20 cpu cores lets say each task takes around 10 second to process 128 mb data. so first 20 tasks run in parallel, once these 20 tasks are done the other 20 tasks are executed and so on... so totally 5 cycles, if we think the most ideal scenario. 10 sec + 10 sec + 10 sec + 10 sec + 8 sec first 4 cycles is to process 80 tasks all of 128 mb, last 8 sec is to process just one task of around 100 mb, so it takes little lesser but 19 cpu cores were free during this time. => 5. is there a possibility of, out of memory error in the above scenario? Each executor has 5 cpu cores and 21 gb ram. This 21 gb RAM is divided in various parts - 300 mb reserved memory, 40% user memory to store user defined variables/data. example hashmap 60% spark memory - this is divided 50:50 between storage memory and execution memory. so basically we are looking at execution memory and it will be around 28% roughly of the total memory allotted. so consider around 6 GB of 21 GB memory is meant for execution memory. per cpu core we have (6 GB / 5 cores) = 1.2 GB execution memory. That means our task can roughly handle around 1.2 GB of data. however, we are handling 128 mb so we are well under this range. I hope you liked the explanation :) My new big data batch is starting tomorrow. DM to know more. #bigdata #career #dataengineering #apachespark

732

26 Comments

Great blog by Grab on how they build a real time lakehouse with Apache Hudi, #Flink & Spark. Link: https://lnkd.in/dEFrMS9y #dataengineering #softwareengineering

62

1 Comment

DataStax ➕ Langflow make RAG 100x easier. Learn how to get started in this new video from Tejas Kumar. 🌟 Be sure to star our repo on Github! https://dtsx.io/social #RAGApplications #Langflow https://dtsx.io/3UnB7Q2

11

PyTorch 2.5 is here 🔥 PyTorch® 2.5 featuring: 🔥 a new CuDNN backend for SDPA, enabling speedups by default for users of SDPA on H100s or newer GPUs 🔥 regional compilation of torch.compile, offering a way to reduce the cold start up time for torch.compile by allowing users to compile a repeated nn.Module (e.g. a transformer layer in LLM) without recompilations 🔥 TorchInductor CPP backend offering solid performance speedup with numerous enhancements like FP16 support, CPP wrapper, AOT-Inductor mode, and max-autotune mode Read more in our PyTorch 2.5 Release Blog: https://hubs.la/Q02TRx6f0 Post Credit : PyTorch

2

1 Comment

Nice Blog from Alex Merced at Dremio on how Apache XTable works in action and how to sync XTable-synced tables to Dremio's native catalog or open source Nessie 🔥 🔗 Blog Link: https://lnkd.in/deSd5qHG #dataengineering #softwareengineering #lakehouse

20

Speaking at #DES24, Pavan Nanjundaiah, Head – Tredence Studio, said today’s data engineer needs a beginner’s mindset to adapt to the growing change. “Today’s data engineers should expand left and right. Focus on your core foundational skills because they are not going to change. You don’t need to handcraft your code anymore.   Understand what is the business problem you’re trying to solve. My recommendation to data engineers is to shift left, which is the person who gave you the requirement and probably understands the business a little better. By doing this, you’re now zooming out of a core data engineer’s profile and understanding the business and the longer impact you can create.”   Read more here: https://lnkd.in/gKhtBwp6 #DataEngineers #Tredence #BeyondPossible

201

1 Comment