Tarun Dua

Cloudflare Dev offering is the best in the entire Tech Industry. Why is no one talking about it. Alternative to Heroku, Netlify & Vercel is Cloudflare workers & Cloudflare Pages. They are also the best alternative to LLM Stack like Together AI, Replicate AI, Anyscale AI etc. You could get so much done for so less. My own personal portfolio website is hosted on Cloudflare Workers for the past 6 months. Ofcourse Cloudflare doesn't do marketing and thus thats why they have the cheapest offering in the industry with very big free limits. Just check this out - https://lnkd.in/dcYiKrjM Founders and builders should definitely check it out.

6

2 Comments

A super clear, straight forward example of how to do cost effective RAG using LangChain for your document pipeline and Groq for compute resources. The code demonstrates how to speed up RAG and reduce costs using quantized embeddings (reducing 32 bit values to 8) Nice job Sahibpreet Singh! https://lnkd.in/gf5QMCKK

44

2 Comments

Understanding How OpenTelemetry Network Uses eBPF for Network Observability   🎯 Key Innovations:   - Integration of eBPF with OpenTelemetry for seamless network observability.   - Use of an enhanced eBPF runtime embedded in the Linux kernel.   - Components that support data collection, enriching, and exporting telemetry data efficiently.   💡 Notable Features:   - Kernel collector: Collects low-level telemetry using eBPF with minimal overhead.   - Kubernetes collector: Monitors Kubernetes API events to gather container-related metrics.   - Cloud collector: Gathers metadata and network events from cloud providers like AWS and GCP.   - Reducer component: Aggregates and enriches data from various collectors to provide aggregated insights.   🛠️ Perfect for:   - DevOps Engineers   - Cloud Architects   - Network Security Professionals   - System Administrators   ⚡️ Impact:   - Enhanced visibility into network performance and anomalies.   - Low resource overhead in monitoring workloads.   - Improved troubleshooting capabilities for network-related issues.   🔍 Preview of the Talk:   In this session, Shivanshu Raj Shrivastava and Jonathan Perry discuss the transformative role of eBPF in network monitoring and observability through the OpenTelemetry Network project. Attendees will gain insights into eBPF's architecture, including its components, how it enables continuous monitoring, and a practical demo illustrating end-to-end telemetry data flow. Key takeaways include understanding various collectors that bolster network visibility while maintaining minimal resource usage, alongside an invitation to participate in the OpenTelemetry community.   Watch the full session video: https://lnkd.in/gY_trgTh

1

Best Practices for Deploying LLM Inference, RAG and Fine Tuning Pipelines on K8s   🎯 Key Innovations:   - Advanced management of large language models (LLMs) lifecycle on Kubernetes.   - Use of inference servers for seamless deployment and auto-scaling of models.   - Integration of retrieval-augmented generation (RAG) with embeddings and vector databases.   💡 Notable Features:   - Customized inference pipelines utilizing NVIDIA's Nim operator and KServe.   - Efficient scheduling techniques for GPU resources with dynamic resource allocation.   - Enhanced security through role-based access control (RBAC) and monitoring capabilities.   🛠️ Perfect for:   - AI/ML Engineers deploying models in production.   - Data Scientists involved in fine-tuning and inference tasks.   - DevOps teams managing cloud-native applications on Kubernetes.   ⚡️ Impact:   - Reduced inference latency via effective model caching techniques.   - Improved GPU utilization optimizing resource allocation and scheduling.   - Increased security and manageability of AI pipelines in enterprise settings.   🔍 Preview of the Talk: In this insightful session, Meenakshi Kaushik and Shiva Krishna Merla from NVIDIA share comprehensive best practices for deploying and managing LLM inference pipelines on Kubernetes. They delve into critical challenges such as minimizing inference latency, optimizing GPU usage, and enhancing security measures. Attendees gain actionable insights on building customizable pipelines and leveraging NVIDIA’s technology stack to ensure efficient model management, ultimately leading to significant performance improvements. For more details, check out the full session here: https://lnkd.in/gRK7zPTM

7

Enjoy your Sunday learning Observability in Platform Engineering Using Inspektor Gadget (a tool and framework for data collection and system inspection on Kubernetes clusters and Linux hosts using eBPF). Thanks to Qasim Sarfraz and Maya Singh for the comprehensive introduction and real-world use cases for optimizing DevEx with Inspektor gadget for your observability needs. What Inspektor Gadget is and how it works, get exposure to several use cases that address common observability challenges that platform engineers face, and have clear next steps to get started with Inspektor Gadget yourself to improve your platform monitoring and observability. Watch the recording https://lnkd.in/dMxDP26b

6

My session on "Debiasing the Data Lifecycle" from the recent PyTorch Conference is now live! It's a quick 11-minute video. Check it out and let me know what you think 😃 #pytorch #dataBias #womenInData

8

2 Comments

🚨 Preparing for Fault Proofs on Base Mainnet 🚨 Exciting changes are on the horizon for Base Mainnet, with fault proofs expected to go live on 30th October! 🗓️ Fault proofs are a crucial step towards decentralization, allowing for a more open and community-driven approach to validating L2 state. Here’s how they’ll enhance the network: 🔍 Permissionless Output Proposals: In traditional L2s, only centralized proposers could submit output roots. With fault proofs, anyone can propose output roots for Base's state, breaking down centralization barriers. ⚔️ Permissionless Challenges: Anyone can now challenge a claim if it appears faulty or fraudulent, enhancing security and trust. These changes mean that anyone can withdraw funds from Base to L1 without depending on centralized actors. 🌐💸 What's Changing for Withdrawals: Withdrawals before the upgrade: Must wait the standard 7-day challenge period before finalization. Withdrawals during/after the upgrade: Will be proven using the Fault Proof system, with a general 7-day finalization period (potential delays if challenged). Note: bridge.base.org now redirects to Superbridge and other "Superchain bridges" for managing deposits and withdrawals. Check out our docs for more details. 🌍🚀 These updates are setting the stage for a more decentralized, secure, and community-governed Base Mainnet. Let’s embrace this exciting step towards decentralization! #Blockchain #DeFi #L2 #Decentralization #Crypto #Ethereum #BaseMainnet #Bridging #Layer2 #Web3 #SmartContracts #FaultProofs #DeFiSecurity #BlockchainScaling #EthereumEcosystem 

3

In this article, you'll learn how the PvcAutoscaler: - Improves cost efficiency and reduces toil for stateful workloads in Kubernetes. - Addresses StatefulSet and PVC limitations, enabling volume expansion, shrinking, and modification. More: https://lnkd.in/gbMPHRzZ

112

2 Comments

𝗕𝘂𝗶𝗹𝗱𝗶𝗻𝗴 𝗦𝗺𝗮𝗿𝘁𝗲𝗿 𝗔𝗽𝗽𝗹𝗶𝗰𝗮𝘁𝗶𝗼𝗻𝘀 𝘄𝗶𝘁𝗵 𝗚𝗲𝗻𝗔𝗜 𝗮𝗻𝗱 𝗥𝗔𝗚 𝗣𝗮𝘁𝘁𝗲𝗿𝗻𝘀! At #ServerlessDays2024BLR, Richa Gaur, Product Manager for Azure Cosmos DB at Microsoft, shared how to create serverless GenAI applications using the Retrieval-Augmented Generation (RAG) pattern. 𝗞𝗲𝘆 𝗧𝗮𝗸𝗲𝗮𝘄𝗮𝘆𝘀: ✅ 𝗥𝗔𝗚 𝗳𝗼𝗿 𝗥𝗲𝗮𝗹-𝗧𝗶𝗺𝗲 𝗔𝗜: Enhance AI applications with advanced data retrieval. ✅ 𝗔𝘇𝘂𝗿𝗲 𝗙𝘂𝗻𝗰𝘁𝗶𝗼𝗻𝘀: Simplify compute workflows with scalable, serverless solutions. ✅ 𝗔𝘇𝘂𝗿𝗲 𝗖𝗼𝘀𝗺𝗼𝘀 𝗗𝗕: Power applications with high-performance, globally distributed serverless storage. 𝗧𝗵𝗶𝘀 𝘀𝗲𝘀𝘀𝗶𝗼𝗻 𝗶𝘀 𝗮 𝗺𝘂𝘀𝘁-𝘄𝗮𝘁𝗰𝗵 𝗳𝗼𝗿 𝗱𝗲𝘃𝗲𝗹𝗼𝗽𝗲𝗿𝘀 𝗮𝗻𝗱 𝗮𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘀 𝗮𝗶𝗺𝗶𝗻𝗴 𝘁𝗼 𝗰𝗿𝗮𝗳𝘁 𝗲𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝘁 𝗮𝗻𝗱 𝘀𝗰𝗮𝗹𝗮𝗯𝗹𝗲 𝗚𝗲𝗻𝗔𝗜 𝘀𝗼𝗹𝘂𝘁𝗶𝗼𝗻𝘀 𝘂𝘀𝗶𝗻𝗴 𝘀𝗲𝗿𝘃𝗲𝗿𝗹𝗲𝘀𝘀 𝗮𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲. 🔔 𝗦𝘂𝗯𝘀𝗰𝗿𝗶𝗯𝗲 𝘁𝗼 𝗔𝗻𝘁𝗦𝘁𝗮𝗰𝗸 𝗳𝗼𝗿 𝗺𝗼𝗿𝗲 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀: https://bit.ly/47fDoCQ #ServerlessDays2024 #GenAI #RAGPattern #AzureFunctions #AzureCosmosDB #TechInsights #AIApplications #ServerlessInnovation

7

As engineering leaders integrate LLMs, robust tracking and monitoring are crucial for: 🔐 Mitigating risks (security, privacy, compliance) 💰 Optimizing costs and performance   🔍 Enabling debugging and model management 🌟 Monitoring LLM with Weaved/Instrumented Agents and AOP: - 🤖 Agents monitor LLM calls using separate scripts - 🧩 AOP intercepts calls without changing original code   - 🤝 Engineers focus on LLM development, AOP handles monitoring - 🌐 Works with any LLM framework, follows OpenTracing standards - 📈 Integrates with Jaeger for end-to-end tracing and visualization - 🔍 Tracks prompts, arguments, and replies Benefits of AOP for LLM applications: - Separation of Concerns - Non-invasive Instrumentation - Centralized Monitoring - Consistency and Standardization - Flexibility and Extensibility 💡 Best Practices for Planning Agentic Systems: 1. ⚠️ Vague sub-tasks, CoT failure, expensive ToT prompting 2. ✅ Use Planner with feedback loop 3. 🧩 Decouple components for production-grade agents   4. 🎯 Use domain-specific models  5. 🌊 Continuous memory support for planning 6. 🔍 Explore symbolic planning and external memory 🚀 Combining agent monitoring and agentic planning ensures reliable, efficient, and optimized LLM applications with comprehensive tracking, shared responsibility, and ethical oversight. 🌟 #LLMMonitoring #AOP #AgenticSystems #OpenTracing #Jaeger #LLM #LLMGuardRails #LangChain Engineering, just write below code, see video to AOP magics kicksin: llm_chain = LLMChain(prompt=prompt, llm=llm) question = "What is the capital of France?" result = llm_chain({"question": question}) See attached demo #Gonnect #LLMMonitoring #AOP #AgenticSystems

11

🚀 In an effort to enhance the OpenTelemetry ecosystem, we are open sourcing Zap OTLP, an encoder and sync for Zap - a popular structured logging library for Go. 🎉 You can use this plugin to send Zap logs to any observability backend which can understand OTLP format like SigNoz. Zap OTLP is designed to enhance logging capabilities in Go applications by enabling the transmission of logs from the Zap logger to an OpenTelemetry Protocol (OTLP) endpoint. This integration allows developers to seamlessly incorporate their application logs into any tool that supports OTLP. The plugin provides a straightforward setup process where users can create two encoders: one for console output and another for OTLP. 👉 It also includes functionality to initialize an OTLP sync, which handles the actual sending of log data to the specified OTLP endpoint. The sync can be configured with options such as batch size, resource schema, and service name, allowing for customization of how logs are grouped and identified. 👉 Additionally, the plugin offers features like batching logs for efficient transmission, setting batch intervals, and the ability to include trace context in log entries. This makes it particularly useful for applications that require detailed logging with distributed tracing capabilities. The plugin also supports secure connections and can be easily configured to work with any OpenTelemetry native observability backend like SigNoz. Do have a look and let us know what you think? Feel free to start discussions in the repo PS : Github repo link in comments

118

7 Comments

The role of databases in application design is crucial, especially when dealing with RDBMS or NoSQL systems where configurations are equally significant. Factors like high availability, data accuracy, and consistency are paramount in meeting the application's requirements. Naishadh Patel, from Ansi ByteCode LLP, sheds light on a key configuration aspect in the NoSQL database Azure Cosmos DB – Consistency Levels. He outlines the 5 levels as follows: 1. Strong 2. Bounded Staleness 3. Session 4. Consistency Prefix 5. Eventual For more information on this topic, you can visit https://lnkd.in/dnb8AVNT.

47

Utilizing GenAI at a large scale? Inferencing is actually a storage intensive workflow that can benefit from a high performance filesystem. With WEKA we go through a real customer utilization of a dynamic inference farm and how can they increase utilization / drop costs / lower prompt latency #genai #ai #weka #inference

33

How LLDB Powers Debugging and Optimization ⚙️ Have you ever wondered how developers uncover the hidden workings of macOS? 🖥️ Tools like LLDB are indispensable for debugging and reverse engineering, allowing engineers to dive deep into system internals and optimize applications. Apple itself uses LLDB to analyze and test its internal functions, setting the standard for macOS development. A notable example comes from Google, where developers used advanced debugging techniques to enhance Chrome’s interaction with macOS APIs. By analyzing system calls with LLDB 🛠️, they pinpointed and fixed memory leaks, dramatically improving browser performance. Similarly, game developers often rely on LLDB 🎮 to reverse-engineer graphics APIs, ensuring seamless performance for demanding applications on macOS. If you’re exploring macOS development or looking to sharpen your debugging skills, LLDB is a tool you can’t ignore. 🚀 This article offers practical insights and step-by-step guidance for using LLDB to dissect macOS functions and better understand the platform’s architecture. Read the full article here 👇 #LLDB #Debugging #ReverseEngineering #Development #Programming #Optimization #Tech

1

1 Comment

#5 𝑲𝒆𝒚 𝑹𝒆𝒂𝒔𝒐𝒏𝒔 𝒇𝒐𝒓 𝑾𝒉𝒚 𝑺𝑳𝑶𝒔 𝒂𝒓𝒆 𝒉𝒊𝒈𝒉𝒍𝒚 𝒆𝒇𝒇𝒆𝒄𝒕𝒊𝒗𝒆 𝒕𝒉𝒂𝒏 𝑴𝑻𝑻𝑹 𝒇𝒐𝒓 𝒎𝒆𝒂𝒔𝒖𝒓𝒊𝒏𝒈 𝒂𝒏𝒅 𝒊𝒎𝒑𝒓𝒐𝒗𝒊𝒏𝒈 𝑰𝑻 𝒓𝒆𝒍𝒊𝒂𝒃𝒊𝒍𝒊𝒕𝒚? 𝟭. 𝗠𝗲𝗮𝘀𝘂𝗿𝗶𝗻𝗴 𝗨𝘀𝗲𝗿 𝗘𝘅𝗽𝗲𝗿𝗶𝗲𝗻𝗰𝗲- SLOs focus on overall service performance from the end user’s perspective while MTTR only measures time to resolve incidents.  𝑬𝒙𝒂𝒎𝒑𝒍𝒆: A streaming service with a latency SLO, '99.9% of videos should start playing within 1 sec'. Focusing only on MTTR would not be able to measure & report slow start times impacting end users, whereas SLO helps in improving video load time and improve user satisfaction. 𝟮. 𝗕𝗲𝗶𝗻𝗴 𝗣𝗿𝗼𝗮𝗰𝘁𝗶𝘃𝗲- As SLOs are used to set targets on availability / latency / error rate, it allows teams to measure and optimize before failures occur in real time instead of reactive tracking of MTTR. 𝑬𝒙𝒂𝒎𝒑𝒍𝒆: An Edutech platform sets an SLO for 99.99% uptime. By monitoring this target, engineering team can focus on optimization opportunities without waiting for incidents to occur/customer complaints. As MTTR is measured after failures, it can be hardly used as a proactive indictor to prevent incidents. 𝟯. 𝗘𝗻𝘀𝘂𝗿𝗶𝗻𝗴 𝗮𝗹𝗶𝗴𝗻𝗺𝗲𝗻𝘁 𝗯𝗲𝘁𝘄𝗲𝗲𝗻 𝗜𝗧 𝗮𝗻𝗱 𝗕𝘂𝘀𝗶𝗻𝗲𝘀𝘀 - SLOs marry service reliability to business objectives, this helps IT to prioritize user impact creating great alignment with business, while MTTR is purely operational & doesn't align with business stakeholders' requirements. 𝑬𝒙𝒂𝒎𝒑𝒍𝒆: A healthcare platform ties its SLO for transaction processing speed (99% of transactions within 1 second), enhancing customer trust that makes business stakeholders happy but MTTR wouldn’t provide insights on user impact. 𝟰. 𝗗𝗮𝘁𝗮-𝗗𝗿𝗶𝘃𝗲𝗻 𝗱𝗲𝗰𝗶𝘀𝗶𝗼𝗻 𝗺𝗮𝗸𝗶𝗻𝗴 𝗳𝗼𝗿 𝗯𝘂𝗶𝗹𝗱 𝗿𝗲𝗹𝗲𝗮𝘀𝗲 - SLOs provide actionable insights for balancing innovation & release velocity with system reliability using error budgets (enforced through a error budget policy), whereas MTTR focuses only on operational incident response efficiency. 𝑬𝒙𝒂𝒎𝒑𝒍𝒆: A SaaS provider uses SLO error budgets to decide whether to release new features or hold on until measurable improvement in system reliability falls in place. Relying only on MTTR wouldn’t offer a balanced view of risk & may lead to increase in incidents. 𝟱. 𝗢𝗽𝘁𝗶𝗺𝗶𝘇𝗲𝘀 𝗢𝗽𝗲𝗿𝗮𝘁𝗶𝗼𝗻𝗮𝗹 𝗘𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝗰𝘆- SLOs enforce continuous monitoring and optimization, leading to focus on performance improvements, increases collaboration to transform service reliability. 𝑬𝒙𝒂𝒎𝒑𝒍𝒆: An ecommerce service uses SLOs for latency, error rate and throughput, guiding continuous improvements. This proactive approach decreases incident rates over time, unlike MTTR where the focus is on incident resolution time measuring ongoing efficiency. In overall, SLOs are transforming IT operations by offering a proactive approach driving better alignment btw engineering goals and business outcomes #SLO #SRE #MTTR

16

3 Comments

Leveraging eBPF for #MySQL query latency and index usage troubleshooting provides deep insights with minimal overhead. This advanced technique uses #Linux's tracing capabilities to monitor system and application metrics effectively. #MySQLPerformance @MinervaDB #dba https://zurl.co/goWN

9

7 days left for SigNoz Launch Week 2.0! At SigNoz, we provide logs, metrics, and traces under a single pane of glass. Powered by OpenTelemetry’s semantic conventions, the correlation of these signals gives developers the right context to troubleshoot issues quickly. Tired of switching between multiple tools to monitor your system? SigNoz has got you covered. 💪🏻 Secure your spot today- RSVP link in comment

27

1 Comment