Rohit Jain

In computing parlance, context switching is when the CPU can favour higher priority tasks. To pause a lower priority task, the CPU caches system instructions and data pointers. To resume it, the CPU refers back to the cached information. Many engineers, data scientists, and analysts also recognize context switching as a euphemism for pivoting from one task to another, usually at the request of product owners and stakeholders. Of course, humans are generally less efficient than computers. We lose days in caching the system state — documenting, preserving, annotating, etc. Likewise, we lose time resuming it — switching gears, reestablishing computing environments, relearning concepts, and more. When the cost of transitioning becomes too great, efficiency suffers. For data scientists, this can be particularly challenging because their work often requires deep concentration. Each time they switch tasks, it takes time to refocus (Diminishing attention, Depleting energy, and Confusing priorities) and get back into the flow, which can reduce their overall productivity and increase the chances of making mistakes. Any thoughts on how best you have tackled this issue? #TheInsightEdge

63

4 Comments

Battling CPU Saturation and Long GC Times: An SRE’s Journey  Last week, we ran into a classic challenge on a shared Linux box: CPU saturation and long garbage collection (GC) times wreaking havoc on our application’s performance. As an SRE, keeping things running smoothly under load is what we do, but this was a perfect storm of high CPU usage, memory management issues, and growing response times. Here’s how we tackled it step-by-step: ✅ Proactive Monitoring: First things first, we ensured all the right metrics were in place—CPU, memory, GC pause times, thread pools. Alerts were set for CPU spikes and excessive GC pauses. You can’t fix what you can’t see! ✅ Incident Response: When CPU saturation hit, we jumped into action with top, htop, and jstack to find the culprit processes and threads. Thread dumps and GC logs revealed that object churn and frequent Full GCs were driving our CPU to its limits. ✅ Immediate Fixes: We had to mitigate quickly, tuning the GC, shedding some non-essential traffic, and scaling resources horizontally to give the system some breathing room. ✅ Post-Incident Analysis: Once the fire was out, we went deeper—analyzing heap dumps, adjusting our JVM configs, and fine-tuning thread pools to ensure this wouldn’t happen again. It’s all about staying calm under pressure, using the right tools, and building resilience into the system. If you’re facing similar issues, make sure you have visibility, responsive strategies, and a game plan for long-term optimization in place! Any other SREs out there with CPU/GC war stories? Let’s compare notes! 🔧 Checklist for AD or Support Teams: Here’s how your Application Development or Support team can help to avoid or reduce CPU saturation and long GC times: • Optimize Object Creation: Reduce unnecessary object churn. Reuse objects where possible, especially in high-throughput areas like request handling. • Tune Garbage Collection Settings: Work with SREs to test and tune JVM GC settings. Consider switching to G1GC, ZGC, or Shenandoah for lower pause times. • Monitor Thread Pools: Ensure thread pool sizes are appropriate for the workload and number of CPU cores. Overloaded pools can increase CPU exhaustion. • Profile and Optimize Hotspots: Use profiling tools like YourKit or VisualVM to identify CPU-intensive code paths. Refactor inefficient algorithms to reduce CPU load. • Handle Cache and Memory Efficiently: Avoid holding onto objects unnecessarily (e.g., poor cache management), which can contribute to long Full GCs and memory bloat. • Ensure Proper Resource Provisioning: Collaborate with the infra team to make sure your application has adequate CPU and memory resources for the expected load. • Test Under Load: Perform load testing to identify potential performance bottlenecks before they reach production. #SRELife #PerformanceEngineering #Java #GarbageCollection #CloudOps #IncidentResponse”

10

3 Comments

Whenever #LangGraph persistence is discussed, it mostly talks about persisting state across nodes and edges throughout the workflow. It can be achieved even by in-memory persistence which is what is mostly talked, documented about. However, there is an additional subtle aspect to it which is to persist it on disk / in DB. It can help an application pick up where it had left off in case of a crash or restart.   I have done a YouTube video on it with details of how it can be done, its nuances, differentiating it from in-memory persistence, and how it can help any AI application. Appreciate any feedback. #GenAI

10

Meta's privacy practices aside, its contributions to the #OpenSource community can’t be left unrecognized. #Llama 🦙 is the foundation of most open-source #LLMs today, and ReactJS remains the top #FrontendDevelopment framework. Both cost a lot of money to develop and maintain, and are shared freely as a gift to humanity. Respect! #ArtificialIntelligence #AI #GenerativeAI #SoftwareEngineering #SoftwareArchitecture

1

Are you tired of struggling to understand complex architecture diagrams? Do you wish there was a way to make documenting them easier and more efficient? Using GenAI, Reading architecture diagram from an image and giving you all the description. >>> /Users/prakashsharma/Desktop/Single-tenant.jpg tell me what this image is about? Added image '/Users/prakashsharma/Desktop/Single-tenant.jpg' The diagram shows two separate AWS accounts (tenant 1 and tenant 2) with their own SaaS control planes, which are connected to each other using an AWS service called SaaS Control Plane. The tenants have multiple Availability Zones and subnets for redundancy. Key Features: Multi-AZ Deployment: Each account has two Availability Zones for high availability and disaster recovery. Subnet Configuration: Subnets are configured to ensure that instances can communicate with each other within the same zone. SaaS Control Plane Connection: The SaaS control planes of both tenants are connected, allowing them to share resources and services. Resource Sharing: Resources such as Auto Scaling groups, RDS instances, and Elastic Block Store (EBS) volumes can be shared between the two tenants. Benefits: Improved High Availability: With multiple Availability Zones, applications can continue running even if one zone goes down. Enhanced Disaster Recovery: Subnets are configured to ensure that instances can communicate with each other within the same zone. Simplified Resource Management: The SaaS control plane connection allows for easy sharing of resources and services between tenants. Conclusion: The diagram shows a well-designed architecture for multi-tenancy in AWS, with multiple Availability Zones and subnets configured to ensure high availability and disaster recovery. The SaaS control plane connection enables resource sharing and simplifies management across both tenants. >>>/exit

74

📢Semantic Kernel for Java 1.2 released 🆕Hugging Face and Gemini Integration 🆕Vector Store Databases 🆕Azure AI Search 🆕Redis Integration Read all about it 👉https://lnkd.in/dagkxH2Y Well done John Oliver, Bruno Borges and Derek Keeler #ai #openai #azure #microsoft #programming #development #java

28

Table Augmented Generation - In Action: LOTUS is LLM Over Table of Unstructured and Structured Data, which is a Python framework/library that combines the power of a relational model with an AI generative model. Typical Text to SQL generate query depends on the table metadata definition provided, after that if we want to further process the data using values in columns/rows and perform semantic operations the same as arithmetic operations, Lotus AI extends the pandas data frames API's and apply semantic operations 1. semantic filter 2. Semantic TopK 3. Sematic Join etc. We can go further and apply a fewshot learning and CoT (Chain of thought reasoning) as well. Head over to the link to see it in Action: https://lnkd.in/gcxwarZV Google Collab link to try it on your own: https://lnkd.in/gnjxCKmK

11

1 Comment

Amazing performance improvement with usage of integrated cache . Do read more about it in the blog below. 1️⃣ Improvement in latencies. 2️⃣P75 latency is down 75% and 3️⃣ P99.9 latency is down over 67% 4️⃣ Reduced infrastructure from 60k cpu cores to 3k https://lnkd.in/g6QA4YVm

24

If you are using SECTOR for LLM Evaluation then please upgrade to latest version as there is a API change that might be incompatible with newer version https://lnkd.in/gUBSztpE Previous version I had one pre- processing function for both LLM response and Context In current version I have seperate function for response and context. This will allow one to handle context document in different formats like plain text, json, word doc etc seperate of response formats You can go through the README file for more details Also, added Auto parameter search in new version. Just feed in your dataset and let the hyper parameter search do the magic for you to find best possible parameters tailored to your use case #artificialintelligence #generativeai

42

2 Comments

Dijkstra’s #algorithm a classic to solve the shortest path problem on positive graphs using a min-priority queue and greedy strategy https://lnkd.in/dpS-V-B3

41

10 Best Enterprise Software Stocks to Buy Now by Umar Farooq via Insider Monkey - Free Hedge Fund and Insider Trading DataInsider Monkey – Free Hedge Fund and Insider Trading Data – Free Hedge Fund and Insider Trading Data, Hedge Fund News, Financial Commentaries - ([Global] oracle cloud) URL: https://ift.tt/7V5Gyzt In this piece, we will take a look at the 10 best enterprise software stocks to buy right now. Enterprise software is integral to the functioning of modern businesses, offering a range of applications designed to streamline and optimize various essential activities. These include enterprise resource planning (ERP), customer relationship management (CRM), business intelligence (BI), supply chain management (SCM), and more. The market is structured around eight core segments based on the functionality of the software: Business Intelligence Software, Content Management Software, Customer Relationship Management Software, eCommerce Software, Enterprise Performance Management Software, Enterprise Resource Planning Software, Supply Chain Management Software, and other specialized enterprise software solutions. The market's evolution is driven by several key factors. Businesses worldwide are increasingly adopting digital transformation strategies, which are fueling demand for flexible, scalable, and user-friendly software solutions. Cloud-based software, or Software as a Service (SaaS), has become particularly popular due to its adaptability and ease of integration with other systems. Companies are also seeking software that can help them reduce costs, streamline operations, and improve their overall performance. Geographically, the United States leads the enterprise software market, driven by the high demand for cloud-based solutions and the rapid adoption of artificial intelligence (AI) and machine learning technologies. In Europe, the market is propelled by the need for digital transformation and compliance with regulations such as the General Data Protection Regulation (GDPR). The Asia-Pacific region is experiencing growth due to the increasing adoption of cloud-based solutions and the rise of small and medium-sized enterprises (SMEs). In China, the market is dominated by domestic companies due to government restrictions on foreign firms, while in India, the market benefits from the proliferation of mobile devices and startups. Latin America’s market is driven by the need for regulatory compliance and efficient business management solutions. The enterprise software market is influenced by various macroeconomic factors, including GDP growth, technological innovation, and government regulations. As businesses across the globe strive to stay competitive in an increasingly digital world, the demand for advanced software solutions continues to rise. The growing adoption of cloud-based platforms is particularly significant, as companies seek to enhance their flexibility while reducing operational costs. Additiona...

1

TAG — Table Augmented Generation: Welcome! It's great to see you all. A Couple of days back I read a research paper that proposed a novel approach to deal with both structured and unstructured data using Language models which is going to be transformational in the data engineering space.  [ LOTUS - Large Language Model Over Table of Structured and UnStructured Data ] First, let's get back to basics to understand the problem with existing approaches There are various ways to ask questions using natural language over custom data to get answers. Two of the most popular methods are. Text2SQL And RAG   RAG -  Answers provide answers by looking up one or a few specific records from the database or data source. It's the point in lookup.   Text2SQL - This method provides answers by converting natural language to SQL query that could be executed on a database.   These methods often fall short of complex questions requiring domain knowledge, semantic reasoning, and world knowledge. I know this is not clear, let’s understand with an example - Take the movie database table containing columns for movie names, gross revenue generated, reviews, and genre. If the user asks the question “Find the highest-grossing romantic movie’ “ Text2SQL generate query like “select max(revenue) from movie where genre=’romance’ “ What if the user asks a question like “summarize the reviews of the highest-grossing romance movie considered a 'classic' “ This question is challenging for Text-To-SQL and RAG systems as it requires not only finding the highest-grossing romance movie but also determining whether it’s a classic or not using world knowledge. While LLM UDF offers logical row-wise reasoning but does not support patterns across rows. Enter the TAG — Table Augmented Generation — it’s a three-step approach Query Synthesis — Translates the user’s query into a database query. In this example, it will generate a query with the filter of genre = romance and order by revenue. Query Execution — Query executed against the database to retrieve relevant data and use language model knowledge to consider it as a classic or not. Answer Generation — Uses the retrieved movie data with a review column and original user query to generate a natural language answer. Paper # https://lnkd.in/gsi6cPsb https://lnkd.in/gkHAuqW5 https://lnkd.in/gZMrytA3 #llm #tag #rag #lotus #tableaugumentdgeneration #semanticoperators #LLMUDF #

24

1 Comment

Valkey: A Robust, Redis-Alternative for Modern Applications! Redis is no longer fully open source, prompting major tech players such as Amazon, Google, Huawei, and Ericsson to explore alternatives. This shift has led to growing support for ValKey, a flexible and secure open-source solution, increasingly recognized as a strong contender in the space. ValKey 8.0's Groundbreaking features: 1. Introduced dual-channel replication and migration state tracking, minimizing data loss during node migrations or rearrangements. 2. Enhanced throughput to 1.2 million requests per second using multi-core and asynchronous I/O threading, surpassing Redis's single-threaded approach. 3. Added per-slot metrics infrastructure, enabling granular performance insights to optimize resource allocation and load balancing. AWS has fully integrated ValKey into its services, including ElastiCache and MemoryDB, while significantly lowering costs: - 33% reduction for ElastiCache Serverless, - 20% for node-based ElastiCache, and - 30% for MemoryDB. The database landscape is evolving, and transitioning to ValKey is a compelling option to consider, whether you're operating on AWS or another platform. Share your suggestions or experiences with in-memory datastores designed to deliver lightning-fast performance and handle massive throughput effectively. https://lnkd.in/gPZjHX3W #Valkey #Redis #InMemoryDB #TechInnovation #FlexibleLicensing #DataSecurity #ModularDesign #DataPersistence #CloudTech

11

What's New ? OpenAI released the new GPT-4o-2024-08-06 model, which supports Structured Outputs. This model achieves perfect scores in JSON schema adherence, ensuring reliable and consistent output formats. It offers significant cost reductions and improved performance. The model features an increased context window of 16,384 output tokens, up from 4,096 tokens. Cost Efficiency and Key Features The GPT-4o-2024-08-06 model offers significant cost reductions and improved performance: 50% lower input costs: $2.50 per million tokens 33% lower output costs: $10.00 per million tokens Increased context window: 16,384 output tokens #OpenAI

5

Excellent read on performance issue debugging!! Step by step breakup of various approaches along with success and failure and validation of each approach.. Finally, the findings are also amusing and ironical. The resource utilisation plugin itself leading to muti sec delay in UI https://lnkd.in/gdhCWnAJ

31

This provides an excellent explanation of a fascinating GitHub bug and the steps taken to resolve it. Loved it! The intricacies of distributed systems are particularly challenging to grasp, requiring a deep focus on even the smallest details to fully understand their complexities. #redis #ratelimit #cache

3

💡 Boosting Application Reliability with the “minReadySeconds” Parameter (Kubernetes) 💡 . Beyond the standard container probing parameters: . 📍startupProbe: Determines when to initiate readiness and liveness probes 📍livenessProbe: Checks if a Pod is still functioning correctly and if it should be restarted 📍readinessProbe: Verifies if a Pod is ready to receive requests and serve traffic Or 📍PreStop hook: This allows you to specify a command or action to be executed before a Pod is terminated. It's often used to introduce a pause or sleep period, enabling other processes, such as updating endpoint configurations (like iptables), to complete before the Pod is shut down 📍terminationGracePeriodSeconds: This parameter sets a time window (in seconds) after which a SIGKILL signal will be sent to forcefully terminate a Pod if it hasn't already terminated gracefully after receiving the initial SIGTERM signal . These parameters contribute to application uptime and facilitate zero-downtime deployments during rolling updates involving image changes or resource setting modifications . Another valuable parameter for enhancing application reliability is 💡minReadySeconds💡 (.spec.minReadySeconds) . What is minReadySeconds? This parameter defines the minimum time (in seconds) a Pod must remain in the "Ready" state (after successfully passing its readinessProbe) before it is considered fully operational and eligible to receive traffic . ℹ️ The `minReadySeconds` value can be specified within various Kubernetes resources, including Deployments, DaemonSets, and StatefulSets ℹ️ The default value for `minReadySeconds` is 0. This means that as soon as a newly created Pod transitions to the "Ready" state, the older Pod will immediately begin its termination process . ℹ️ Use Case: In certain scenarios, merely passing the readinessProbe might not be sufficient for an application to be considered fully prepared. Some applications, such as those built with Java, might require a warm-up period before handling the full traffic load. This is where `minReadySeconds` proves beneficial . For further insights into Kubernetes best practices, you can refer to: https://lnkd.in/gi4WCTdS #kubernetes #reliability #minReadySeconds

33

1 Comment