TiDB, powered by PingCAP’s Post

This testiminial offers great insight. Many customers are suffering the same cloud database scaling issues. #Cloud #mysql #aws #awsaurora #azure #gcp #Spanner #distributedsql #htap

The growing number of demands from business verticals and offerings introduces complex microservices and dependency call graphs. As a result, applications demand low latency, higher performance, and scalability from the database, while simultaneously generating higher workloads. I was recently reading an article on the challenges that Uber's distributed database "𝐃𝐨𝐜𝐬𝐭𝐨𝐫𝐞" (MySQL based) faces to serve low-latency requests at high scale. Here are few I found interesting: 1. 𝐒𝐩𝐞𝐞𝐝 𝐨𝐟 𝐝𝐚𝐭𝐚 𝐫𝐞𝐭𝐫𝐢𝐞𝐯𝐚𝐥 𝐟𝐫𝐨𝐦 𝐝𝐢𝐬𝐤 𝐡𝐚𝐬 𝐚 𝐭𝐡𝐫𝐞𝐬𝐡𝐨𝐥𝐝: There’s a limit to how far one can optimize application data models and queries to improve database latency and performance. Beyond that, squeezing out more performance is not possible. 2. 𝐕𝐞𝐫𝐭𝐢𝐜𝐚𝐥 𝐬𝐜𝐚𝐥𝐢𝐧𝐠: Assigning more resources or upgrading to better hosts with higher performance has its limitations where the database engine itself becomes a bottleneck. 3. 𝐇𝐨𝐫𝐢𝐳𝐨𝐧𝐭𝐚𝐥 𝐬𝐜𝐚𝐥𝐢𝐧𝐠: Splitting shards further across more numerous partitions helps solve the challenges to an extent however doing so is an operationally more complex and lengthy process. We have to ensure data durability and resiliency without any downtime. Also this solution doesn’t fully help to solve the issues of hot keys/partitions/shards. 4. 𝐑𝐞𝐪𝐮𝐞𝐬𝐭 𝐢𝐦𝐛𝐚𝐥𝐚𝐧𝐜𝐞: Oftentimes the incoming rate of read requests is orders of magnitude higher than write requests. In such cases, the underlying MySQL node will struggle to keep up with the heavy workload and further impact latencies. 5. 𝐂𝐨𝐬𝐭: Vertical and horizontal scaling to improve latencies are costly in the long term. Additionally, scaling doesn’t fully address the problem. 𝐖𝐡𝐚𝐭 𝐜𝐨𝐦𝐞𝐬 𝐭𝐨 𝐫𝐞𝐬𝐜𝐮𝐞: Caching 😅 #distributedsql #lowlatency #uber #caching #scalability #mysql

Don't let database issues slow you down. Dive into the world of MySQL master-slave replication and unlock the full potential of your database infrastructure. With Docker, the process becomes a breeze. Simplify deployment, improve scalability, and ensure data consistency effortlessly 💁♀️ Full of the guidline👉👉https://lnkd.in/gYm5zxEf #ISBVietnam #IVC #Outsourcing #Softwaredevelopment #MySQL #Docker #Masterslavereplication

Good Job versus Sidekiq with Redis It is almost mandatory for a Ruby developer to manage background tasks using the combination of Redis and Sidekiq. However, some small businesses face budgetary constraints and require alternative solutions for background tasks. At Purple Stock, we utilize the GoodJob gem to handle cron jobs and background tasks. GoodJob works well for API integration as well. Our dashboard monitors purchases grouped by different third-party stores, synchronizing data from third-party APIs. The service is managed by the GoodJob Gem. Through monitoring with Sentry, I’ve concluded that fine-tuning memory and performance issues is very similar to those encountered with Redis and Sidekiq Let me explain some improvements . We divided job tasks into distinct responsibilities. Specifically, we separated the job responsible for creating new records from the one responsible for updating their status. This division proved invaluable when dealing with inconsistencies. We further segregated the job updates based on status. For instance, if a canceled status existed in the integrated ERP but not in our application, we could quickly pinpoint the issue—perhaps the “CanceledStatusUpdater” had made an incorrect adjustment. This approach also helped us identify memory leaks during Black Friday events. In summary, our strategy revolves around limiting the responsibilities of each background job. The narrower the scope, the easier it becomes to manage. Another effective tactic we implemented was raising exceptions. Prior to my changes, the system relied on a try-catch strategy when a background task encountered an exception. Unfortunately, this approach hindered thorough investigation of logs. Sentry proves invaluable for identifying issues like N + 1 queries, memory leaks, and bugs. Given that our application is hosted on Heroku with a basic memory quota plan, paying attention to Sentry’s memory reports is crucial. Wrap it up At the end, you will face the same problems and a base in computer programming and how ruby works will be very important for inheritance decisions. #ruby #developer #rubyorails #backend #ROR

🚀 Check out my step-by-step guide how to effortlessly set up MySQL and MongoDB on your local machine using Docker Compose! #Docker #MySQL #MongoDB #DevOps #LocalDevelopment #Tutorial

Vendor lock-in restricts you, stifles progress and limits your options! Avoid lock-in with FerretDB - the open-source alternative to MongoDB Find out more on why developers need FerretDB! Link:

As part of my new experience in #developing MongoDB-based applications in VIQ Systems, I have started running performance tests to evaluate the performance of queries to this #NoSQL database. I consider these tests critical as they allow us to ensure that our applications can scale and maintain a high level of performance under significant workloads. Using tools like #ApacheJMeter and #MongoDB's own #PerformanceAdvisor, I have designed realistic #test scenarios that simulate the behavior of #concurrent users performing complex queries and CRUD operations in our MongoDB #database. These tests allow us to measure key metrics such as query response time, server resource utilization, and horizontal scalability of the infrastructure. In summary, I consider #performance testing in MongoDB crucial in our development process, ensuring that our applications can maintain a high level of performance even in demanding production environments. Thanks Jesus Yasser Arguelles Guarnalusse for allowing me to be part of this great project.

🌊 Upgrading your Amazon RDS MySQL from 5.7 to 8? 🏄♂️ There are a lot of moving parts, and things to take into consideration. 🌴 Read our latest blog written by Alon Kafry, and learn about the challenges from a guy who upgraded hundreds of DB instances 🤓

At .NET Day 2024, we dove deep into #AzureSQL with loads of sessions! 😎 I've said it before, and I'll say it again: SQL skills are like a secret weapon for devs! 💪 It's popping up everywhere, and our event was jam-packed with data talks. Check out the sessions our team rocked - they're chock-full of stuff you can start using today! 🚀 - GitHub Copilot for SQL Development: Integrating the Power of AI into Database Development - Subhojit Basak - Event-Driven Architectures with Azure SQL, .NET and Azure Functions - Davide Mauri - T-SQL for cloud-native developers - Abhiman Tiwari - Unlocking Scalability: Azure SQL DB Hyperscale and the Power of Named Replicas - Attinder Pal Singh - Create a Change Data Stream in Minutes with .NET, Azure SQL, and Azure Functions - Brian Spendolini - Auto-Generate and Host Data API Builder on Azure Static Web Apps - Frank Boucher & Jerry Nixon - The most minimal API code of all... none - Frank Boucher & Jerry Nixon

Because Postgres and MySQL are designed to scale vertically, rather than horizontally, it's only a matter of time before things start creaking. Because we're wire-compatible, scaling your backend doesn't need to take months of re-engineering. See what the migration path can look like below.