What keeps me fascinated about distributed systems is the plethora of approaches for you to solve a bottleneck problem. With distributed systems, your sole job is to keep moving the bottleneck to inexpensive layers, or to get rid of them completely. Having options to solve bottleneck problems is important. For instance, many practitioners believe RDMA is the future of scalable computing. However, despite of the benefits of the approach, it requires substantial investment and set up to get it working. You need specialized NIC supporting InfiniBand, RoCE, or iWARP. Moreover, you need to use a software stack that knows how to talk to low-level verbs exposed. Other approaches are also available. The blog post below describes some strategies you can pursue in different hardware layers, such as networking, storage, and compute. Approaches like UDP-based protocols like QUIC, Smart NICs, Zonal SSDs, NVMe with the newer copy command, Unikernels, and TrueTime are described, along with their inherent trade-offs. However, despite the advances each one of these technologies may provide, the sad reality is their availability is limited in many computing environments. Including the cloud.
