Brendan Bouffler

QCon London 2023: Azure Cosmos DB: Low Latency and High Availability at Planet Scale Mei-Chin Tsai and Vinod Sridharan discuss the internal architecture of #AzureCosmosDB and how it achieves high availability, low latency, and scalability. https://lnkd.in/e36nHbS3

1

As we enter the #AI era #systemsarchitecture has to adapt to the new reality of #hybridcloud and #multicloud. Wyatt Gorman HPC Solutions Manager for Google Cloud joins me in the first episode of #partlycloudy to discuss #manageability challenges related to #aiinfrastructure https://lnkd.in/ga25iFFk

6

1 Comment

Numerical Weather Prediction has been one of the standard workloads of #hpc for decades. And my Colorado neighbors over at NSF NCAR - The National Center for Atmospheric Research have been key to advancing research in this area. One of their most popular developments in support of these efforts is the open source Weather Research and Forecasting (WRF) model, with > 50k users from over 160 countries. See below for detailed instructions on how you can perform your own #weather prediction on Amazon Web Services (AWS) using AWS ParallelCluster, Amazon FSx for Lustre, and WRF.

44

It's disappointing to see the news that the CMA has chosen to renew its contract with AWS – despite currently investigating the impact that hyperscaler anti-competitive practices are having on UK cloud.   In all honesty, it’s a huge blow to all those operating in UK cloud. This could have been a way to promote competition in the UK market and support the challenger and homegrown providers, rather than signing the biggest deal yet with the hyperscaler provider.   I shared my thoughts on the situation with Christine Horton, who explored the impact of agreements, like the UK’s One Government Value Agreement 2.0 (OGVA), have had on public sector IT spending.   There’s an election on the horizon and whoever’s in No.10 this time next year needs to be considering this with some urgency.    Take a look at Christine’s article in Channel Futures here: https://lnkd.in/ek3H4743

18

6 Comments

DB2 modernization is as easy as …..

8

In this developer coaching session, Oracle experts will explore the incredible potential of GPU virtualization in Kubernetes, delve into the intricacies of deploying GPU-intensive workloads, and cover the benefits of leveraging Kubeflow. https://lnkd.in/edgcSqYJ

1

🌟 **Achieving High Availability in AWS with NAT Gateways!** 🌟 In my latest video, I explore the process of **setting up multiple NAT gateways** in AWS to enhance **high availability** and **fault tolerance** within your cloud architecture. By deploying NAT gateways across multiple Availability Zones, we can ensure that our applications remain resilient and operational, even in the face of potential failures. This strategy is crucial for maintaining seamless connectivity and performance in your VPC. Join me as I walk you through this step-by-step lab guide to fortify your AWS infrastructure! 💡 #AWS #NATGateway #HighAvailability #FaultTolerance #CloudArchitecture #CloudComputing #Networking

16

Any significant AI/ML workload will need GPU power. If you're running on Kubernetes, that's no different. In this blog post, I break down: 👉 Why you'd choose to use GPUs on Kubernetes 👉 The key differences between GPUs in the cloud and on-prem 👉 How to implement GPUs in Kubernetes and A LOT more. check it out at the link below. https://buff.ly/4h70Tm3 #kubernetes #devops #nvidia

24

For those interested in how cloud scale database systems work, below is a deep dive on Amazon MemoryDB. MemoryDB delivers microsecond-level read latencies, and throughput in the hundreds of millions of queries per second, while offering multi-AZ durability with automatic failover.

22

Incredibly proud of the latest Kinesis Client Library version (3.0) which, when compared to previous KCL versions, reduces compute costs by up to 33%. KCL 3.0 introduces an enhanced load balancing algorithm that continuously monitors resource utilization of the stream processing workers and automatically redistributes the load from over-utilized workers to other underutilized workers. This ensures even CPU utilization across workers and removes the need to over-provision the stream processing compute workers which reduces cost. Additionally, KCL 3.0 is built with the AWS SDK for Java 2.x for improved performance and security features, fully removing the dependency on the AWS SDK for Java 1.x. #Kinesis #DataStreaming #KCL

94

🕵️‍♀️ Kubernetes Secrets got you puzzled? Sam Gabrail, Platform Engineer and Trainer, has your back. In his latest video and blog, Sam demystifies Kubernetes Secrets, busts common myths, and shows you how to secure them effortlessly with Akeyless Security. Learn: ✔️What Kubernetes Secrets are and why you need them. ✔️Types of Kubernetes Secrets and their uses. ✔️Common misconceptions debunked. ✔️Hands-on demo with Akeyless. Check the blog and video for clear, actionable insights and take control of your Kubernetes Secrets. Read here to get started! https://okt.to/sEPfSu #DevSecOps #DevOps #SecretsManagement

21

Five reasons large HPC users still struggle to reliably use Spot instances at scale, even with AWS EC2 instance hibernation turned "on". 1. Hibernate requires the exact same instance and exact same subnet for the restore, making it less likely to find a suitable new instance to continue running on quickly. 2. There are Spot instances available for some GPU instances (if you can survive the interruption) but hibernation is not available for GPUs. I hear from customers using Runai that they face the same limitations and reserved instances were suggested vs. trying to use Spot instances. 3. Hibernate must save the entire machine image apps+OS, hence it is more costly and inefficient compared to an approach that only saves the app and process trees (less data movement intensive). It is not uncommon for large Spot instance interruption "storms" to occur and customers have reported upwards of 30-40% reclaim rates, efficiency matters. 4. Users must configure a root volume size that is guaranteed to have enough free space and throughput to store the data on RAM. This EBS cost can add up for larger machines. File system based methods to persist the data on RAM can be much more cost effective. 5. Limited selection of EC2 instances is supported. For Linux machines, you can only use instances with 150GB of RAM or less. Many customers require 256, 512, 768, and larger instances for long periods of time and reliably using Spot instances is out of reach. It seems like most users are stuck with using on-demand and reserved instances for many use cases instead of running them on Spot and at best only using Spot for small VM workloads in a limited number of cases. In my opinion, our approach to memory checkpointing addresses all of the limitations above and some of the largest fortune 500 companies are beginning to adopt our solution. What has your experience using EC2 instance hibernation been like? #HPC #HTC #AWS #EC2 #Hibernation

15

9 Comments

Amazon RDS Multi-AZ deployment with two readable standbys supports 6 additional AWS Regions Three-node Multi-AZ DB Cluster is now available in Asia Pacific (Hyderabad), Asia Pacific (Melbourne), Europe (Spain), Europe (Zurich), Middle East (UAE), and Israel (Tel Aviv) Regions. This deployment option supports minor version upgrades and system maintenance updates with typically less than one second of downtime when using Amazon RDS Proxy or any one of the open-source AWS Advanced JDBC Driver, PgBouncer, or ProxySQL. https://lnkd.in/g9D9RGVB #Amazon #RDS #PostgreSQL #MySQL

90

1 Comment