Our Journey with Multi-region Cloud Architecture and Data Residency
Authors: Jatin K Malik & Yashika Ahuja
Introduction
In today's interconnected world, Data Residency, which we often refer to as DaRe, has become an imperative topic. As businesses expand globally and face a myriad of regulations, it's essential to know where our data resides and ensure its compliance.
In this article, we'll shed light on how we at Atlassian tackled these challenges through our multi-region cloud architecture.
Data Residency at Atlassian
At its essence, data residency points to the physical or geographic location of an organization's data. The location of our data can have profound implications, especially when considering government regulations like FedRAMP, GDPR, and CCPA or industry norms like HIPAA and EBA.
At Atlassian, we prioritize Data Residency through our enterprise platform, instilling trust and offering data governance capabilities to our customers.
As of January 2024, we proudly support data pinning in 12 regions:
Australia 🇦🇺
Canada 🇨🇦
European Union 🇪🇺
Germany 🇩🇪
Singapore 🇸🇬
USA 🇺🇸
Brazil 🇧🇷
India 🇮🇳
Japan 🇯🇵
South Korea 🇰🇷
Switzerland 🇨🇭
United Kingdom 🇬🇧
Guiding Principles
Our approach to multi-region cloud architecture rests on three pivotal principles:
Region Creation
Establishing a new region in the cloud goes beyond just provisioning servers. It's a complex process that demands careful coordination across several teams and systems. At Atlassian, we've meticulously segmented the region creation process into several phases, ranging from initial preparation and fixture setup to core and non-core platform configurations. Each phase is structured with specific prerequisites to ensure a seamless execution.
Initially, the logic to handle data residency migrations was contained in a monolith service that was shared among multiple teams. However, the team decided to refactor the code and extract it into a new microservice. During this refactoring process, we made several enhancements, such as transitioning from to , utilizing internal mimeograph libraries, and incorporating the sqs starter library.
This refactoring proved to be a significant change for us. We managed to reduce our codebase by 83%, leading to a decrease in our PR cycle time. Additionally, it allowed for a faster rollout of new regions, enabling us to launch six new regions ahead of schedule!
Service Hydration
Once a region is established and ready, our immediate challenge is "Service Hydration". This crucial step encompasses several tasks:
Defining the Scope: We outline the range of services that will be available in the newly created region.
Hydration: This process involves populating the new region with the necessary services, ensuring they are set up correctly and ready for use.
Rigorous Testing: Before any service goes live, it undergoes extensive testing to guarantee optimal performance, reliability and compliance.
Tenant Migration
After the groundwork of configuring and hydrating services, our attention shifts to tenant migration. This phase is crucial, as it translates all our preparatory work into tangible benefits for the end-users.
Architecture Components
Central to our multi-region cloud architecture are 4 core components:
Tenant Placement Service: This is responsible for placing, estimating, and migrating tenant resources for Atlassian products.
Cloud Provisioner: This is essentially a control plan for entities like sites, enterprises, etc. It interacts with the control planes of various services that comprise products via defined workflows.
Catalog Service: This serves as the touchstone for all tenant details.
Tenant Context Service: Designed to offer low-latency access to consistent, read-only views of tenant state data.
Enabling Self Serve for our Customers
Self Serve migrations, which were initially handled by the Atlassian Support team, can now be directly performed from the Data Residency Admin Hub Panel. According to recent statistics, 92% of our customers have already chosen this option and successfully migrated their data to their preferred locations. This feature is highly recommended for the following reasons:
Customers have the freedom to select the location where they want to move their data.
Customers are provided with visibility on approximate downtime, allowing them to choose a migration schedule that suits them.
Currently, we offer the following move windows to our customers:
Maintenance Window: The migration will take place during the customer's maintenance window and can occur on any day within the next 7 days once it is scheduled.
Anytime (ASAP)/Immediately: The migration will be carried out within the next 48 hours based on estimated downtime.
Specified: Once scheduled, the migration will start on a date and time chosen by you and will take approximately 24 hours to complete. For this option, please request your move at least three days in advance.
You can learn more about the different Move Windows here
To ensure a seamless process, customers who opt for Self Serve also receive regular updates via email regarding their migration status.
For more details, kindly go through our official documentation on how to Manage Data Residency
In Retrospect and Looking Ahead
Data residency is one of the key unblockers for our enterprise customers. At Atlassian, we understand the importance of data residency and have taken significant steps to address it. With our multi-region cloud architecture, we now cover over 30% of the population of Earth with our 12 Data Residency regions. This extensive coverage allows our customers to have their data stored in the location of their choice, ensuring compliance with regulations and industry norms. By offering data pinning in regions such as Australia, Canada, European Union, Germany, Singapore, USA, Brazil, India, Japan, South Korea, Switzerland, and the United Kingdom, we provide our customers with the flexibility and peace of mind they need.
With our commitment to data residency and continuous expansion, we strive to meet the evolving needs of our customers and empower them to grow their businesses with confidence!