Which “Lock” would you choose?
Poly Cloud¹ and Multi-Cloud² environments are already at our doorstep. More and more organizations are adopting this reality because of the myriad features and diverse capabilities it provides. AWS, Azure and GCP being the top three Cloud provider contenders provide their own object storage services. All have their unique selling proposition and more or less similar but still each comes with their own unique features.
Before we jump into the locks, let’s browse through the key similarities and differences of the storage mechanisms for each of the Cloud providers.
Once we have decided where to store the data, the next question is how do we protect, retain and manage their lifecycle till its purged. All these Cloud providers provide Records Management features which comply with SEC³, FINRA⁴ and CFTC⁵ but each of them differ from the other in some key aspects. There is no good or bad. It depends on what your needs are or which specific feature is more important for your objects and your functional and/or organizational needs.
All these Cloud platforms follow the WORM (write-once-read-many) model for storing objects. The Locks on the objects help prevent objects from being deleted or overwritten for a user-defined interval or indefinitely. Organizations usually have compliance, legal and regulatory requirements for retaining data for specific lengths of time. To help meet these requirements, the Object/Bucket Lock provides time-based retention policies or Object Holds.
Unique features of:
AWS➟ -Under Governance mode, bucket/object settings can be changed with special permission -Retention period can be applied at individual object level as well -Can change (extend/reduce) the retention period -Existing buckets cannot be object locked without AWS support; need to enforce retention on existing objects -For an object to be put on Legal hold, the bucket needs to be object locked -Legal hold can be applied at individual object level only
Azure➟ -Max 5 edits to extend time-based retention -Cannot reduce retention period -Existing blobs can be locked with immutable storage -Legal Holds can be applied without time-based retention -Legal Holds cannot be applied selectively at object level; it has to be applied at Container level
GCP➟ -Existing buckets can be locked with retention policy -Locking through retention policy is an irreversible action. Need to delete 'entire' bucket to remove bucket retention policy. Also need to wait till all objects expire retention period -Retention period can be as low as 1 sec -Cannot reduce retention period -Object hold can be applied without Retention policy -Object hold can be applied at object level
Strength of AWS lies in the fact that object retention period can be applied at individual object level and the retention period can be extended or reduced. In case of Azure and GCP, their strength is that Locks can be applied to exisitng objects without vendor support.
References: AWS, Azure Immutable Storage, GCP Bucket Lock, GCP Object Holds
Footnote: 1 – run specific parts of workloads on the best cloud provider for specific function (e.g., compute with Lambda for data lake on Azure) 2 – run same workloads on multiple vendors (e.g., because of location constraint) 3 – SEC - Securities and Exchange Commission 4 – FINRA - Financial Industry Regulatory Authority 5 – CFTC - Commodity Futures Trading Commission