Database Method: Staged Migration

Method Name: Staged Migration

When to use:

1. Migrate to another database vendor
2. Lower the risk of an upgrade when the database access is not controlled and upgrade is risky because of deprication

Tradeoff: Upgrade takes longer to make thus increasing cost

Variant methods: Model splitting staged migration

How to use it:

Work is done in three stage with Stage 0 as the original stage and Stage 3 as the aim for the stage. In stages 1 and 2 it is possible to go back to Stage 0.

Stage 1: Where we query and CRUD to new database and ignore results

1. A new database is created with original schema. Take notes on all places where the schema needs to be adjusted
2. Make a model creation script in source control for the adjusted schema. Be sure that the metadata (general configuration typically in lookup tabels) and pseudo-metadata (customer specific configuration typically in specific lookup tables) also is present as they will be a large source of errors
3. Enable more detailed error logging on the new database if needed as original queries and CRUDing will need to have both parameters and SQL to be able to correct it. Because of parameters logging the log itself needs to be kept secure as personal and/or secure data can be compromised
4. In the code make the changes to prepare for querying and CRUDing to the new database. This is done by adding an anti-corruption layer where querying and CRUDing is done. The querying and CRUDing should be done in parallel if at all possible to not slow down the original system. The errors and results from the new database should be ignored for now
5. Extending the anticorruption layer. First stage the querying and CRUDing towards the new database should be just passing through the anticorruption layer without any changes. If an error occurs the anticorruption layer is modified to mitigate this. This is an iterative process that is done every time and error is encountered. This step can be done before actual data goes to the new database when obvious changes needs to be made to avoid simple
6. Prepare for Stage 2 where querying and CRUDing will be from the new database and make it easy to switch between Stage 1 and 2
7. Either migrate data from the original database or make a lookup fallback to the original database if there are limited querying of old data
8. When all identified errors are handled switch to Stage 2

Stage 2: Where we validate our new database is working

1. Querying and CRUDing is monitored closely and if an identifiable error occurs switch to Stage 1, perhaps rerun querying and correct the error in the new database. This switch can be done automatically or manually depending on the error tolerance in the application
2. When you are confident that all errors has been handled in appropriate ways switch to Stage 3.

Stage 3: Everything is working

1. Remove all code that goes to the original database both for querying and CRUDing
2. All identified errors are handled only in the new database

Model splitting staged migration

This variant is that Stage 1-3 are done on parts of the original datamodel slowly strangling the original database as load is moved to the new databases