Data migration checklist
Points to check before any data migration.
- Process constraints:
- optimal and maximum allowed time to perform all operations (e.g. weekend, 48 hours)
- rollback case: partial, full?
- resume migration from the last failed operation or restart all?
- monitoring: logs, UI, console output, technical data...
- Source data:
- formats and descriptions
- data volume estimation (bytes, rows, tables, tuples...)
- who is responsible to explain data logic in terms of business logic?
- data access:
- directly to DBMS - how to connect?
- exported files - consider the time to export fresh data to file, file access permissions
- is data cleanup required?
- Destination data:
- database access (how to, and what is minimal permissions level required e.g. DB owner)
- database model (schema) with description
- who is responsible to explain data logic in terms of business logic?
- any constraints that may affect data import: storage limits, triggers, replication...
- ETL/ELT:
- allowed/recommended software (e.g. SQL Server Integration Services, custom .NET or Python application...)
- running environment, required access and authorizations, data connections (CLI components)
- transformations: rules to validate expected data integrity and consistency
- Dev environment:
- almost same software configurations as for production environment: software versions, OS, CPU architecture
- infrastructure may be different
- Test environments:
- same configuration/infrastructure as for production environment
- performance testing is "the must"