Data migration checklist

| category: Process | author: st
Tags:

Points to check before any data migration.

  1. Process constraints:
    • optimal and maximum allowed time to perform all operations (e.g. weekend, 48 hours)
    • rollback case: partial, full?
    • resume migration from the last failed operation or restart all?
    • monitoring: logs, UI, console output, technical data...
  2. Source data:
    • formats and descriptions
    • data volume estimation (bytes, rows, tables, tuples...)
    • who is responsible to explain data logic in terms of business logic?
    • data access:
      • directly to DBMS - how to connect?
      • exported files - consider the time to export fresh data to file, file access permissions
      • is data cleanup required?
  3. Destination data:
    • database access (how to, and what is minimal permissions level required e.g. DB owner)
    • database model (schema) with description
    • who is responsible to explain data logic in terms of business logic?
    • any constraints that may affect data import: storage limits, triggers, replication...
  4. ETL/ELT:
    • allowed/recommended software (e.g. SQL Server Integration Services, custom .NET or Python application...)
    • running environment, required access and authorizations, data connections (CLI components)
    • transformations: rules to validate expected data integrity and consistency
  5. Dev environment:
    • almost same software configurations as for production environment: software versions, OS, CPU architecture
    • infrastructure may be different
  6. Test environments:
    • same configuration/infrastructure as for production environment
    • performance testing is "the must"