Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Data Migration from Amsterdam to Casablanca

  • AAI
  • SO
  • SDC
  • SDNC

Data MigrationSteps for Data Migration



AAI

Anchor
aai
aai


Amsterdam Prod environment

Step 1: Extract Data from A&AI HBase DB

  • Connect to A&AI Environment
  • Go inside aai-resources Pod
  • Change user to aaiadmin & Go to /opt/app/aai-resources directory
  • Execute the below command :
Code Block
languagetext
aaiadmin@aai-resources-7bc44cb67f-ls4nt:/opt/app/aai-resources$ sh bin/dataSnapshot.sh

Step 2: Copy the Snapshot
From Amsterdam aai-resources Pod to Casablanca aai-graphadmin Pod on location (/opt/app/aai-graphadmin/logs/data/dataSnapshots/)

Casablanca Prod environment

Step 3: Import snapshot to Casablanca Cassandra DB

  • Connect to A&AI Casablanca Environment
  • Go inside aai-graphadmin pod 
  • Change user to aaiadmin & Go to /opt/app/aai-graphadmin/bin directory
  • Import snapshot data to Cassandra DB using below command:
Code Block
languagetext
aaiadmin@aai-graphadmin:/opt/app/aai-graphadmin/bin$ ./dataRestoreFromSnapshot.sh <Datasnapshot-FileName>

<Datasnapshot-FileName>: File copied in Step 2 from Amsterdam to Casablanca aai-graphadmin Pod
(Only name required, full path is not required)


Step 4: Migrate Database Schema

Migrate Schema using below command on same location (/opt/app/aai-graphadmin/bin directory):

Code Block
languagetext
aaiadmin@aai-graphadmin:/opt/app/aai-graphadmin/bin$ ./run_Migrations.sh -e UpdateAaiUriIndexMigration
 --commit --skipPreMigrationSnapShot --runDisabled RebuildAllEdges


Step 5: Migration Verification

Verify there are no errors using below command on same location (/opt/app/aai-graphadmin/bin directory):

Code Block
languagetext
aaiadmin@aai-graphadmin:/opt/app/aai-graphadmin/bin$ ./migration_verification.sh <last_modified> <logs_path>


<logs_path>: Directory containing all of the logs. Default path is /opt/app/aai-graphadmin/logs/migration.
<last_modified>: An integer indicating log files created in last <> minutes should be parsed.
 
Example: migration_verification.sh 60 /opt/app/aai-graphadmin/logs/migration


Step 6: Restart aai-resources pod





SO

Anchor
so
so


SO database will be migrated from Amsterdam to Casablanca during installation/startup.

Step 1: New Prod environment (Casablanca):

To allow SO mariadb container to run the migrations following overrides must be provided as override file or command-line:

Code Block
languagetext
so:
  so-mariadb:
    config:
      gerritBranch: nso/casablanca
      gerritProject:<Gerrit path to so docker-config>
      migration:
        enabled: true
        remote_host: <Set to Amsterdam Cluster IP>
        remote_port: <Set to SO MariaDB NodePort on Amsterdam Prod>


< ... >: User sets the actual values in the override file

Step 2. Install SO (Casablanca) using OOM

Step 3. Verify Migration steps using logs

select * from requestdb.schema_version where success =0
select * from catalogdb.schema_version where success =0

The above two queries should return no record




SDC

Anchor
sdc
sdc

Step 1:

Execute nodetool flush utility on the Amsterdam’s Cassandra instance to flush the memtables to disk
  • Connect to SDC Amsterdam Environment
  • Go inside sdc-cs pod
  • Execute command: nodetool flush

Step 2:

Copy the Cassandra data from Amsterdam to Casablanca:

  1. Source: /dockerdata-nfs/<namespace>/sdc/sdc-cs/CS/data
  2. Destination: /dockerdata-nfs/<Release name>-sdc/sdc/sdc-cs/CS/ (This directory has to be created breforehand).

    Note: replace /dockerdata-nfs with real nfs mount path.

Step 3:

Following overrides must be provided either via overrides file or via command line:

Code Block
languagetext
Using override file:
sdc:
  sdc-cs:
    config:
       migration: true

Using Command-line: 
--set sdc.sdc-cs.config.migration=true


Step 4: Install SDC (Casablanca) using OOM

Step 5: Verify the migration job (with name containing <migrate-cassandra>) is executed successfully (check for this pod's log using the kubectl logs command) after all the SDC pods are up and running.

Step 6: Redistribute the migrated services

Step 7: After completion of migration, set the "migration: false" in the override file for future upgrades




SDNC

Anchor
sdnc
sdnc


Step 1 : MD-SAL Data backup from Amsterdam Prod env

  1. Export Amsterdam data to a file using OpenDaylight API. This will create backup file with given name (file-path in request body) on Amsterdam ODL pod – can be identified by name sdnc-<UUID>
    Request details to export data:

    Code Block
    languagetext
     POST /restconf/operations/cluster-admin:backup-datastore HTTP/1.1
     Host: <Amsterdam_sdnc_host>:30202
     Content-Type: application/json
     Authorization: Basic YWRtaW46S3A4Yko0U1hzek0wV1hsaGFrM2VIbGNzZTJnQXc4NHZhb0dHbUp2VXkyVQ==
     {
        "input": {
            "file-path": "/tmp/datastore_backup_b"
        }
     }


  2. Take the backup file out of the Amsterdam sdnc pod
Code Block
languagetext
kubectl cp <namespace>/<Amsterdam_sdnc_pod>:/tmp/datastore_backup_b  /path/to/host/vm'


3. Transfer the backup file to Casablanca ODL instance before startup.

Target directory: /dockerdata-nfs/<namespace>/sdnc/migration.

This directory is mapped to /opt/opendaylight/clustered-datastore-restore on Casablanca sdnc pod.

Note1: /dockerdata-nfs/<namespace>/sdnc/migration must be created manually.

Note2: Replace /dockerdata-nfs with actual nfs mount path.

Step 2 : Overrides for MySql Data Migration

  1. Following overrides must be provided either via overrides file or via command line:
Code Block
languagetext
Overrides file:
sdnc:
  config:
     migration:
       enabled: true
       amsterdamHost: <Amsterdam Cluster IP>
       amsterdamPort: <Amsterdam SDNC MySQL nodeport>

Command-line:
sdnc.config.migration.enabled=true
sdnc.config.migration.amsterdamHost=<Amsterdam Cluster IP>
sdnc.config.migration.amsterdamPort=<Amsterdam SDNC MySQL nodeport>

< ... >: User sets the actual values in the override file


Step 3: Install SDNC (Casablanca) using OOM

Step 4: After completion of migration, set the "migration: enabled: false" in the override file for future upgrades