OpenStack: VM Migration on OpenStack

Hello All! Today’s post is on some work I did for exporting VMs between OpenStacks. Here are two scripts for doing the export and import; one for migrating between hypervisors (same OpenStack) and one for migrating between OpenStack implementations. They stop short of launching the VM but that could be done with a bit more work to identify flavors, map networks, and so on. Enjoy!

For the impatient: Here are the scripts:

Migrate VM between Hypervisors

Moving a VM between two hypervisors is very simple; you simply stop the VM, issue the nova migrate command, and then authorize the movement. In fact, if you have proper shared storage between your hypervisors you can use live migration via the nova live-migration command. I don’t have proper high-speed shared storage, so I have to do things the slower way. Let’s see the steps:

  1. Shutoff the machine using nova stop [vm].
  2. Migrate the VM using nova migrate [vm].
  3. Wait for the VM state to change to VERIFY_RESIZE; you can use nova show [vm] to get this state.
  4. Confirm the “resize” (really, the migration!) using the nova resize-confirm [vm].

The script above automates this process.

Migrate VM between OpenStacks

Moving a VM between OpenStacks requires a bit more work. The attached script automates the entire process except for launching (booting) the migrated VM.

  1. You need to have your “keystone” script files that provide credentials to both OpenStacks. You supply these files to the script.
  2. Get the VM ID to move by issuing nova show [VM] and noting the ID.
  3. Invoke the script by passing the following elements:
    • src_keystone_rc – file with OS_USERNAME, OS_AUTH_URL, and so on to the source OpenStack
    • dst_keystone_rc – file with OS_USERNAME, OS_AUTH_URL, and so on to the destination OpenStack
    • VM_ID – UUID for the VM to migrate (get this using nova show on the source OpenStack)
    • img_type – Glance image type for the created snapshot (“file”, “rbd”, or “swift”)
    • step – Nifty feature that allows you to indicate the starting “step” number; this permits you to rerun an interrupted script without waiting for all the intermediate steps to run over. Just pass in a 0 (zero) to run everything from the beginning.
  4. The script begins by verifying the VM ID on the source OpenStack:
    # get info
    l_rc=0
    source "$l_in_src_keystone_rc"
    echo "$$: nova show \"$l_in_vm_id\" 2>/dev/null > $l_tmp" >> $l_log
    nova show "$l_in_vm_id" 2>/dev/null > $l_tmp
    l_rc=$?
    if [ $l_rc -ne 0 ]; then
      do_exit 3 "Invalid VM ID '$l_in_vm_id'"
    fi
    
    # get initial state of VM
    l_vm_id=$(cat $l_tmp | grep -e "^| id" | cut -d'|' -f 3 | sed -e 's#^ \+##; s#[ \t]*$##')
    l_vm_name=$(cat $l_tmp | grep -e "^| name" | cut -d'|' -f 3 | sed -e 's#^ \+##; s#[ \t]*$##')
    l_vm_tenant_id=$(cat $l_tmp | grep -e "^| tenant_id" | cut -d'|' -f 3 | sed -e 's#^ \+##; s#[ \t]*$##')
    l_vm_state=$(cat $l_tmp | grep -e "^| OS-EXT-STS:vm_state" | cut -d'|' -f 3 | sed -e 's#^ \+##; s#[ \t]*$##')
    l_vm_status=`cat "$l_tmp" | grep status | awk '{print $4}'`
    l_vm_tenant=$(keystone tenant-get $l_vm_tenant_id | grep -e '  name  ' | cut -d'|' -f 3 | sed -e 's#^ \+##; s#[ \t]*$##')
    echo "$$: l_vm_id='$l_vm_id'" >> $l_log
    echo "$$: l_vm_name='$l_vm_name'" >> $l_log
    echo "$$: l_vm_tenant_id='$l_vm_tenant_id'" >> $l_log
    echo "$$: l_vm_state='$l_vm_state'" >> $l_log
    echo "$$: l_vm_status='$l_vm_status'" >> $l_log
    echo "$$: l_vm_tenant='$l_vm_tenant'" >> $l_log
    
  5. Next, the script verifies the VM is stopped, and checks to see if a snapshot already exists. If you pass in a step of 1 (one), then if a snapshot already exists the script uses it. Otherwise it creates a new snapshot and gets the snapshot image information (type, container format):
    # delete any existing snapshot
    l_snap_name="$l_vm_name-snap"
    l_snap_id=""
    l_continue=1
    while [ $l_continue -eq 1 ]; do
      # check for an existing snapshot ID
      echo "$$: glance --os-tenant-name=\"$l_vm_tenant\" image-list | grep -e \"$l_snap_name\" | head -n 1 | cut -d'|' -f 2 | sed -e 's#^ \+##; s#[ \t]*$##'" >> $l_log
      l_snap_id=$(glance --os-tenant-name="$l_vm_tenant" image-list | grep -e "$l_snap_name" | head -n 1 | cut -d'|' -f 2 | sed -e 's#^ \+##; s#[ \t]*$##')
      if [[ "$l_snap_id" != "" ]]; then
        if [ "$l_in_step" -ge 1 ]; then
          echo "$$: $(date +"%Y%d%m-%H%M"): Step 0: Reuse old snapshot '$l_snap_id'" | tee -a $l_log
          l_continue=0
        else
          echo -n "$$: $(date +"%Y%d%m-%H%M"): Step 0: Remove old snapshot '$l_snap_id'..."
          echo "$$: glance --os-tenant-name=\"$l_vm_tenant\" image-delete $l_snap_id 2>&1" >> $l_log
          l_msg=$(glance --os-tenant-name="$l_vm_tenant" image-delete $l_snap_id 2>&1)
          l_rc=$?
          echo "$$: l_msg='$l_msg'" >> $l_log
          if [ $l_rc -eq 0 ]; then echo "OK"; else do_exit 5 "Snapshot cleanup: $l_msg"; fi
          l_snap_id=""
        fi
      else
        l_continue=0
      fi
    done
    
    # create snapshot unless already exists
    l_snap_container_format=""
    l_snap_disk_format=""
    if [ "$l_snap_id" = "" ]; then
      echo -n "$$: $(date +"%Y%d%m-%H%M"): Step 0: Create new snapshot..."
      echo "$$: nova --os-tenant-name=\"$l_vm_tenant\" image-create --poll $l_vm_id \"$l_snap_name\" 2>&1" >> $l_log
      l_msg=$(nova --os-tenant-name="$l_vm_tenant" image-create --poll $l_vm_id "$l_snap_name" 2>&1)
      l_rc=$?
      echo "$$: l_msg='$l_msg'" >> $l_log
      if [ $l_rc -ne 0 ]; then do_exit 5 "Snapshot creation: $l_msg"; fi
    
      # get the snapshot ID
      echo "$$: glance --os-tenant-name=\"$l_vm_tenant\" image-list | grep -e \"$l_snap_name\" | head -n 1 | cut -d'|' -f 2 | sed -e 's#^ \+##; s#[ \t]*$##'" >> $l_log
      l_snap_id=$(glance --os-tenant-name="$l_vm_tenant" image-list | grep -e "$l_snap_name" | head -n 1 | cut -d'|' -f 2 | sed -e 's#^ \+##; s#[ \t]*$##')
      l_rc=$?
      echo "$$: l_snap_id='$l_snap_id'" >> $l_log
      if [ $l_rc -eq 0 ]; then echo "OK ($l_snap_id)"; else do_exit 5 "Snapshot identification"; fi
    fi
    
    # get snapshot information
    echo -n "$$: $(date +"%Y%d%m-%H%M"): Step 1: Get snapshot info..."
    echo "$$: glance --os-tenant-name=\"$l_vm_tenant\" image-show $l_snap_id | grep -e 'container_format' | cut -d'|' -f 3 | sed -e 's#^ \+##; s#[ \t]*$##'" >> $l_log
    l_snap_container_format=$(glance --os-tenant-name="$l_vm_tenant" image-show $l_snap_id | grep -e 'container_format' | cut -d'|' -f 3 | sed -e 's#^ \+##; s#[ \t]*$##')
    echo "$$: glance --os-tenant-name=\"$l_vm_tenant\" image-show $l_snap_id | grep -e 'disk_format' | cut -d'|' -f 3 | sed -e 's#^ \+##; s#[ \t]*$##'" >> $l_log
    l_snap_disk_format=$(glance --os-tenant-name="$l_vm_tenant" image-show $l_snap_id | grep -e 'disk_format' | cut -d'|' -f 3 | sed -e 's#^ \+##; s#[ \t]*$##')
    echo "container_format=$l_snap_container_format, disk_format=$l_snap_disk_format" | tee -a $l_log
    
  6. The script next pulls down the created snapshot image; make sure you have sufficient space on your local machine!
    # pull down if not exists
    if [ ! -f "$l_local_fname" ]; then
      echo -n "$$: $(date +"%Y%d%m-%H%M"): Step 1: Pull snapshot '$l_snap_id' to '$l_local_fname'..."
      echo "$$: glance --os-tenant-name=\"$l_vm_tenant\" image-download --file \"$l_local_fname\" $l_snap_id 2>&1" >> $l_log
      l_msg=$(glance --os-tenant-name="$l_vm_tenant" image-download --file "$l_local_fname" $l_snap_id 2>&1)
      l_rc=$?
      echo "$$: l_msg='$l_msg'" >> $l_log
      if [ $l_rc -eq 0 ]; then echo "OK"; else do_exit 5 "Snapshot pull: $l_msg"; fi
    fi
    
  7. The script sources in the destination keystone RC file, and uploads the snapshot:
    # check to see if the image is already uploaded to destination
    l_dst_image_name="$l_local_fname"
    l_dst_image_id=""
    l_continue=1
    while [ $l_continue -eq 1 ]; do
      # check for an existing snapshot ID
      echo "$$: glance --os-tenant-name=\"$l_vm_tenant\" image-list | grep -e \"$l_dst_image_name\" | head -n 1 | cut -d'|' -f 2 | sed -e 's#^ \+##; s#[ \t]*$##'" >> $l_log
      l_dst_image_id=$(glance --os-tenant-name="$l_vm_tenant" image-list | grep -e "$l_dst_image_name" | head -n 1 | cut -d'|' -f 2 | sed -e 's#^ \+##; s#[ \t]*$##')
      if [[ "$l_dst_image_id" != "" ]]; then
        if [ "$l_in_step" -ge 3 ]; then
          echo "$$: $(date +"%Y%d%m-%H%M"): Step 2: Reuse old destination image '$l_dst_image_id'" | tee -a $l_log
          l_continue=0
        else
          echo -n "$$: $(date +"%Y%d%m-%H%M"): Step 2: Remove old destination image '$l_dst_image_id'..."
          echo "$$: glance --os-tenant-name=\"$l_vm_tenant\" image-delete $l_dst_image_id 2>&1" >> $l_log
          l_msg=$(glance --os-tenant-name="$l_vm_tenant" image-delete $l_dst_image_id 2>&1)
          l_rc=$?
          if [ $l_rc -eq 0 ]; then echo "OK"; else do_exit 5 "Dst image cleanup: $l_msg"; fi
          echo "$$: l_msg='$l_msg'" >> $l_log
          l_dst_image_id=""
        fi
      else
        echo "$$: l_dst_image_id='$l_dst_image_id'" >> $l_log
        l_continue=0
      fi
    done
    
    # upload the image to destination if necessary
    if [ "$l_dst_image_id" = "" ]; then
      echo -n "$$: $(date +"%Y%d%m-%H%M"): Step 2: Upload snapshot to destination..."
      echo "$$: glance --os-tenant-name=\"$l_vm_tenant\" image-create --store=$l_in_img_type --container-format=$l_snap_container_format --disk-format=$l_snap_disk_format --is-public=false --name=\"$l_dst_image_name\" < \"$l_local_fname\" 2>&1" >> $l_log
      l_msg=$(glance --os-tenant-name="$l_vm_tenant" image-create --store=$l_in_img_type --container-format=$l_snap_container_format --disk-format=$l_snap_disk_format --is-public=false --name="$l_dst_image_name" < "$l_local_fname" 2>&1)
      l_rc=$?
      echo "$$: l_msg='$l_msg'" >> $l_log
      if [ $l_rc -ne 0 ]; then do_exit 5 "Snapshot upload: $l_msg"; fi
    
      # get the uploaded image ID
      echo "$$: glance --os-tenant-name=\"$l_vm_tenant\" image-list | grep -e \"$l_local_fname\" | head -n 1 | cut -d'|' -f 2 | sed -e 's#^ \+##; s#[ \t]*$##'" >> $l_log
      l_dst_image_id=$(glance --os-tenant-name="$l_vm_tenant" image-list | grep -e "$l_local_fname" | head -n 1 | cut -d'|' -f 2 | sed -e 's#^ \+##; s#[ \t]*$##')
      l_rc=$?
      if [ $l_rc -eq 0 ]; then echo "OK ($l_dst_image_id)"; else do_exit 5 "Snapshot upload identification"; fi
      echo "$$: l_dst_image_id='$l_dst_image_id'" >> $l_log
    fi
    

That is pretty much it. The nice thing about the script is that you can use it to perform mass migrations. Just be sure to delete any intermediate files so you don’t fill up your hard disk.

Enjoy!

Team-oriented systems mentor with deep knowledge of numerous software methodologies, technologies, languages, and operating systems. Excited about turning emerging technology into working production-ready systems. Focused on moving software teams to a higher level of world-class application development. Specialties:Software analysis and development...Product management through the entire lifecycle...Discrete product integration specialist!

Tagged with: ,
2 comments on “OpenStack: VM Migration on OpenStack
  1. Abhishek Talwar says:

    Hi,

    I am trying to migrate VM’s between OpenStacks using your scipt. But I am not clear what kind of keystone scipt files we will need to use your script. If you can kindly me some sample keystone script so that I can procede further.

    Thanks

    • Andrew Bruce says:

      Here you go:

      
      #!/bin/sh
      export OS_USERNAME=[username]
      export OS_PASSWORD=[passwword]
      export OS_TENANT_NAME=[tenant]
      export OS_AUTH_URL=http://[URL]:35357/v2.0
      export PS1='[\u@\h \W(lvosksclu100-rc-admin)]$ '
      

Leave a Reply

Your email address will not be published. Required fields are marked *

*

Human Verification: In order to verify that you are a human and not a spam bot, please enter the answer into the following box below based on the instructions contained in the graphic.