[pve-devel] [RFC/PATCH] make storage migration more flexible

[Fabian Ebner]

This series aims to allow offline migration with '--targetstorage'
and improve handling unsued/orphaned disks.

An abstract method for recursively updating volume IDs throughout
a config file is provided. Like this it should also work when there
are snapshots.

The API for 'pvesm import' is changed twice.
First, to automatically rename volumes on import if the requested
name already exists.
Second, to allow specifying only the storage ID and a VM ID instead
of a full volume ID.

The first change (patch 3 and requires 1,5,12 and 13) is enough to
avoid collision with orphaned volumes on the target and (more
importantly) when there are two disks on different storages with
the same name that would collide on the target storage.
The second change (patches 6,7,8 and requires 1,5,12 and 13)
would allow to migrate images between storages which don't have the
same naming schema, e.g. migrating a raw image between an lvm and
dir storage.
The interface after the second change is rather ugly. An alternative
approach would be to create a function which translates the volid from
the original storage to a valid volid for the target storage. Is it even
enough to translate between storages with path and without path, i.e.
123/vm-123-disk-0.raw <-> vm-123-disk-0?

To get the name a volume was actually imported with,
'storage_migrate' logs and matches the output of 'pvesm import'.

The code in qemu-server is adapted to keep track of the new volume
IDs as well and calls 'update_volume_ids' after migration.


The patches 2,4,10 and 11 are not directly related to the series.


No patches for LXC are included at the moment.
Aaron, you were working on migration for containers, so
would you mind taking a look at the relevant patches and telling
me what changes would be needed or if the same approach
as I took for QEMU would work for LXC?


guest-common:

Fabian Ebner (1):
  Implement update_volume_ids and add required helpers: foreach_volume
    and print_volume


storage:

Fabian Ebner (8):
  Remove unused string
  volume_import: Use a new name when the the name to import with already
    exists
  storage_migrate: also log with an insecure connection if there is a
    log function
  storage_migrate: return volume ID of migrated volume
  pvesm import: allow specifying storage+vmid instead of full volumeid
  volume_import_formats: if no volume name is specified, return all
    formats the storage supports
  storage_migrate: use only storeid when no volume name is specified
  storage_migrate: Make error message more verbose

 PVE/CLI/pvesm.pm             | 41 ++++++++++++++++----
 PVE/Storage.pm               | 73 +++++++++++++++++++++++++++---------
 PVE/Storage/LVMPlugin.pm     | 34 +++++++++++------
 PVE/Storage/Plugin.pm        | 59 +++++++++++++++++++----------
 PVE/Storage/ZFSPoolPlugin.pm | 34 ++++++++++++-----
 5 files changed, 173 insertions(+), 68 deletions(-)


qemu-server:

Fabian Ebner (7):
  rename 'volid' to 'drivestr' where it's not only a volume ID
  Extract volume ID before calling 'parse_volume_id'
  Implement abstract foreach_volume and print_volume
  Take note of changes to volume ids when migrating and update config
  Allow specifying targetstorage for offline migration
  sync_disks: log output of storage_migrate
  Update volume IDs in one go

 PVE/API2/Qemu.pm   |  3 ---
 PVE/QemuConfig.pm  | 12 ++++++++++++
 PVE/QemuMigrate.pm | 42 +++++++++++++++++++++++++++++-------------
 PVE/QemuServer.pm  |  4 ++--
 4 files changed, 43 insertions(+), 18 deletions(-)

-- 
2.20.1