[pve-devel] [PATCH-SERIES v5] Make storage migration more flexible

[Fabian Ebner]

Previous discussion here: [0].

This series aims to allow offline migration with '--targetstorage'
and improve handling unsued/orphaned disks when migrating.
It also makes it possible to migrate volumes between storages with
a 'path' and storages without if the target storage uses the standard
naming scheme and the source format is supported (e.g. migrating raw
volumes between storages with a path and lvm storages).

It also adds an apiinfo call to pvesm that can be used to determine
APIVER and APIAGE of the remote node and does some general refactoring
regarding volume iterators.


The series is organised as follows:

#1-#3 make use of volume-related helpers
These are not needed for the rest of the series.

#4-#7 storage_migrate interface change
Dependencies: qemu-server,guest-common,container -> storage

#8-#15 do the work to allow offline --targetstorage and to avoid collisions
#14 is optional, but consolidates updating the volume IDs in the config
Dependencies: qemu-server -> guest-common,storage

#16-#19 further improvements
#19 makes migrating raw between dir and lvm possible


Changes from v4:
    * rebased remaining patches on current master
      which made three non-trivial changes necessary:
	* replaced a newly introduced foreach_drive
	* increased storage API bump
	* adapted check in cleanup_remotedisks()


[0]: https://pve.proxmox.com/pipermail/pve-devel/2020-March/042587.html


qemu-server:

Fabian Ebner (10):
  Switch to using foreach_volume instead of foreach_drive
  Use new storage_migrate interface
  Allow parsing vmstate entries
  Take note of changes to the volume IDs when migrating and update the
    config
  Allow specifying targetstorage for offline migration
  Update volume IDs in one go
  sync_disks: use allow_rename to avoid collisions on the target storage
  sync_disks: be more verbose if storage_migrate fails
  sync_disks: log output of storage_migrate
  cleanup_remotedisks: also include those migrated with storage_migrate

 PVE/API2/Qemu.pm            |  9 ++--
 PVE/QemuConfig.pm           | 15 ++++++-
 PVE/QemuMigrate.pm          | 82 +++++++++++++++++++++++++++----------
 PVE/QemuServer.pm           | 71 ++++++++++++++++++++++++++------
 PVE/QemuServer/Cloudinit.pm |  2 +-
 PVE/QemuServer/Drive.pm     | 61 ---------------------------
 PVE/VZDump/QemuServer.pm    |  3 +-
 7 files changed, 138 insertions(+), 105 deletions(-)


container:

Fabian Ebner (3):
  Use foreach_volume instead of foreach_mountpoint-variants
  Use parse_volume instead of parse_ct-variants
  Use new storage_migrate interface

 src/PVE/API2/LXC.pm        | 19 +++--------
 src/PVE/API2/LXC/Config.pm |  2 +-
 src/PVE/API2/LXC/Status.pm |  2 +-
 src/PVE/CLI/pct.pm         |  7 ++--
 src/PVE/LXC.pm             | 21 ++++++------
 src/PVE/LXC/Config.pm      | 69 +++++++++-----------------------------
 src/PVE/LXC/Create.pm      |  4 +--
 src/PVE/LXC/Migrate.pm     | 16 +++++----
 src/PVE/VZDump/LXC.pm      |  2 +-
 src/lxc-pve-prestart-hook  |  2 +-
 10 files changed, 49 insertions(+), 95 deletions(-)


storage:

Fabian Ebner (4):
  Collect optional parameters for storage_migrate into $opts
  Add apiinfo helper to pvesm
  Introduce allow_rename parameter for pvesm import and storage_migrate
  storage_migrate: add volname_for_storage helper

 PVE/API2/Storage/Content.pm  |  2 +-
 PVE/CLI/pvesm.pm             | 52 +++++++++++++++++---
 PVE/Storage.pm               | 94 ++++++++++++++++++++++++++++++------
 PVE/Storage/LVMPlugin.pm     | 17 ++++---
 PVE/Storage/Plugin.pm        | 16 ++++--
 PVE/Storage/ZFSPoolPlugin.pm | 13 +++--
 6 files changed, 155 insertions(+), 39 deletions(-)


guest-common:

Fabian Ebner (2):
  Use new storage_migrate interface
  Add update_volume_ids

 PVE/AbstractConfig.pm | 30 ++++++++++++++++++++++++++++++
 PVE/Replication.pm    | 12 ++++++++++--
 2 files changed, 40 insertions(+), 2 deletions(-)
-- 
2.20.1