[pve-devel] [PATCH-SERIES v3] Make storage migration more flexible

[Fabian Ebner]

This is the second half for the previous series [0].

This series aims to allow offline migration with '--targetstorage'
and improve handling unsued/orphaned disks when migrating.
It also makes it possible to migrate volumes between storages with
a 'path' and storages without if the target storage uses the standard
naming scheme and the source format is supported (e.g. migrating raw
volumes between storages with a path and lvm storages).

It also adds an apiinfo call to pvesm that can be used to determine
APIVER and APIAGE of the remote node and does some general refactoring
regarding volume iterators.

The series is organised as follows:
#1-#7 introduce and implement volume related helpers
Mutual dependencies for both qemu-server and container with guest-common,
to be precise #1-#3 <-> #5 and #1-#3 <-> #7.
#8 adds the apiinfo helper
#9-#12 changes storage_migrate interface
Another round of mutual dependencies, this time for storage with each
of guest-common, qemu-server, container.
#13-#18 implement the goals for this series, and the last inter-package
dependency is here, because #15 (qemu-server) depends on #13 (storage)
#19-#22 improve logging and refactor some code with foreach_volume

Changes from v2:
    * add apiinfo call and check if remote side supports allow_rename
    * add valid_volume_keys and implement (and make use of) foreach_volume iterator
    * make volname_for_storage a local function and avoid
      requiring a new print_volname for plugins
    * parse vm_state in parse_volume instead of parse_drive

[0]: https://pve.proxmox.com/pipermail/pve-devel/2020-February/041910.html


guest-common:

Fabian Ebner (5):
  Add interfaces for volume-related helpers
  Add foreach_unused_volume
  foreach_volume: generalize and implement function
  Use new storage_migrate interface
  Add update_volume_ids

 PVE/AbstractConfig.pm | 113 ++++++++++++++++++++++++++++++++++++++----
 PVE/Replication.pm    |  12 ++++-
 2 files changed, 112 insertions(+), 13 deletions(-)


container:

Fabian Ebner (3):
  Implement volume-related helpers and use new foreach_volume
  Use new storage_migrate interface
  Use foreach_volume instead of foreach_mountpoint_full to avoid
    duplication

 src/PVE/API2/LXC.pm        |  4 +--
 src/PVE/API2/LXC/Config.pm |  2 +-
 src/PVE/CLI/pct.pm         |  2 +-
 src/PVE/LXC/Config.pm      | 53 +++++++++++++++++++++-----------------
 src/PVE/LXC/Migrate.pm     | 10 ++++---
 5 files changed, 40 insertions(+), 31 deletions(-)


qemu-server

Fabian Ebner (10):
  Implement volume-related helpers and use new foreach_volume
  Allow parsing vmstate entries
  Use new storage_migrate interface
  Take note of changes to the volume IDs when migrating and update the
    config
  Allow specifying targetstorage for offline migration
  Update volume IDs in one go
  sync_disks: use allow_rename to avoid collisions on the target storage
  sync_disks: be more verbose if storage_migrate fails
  sync_disks: log output of storage_migrate
  update_disk_config: use config volume iterators instead of loops

 PVE/API2/Qemu.pm        |  3 ---
 PVE/QemuConfig.pm       | 51 +++++++++++++++++++++++++++++++++++------
 PVE/QemuMigrate.pm      | 44 +++++++++++++++++++++++++++--------
 PVE/QemuServer.pm       | 50 +++++++++++++++++++++-------------------
 PVE/QemuServer/Drive.pm |  1 +
 5 files changed, 105 insertions(+), 44 deletions(-)


storage:

Fabian Ebner (4):
  Add apiinfo helper to pvesm
  Collect optional parameters for storage_migrate into $opts
  Introduce allow_rename parameter for pvesm import and storage_migrate
  storage_migrate: add volname_for_storage helper

 PVE/API2/Storage/Content.pm  |  2 +-
 PVE/CLI/pvesm.pm             | 52 +++++++++++++++++---
 PVE/Storage.pm               | 94 ++++++++++++++++++++++++++++++------
 PVE/Storage/LVMPlugin.pm     | 17 ++++---
 PVE/Storage/Plugin.pm        | 16 ++++--
 PVE/Storage/ZFSPoolPlugin.pm | 13 +++--
 6 files changed, 155 insertions(+), 39 deletions(-)

-- 
2.20.1