[pve-devel] applied-partially: [PATCH-SERIES v4] Make storage migration more flexible

[Fabian Grünbichler]

in addition to 1 and 2 I applied:

3-7, 9 + versioned depends / breaks accordingly between 
qemu-server/pve-container and pve-guest-common

the rest reads a rebase anyway - and possibly a slightly bigger one in 
case my vm_start series goes in faster ;)

On March 26, 2020 9:09 am, Fabian Ebner wrote:
> Previous discussion here: [0].
> 
> This series aims to allow offline migration with '--targetstorage'
> and improve handling unsued/orphaned disks when migrating.
> It also makes it possible to migrate volumes between storages with
> a 'path' and storages without if the target storage uses the standard
> naming scheme and the source format is supported (e.g. migrating raw
> volumes between storages with a path and lvm storages).
> 
> It also adds an apiinfo call to pvesm that can be used to determine
> APIVER and APIAGE of the remote node and does some general refactoring
> regarding volume iterators.
> 
> NOTE! that the bump here conflicts with the bump here[1] and either
> one would need to be adapted if/when applying both.
> 
> 
> The series is organised as follows:
> 
> #1 and #2 check for content types
> These are independent from the rest.
> 
> #3-#11 introduce and make use of volume-related helpers
> #7,#8,#10,#11 are not needed for the rest of the series.
> Dependencies: qemu-server,container <-> guest-common
> 
> #12-#15 storage_migrate interface change
> Dependencies: qemu-server,guest-common,container -> storage
> 
> #16-#23 do the work to allow offline --targetstorage and to avoid collisions
> #22 is optional, but consolidates updating the volume IDs in the config
> Dependencies: qemu-server -> guest-common
>               qemu-server <-> storage
> 
> #24-#27 further improvements
> #27 makes migrating raw between dir and lvm possible
> 
> 
> Changes from v3:
>     * new patches to check for content types
>     * new patches to replace foreach_drive/mountpoint with foreach_volume
>     * new patch to replace parse_ct_rootfs/mountpoint with parse_volume
>     * new patch for cleanup_remotedisks
>     * fixed/changed other smaller things pointed out by Fabian G.
> 
> [0]: https://pve.proxmox.com/pipermail/pve-devel/2020-March/042007.html
> [1]: https://pve.proxmox.com/pipermail/pve-devel/2020-March/042539.html
> 
> 
> guest-common:
> 
> Fabian Ebner (5):
>   Add interfaces for volume-related helpers
>   Add foreach_unused_volume
>   foreach_volume(_full): generalize and implement function
>   Use new storage_migrate interface
>   Add update_volume_ids
> 
>  PVE/AbstractConfig.pm | 122 ++++++++++++++++++++++++++++++++++++++----
>  PVE/Replication.pm    |  12 ++++-
>  2 files changed, 121 insertions(+), 13 deletions(-)
> 
> 
> container:
> 
> Fabian Ebner (4):
>   Implement volume-related helpers and use new foreach_volume
>   Use foreach_volume instead of foreach_mountpoint-variants
>   Use parse_volume instead of parse_ct-variants
>   Use new storage_migrate interface
> 
>  src/PVE/API2/LXC.pm        | 25 ++++------
>  src/PVE/API2/LXC/Config.pm |  2 +-
>  src/PVE/API2/LXC/Status.pm |  2 +-
>  src/PVE/CLI/pct.pm         |  9 ++--
>  src/PVE/LXC.pm             | 21 ++++-----
>  src/PVE/LXC/Config.pm      | 95 +++++++++++++++-----------------------
>  src/PVE/LXC/Create.pm      |  4 +-
>  src/PVE/LXC/Migrate.pm     | 16 ++++---
>  src/PVE/VZDump/LXC.pm      |  2 +-
>  src/lxc-pve-prestart-hook  |  2 +-
>  10 files changed, 75 insertions(+), 103 deletions(-)
> 
> 
> qemu-server:
> 
> Fabian Ebner (13):
>   migration with targetstorage: check if target storage supports images
>   Implement volume-related helpers and use new foreach_volume
>   update_disk_config: use config volume iterators instead of loops
>   Switch to using foreach_volume instead of foreach_drive
>   Use new storage_migrate interface
>   Allow parsing vmstate entries
>   Take note of changes to the volume IDs when migrating and update the
>     config
>   Allow specifying targetstorage for offline migration
>   Update volume IDs in one go
>   sync_disks: use allow_rename to avoid collisions on the target storage
>   sync_disks: be more verbose if storage_migrate fails
>   sync_disks: log output of storage_migrate
>   cleanup_remotedisks: also include those migrated with storage_migrate
> 
>  PVE/API2/Qemu.pm            |   9 +--
>  PVE/QemuConfig.pm           |  53 +++++++++++++---
>  PVE/QemuMigrate.pm          |  91 +++++++++++++++++++-------
>  PVE/QemuServer.pm           | 123 +++++++++++++++++++++++++-----------
>  PVE/QemuServer/Cloudinit.pm |   2 +-
>  PVE/QemuServer/Drive.pm     |  61 ------------------
>  PVE/VZDump/QemuServer.pm    |   3 +-
>  7 files changed, 205 insertions(+), 137 deletions(-)
> 
> 
> storage:
> 
> Fabian Ebner (5):
>   storage_migrate: check if target storage supports content type
>   Add apiinfo helper to pvesm
>   Collect optional parameters for storage_migrate into $opts
>   Introduce allow_rename parameter for pvesm import and storage_migrate
>   storage_migrate: add volname_for_storage helper
> 
>  PVE/API2/Storage/Content.pm  |  2 +-
>  PVE/CLI/pvesm.pm             | 52 ++++++++++++++++---
>  PVE/Storage.pm               | 99 ++++++++++++++++++++++++++++++------
>  PVE/Storage/LVMPlugin.pm     | 17 ++++---
>  PVE/Storage/Plugin.pm        | 16 ++++--
>  PVE/Storage/ZFSPoolPlugin.pm | 13 +++--
>  6 files changed, 160 insertions(+), 39 deletions(-)
> 
> -- 
> 2.20.1
> 
> 
> _______________________________________________
> pve-devel mailing list
> pve-devel at pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
>