[pve-devel] [PATCH v3 qemu-server++ 0/21] remote migration

[Fabian Grünbichler]

some feedback/known issues/.. from Dominik's testing:
- there is some mishandling with the export/import (hash keys with '-' 
  vs '_') that breaks rename and qcow2 support (already fixed locally)
- we could probably drop the targetnode parameter and replace it with 
  localhost, if we punt the node -> endpoint selection to the client - 
  otherwise we need to do a lookup including IP in the remote migration 
  API call
- targetstorage should get an adapted description ('1' not valid)
- ceph plugin doesn't implement import/export (yet)
- plugins not implementing import/export should be handled with a good 
  error message and as early as possible (when scanning disks?)
- source being kept around locked if --delete is not passed might not be 
  the best solution - alternatives?
- mtunnel is missing a timeout for the initial connection/first read

thanks for testing and discussion (off-list)!

please add if I forgot anything ;)

On December 22, 2021 2:52 pm, Fabian Grünbichler wrote:
> this series adds remote migration for VMs.
> 
> both live and offline migration including NBD and storage-migrated disks should
> work. groundwork for extending to pve-container and pvesr already laid.
> 
> new in v3: lots of refactoring and edge-case handling
> 
> new in v2: dropped parts already applied, incorporated Fabian's and
> Dominik's feedback (thanks!)
> 
> new in v1: explicit remote endpoint specified as part of API call instead of
> remote.cfg
> 
> overview over affected repos and changes, see individual patches for
> more details.
> 
> proxmox-websocket-tunnel:
> 
> new tunnel helper tool for forwarding commands and data over websocket
> connections, required by pve-guest-common on source side
> 
> pve-guest-common:
> 
> new/refactored generic WS/SSH tunnel fork/read/.. helpers
> new storage migration over WS tunnel helpers
> 
> pve-storage:
> 
> refactor storage_migrate to make code-reuse possible
> 
> qemu-server:
> 
> some refactoring, new mtunnel endpoints, new remote_migration endpoints
> TODO: handle pending changes and snapshots
> TODO: proper CLI for remote migration
> TODO: handle C^c
> TODO: strict parser for FW config as well?
> potential TODO: precond endpoint?
> 
> as usual, some of the patches are best viewed with '-w', especially in
> qemu-server..
> 
> required dependencies are noted
> 
> pve-guest-common:
> 
> Fabian Grünbichler (3):
>   migrate: handle migration_network with remote migration
>   add tunnel helper module
>   add storage tunnel module
> 
>  src/Makefile               |   2 +
>  debian/control             |   1 +
>  src/PVE/AbstractMigrate.pm |  37 ++--
>  src/PVE/StorageTunnel.pm   | 231 ++++++++++++++++++++++++
>  src/PVE/Tunnel.pm          | 356 +++++++++++++++++++++++++++++++++++++
>  5 files changed, 611 insertions(+), 16 deletions(-)
>  create mode 100644 src/PVE/StorageTunnel.pm
>  create mode 100644 src/PVE/Tunnel.pm
> 
> proxmox-websocket-tunnel:
> 
> Fabian Grünbichler (4):
>   initial commit
>   add tunnel implementation
>   add fingerprint validation
>   add packaging
> 
> qemu-server:
> 
> Fabian Grünbichler (10):
>   refactor map_storage to map_id
>   schema: use pve-bridge-id
>   parse_config: optional strict mode
>   update_vm: allow simultaneous setting of boot-order and dev
>   nbd alloc helper: allow passing in explicit format
>   migrate: move tunnel-helpers to pve-guest-common
>   mtunnel: add API endpoints
>   migrate: refactor remote VM/tunnel start
>   migrate: add remote migration handling
>   api: add remote migrate endpoint
> 
>  PVE/API2/Qemu.pm                      | 721 ++++++++++++++++++++++++-
>  PVE/QemuMigrate.pm                    | 738 ++++++++++++++++----------
>  PVE/QemuServer.pm                     | 100 ++--
>  debian/control                        |   2 +
>  test/MigrationTest/QemuMigrateMock.pm |  28 +-
>  5 files changed, 1240 insertions(+), 349 deletions(-)
> 
> pve-storage:
> 
> Fabian Grünbichler (4):
>   volname_for_storage: parse volname before calling
>   storage_migrate: pull out snapshot decision
>   storage_migrate: pull out import/export_prepare
>   add volume_import/export_start helpers
> 
>  PVE/Storage.pm | 205 ++++++++++++++++++++++++++++++++++++-------------
>  1 file changed, 153 insertions(+), 52 deletions(-)
> 
> 
> _______________________________________________
> pve-devel mailing list
> pve-devel at lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>