[pve-devel] [PATCH-SERIES 0/21] remote migration

[Fabian Grünbichler]

this series adds remote migration for VMs (and if the last pve-container
patch is applied, CTs ;)).

both live and offline migration of VMs including NBD and storage-migrated disks
should work, containers don't have any live migration so both offline
and restart mode work identical except for the restart part.

groundwork for extending to pvesr already laid.

uncovered (but not fixed) https://bugzilla.proxmox.com/show_bug.cgi?id=3873
(migration btrfs -> btrfs with snapshots)

new in v5: lots of edge cases fixed, PoC for pve-container, some more
helper moving for re-use in pve-container without duplication

new in v4: lots of small fixes, improved bwlimit handling, `qm` command
(thanks Fabian Ebner and Dominik Csapak for the feedback on v3!)

new in v3: lots of refactoring and edge-case handling

new in v2: dropped parts already applied, incorporated Fabian's and
Dominik's feedback (thanks!)

new in v1: explicit remote endpoint specified as part of API call
instead of
remote.cfg

overview over affected repos and changes, see individual patches for
more details.

pve-guest-common:

new/refactored generic WS/SSH tunnel fork/read/.. helpers
new storage migration over WS tunnel helpers

pve-storage:

refactor storage_migrate to make code-reuse possible
use raw+size for btrfs when no snapshots exist

qemu-server:

some refactoring, new mtunnel endpoints, new remote_migration endpoints,
new `qm` command
TODO: handle pending changes and snapshots
TODO: improve handling of C^c
TODO: strict parser for FW config as well?

as usual, some of the patches are best viewed with '-w', especially in
qemu-server..

pve-container:

--target-storage for regular migration
remote migration PoC
TODO: handle pending changes and snapshots
TODO: strict parser for FW config as well?

pve-common:

map_id helper moved from qemu-server's map_storage

required dependencies are noted, patch overview follows..

pve-common:
  add 'map_id' helper for ID maps

pve-container:
  fix #1532: add target-storage support to migration
  config: add strict parser
  migration: add remote migration

qemu-server:
  move map_storage to PVE::JSONSchema::map_id
  schema: use pve-bridge-id
  parse_config: optional strict mode
  update_vm: allow simultaneous setting of boot-order and dev
  nbd alloc helper: allow passing in explicit format
  migrate: move tunnel-helpers to pve-guest-common
  mtunnel: add API endpoints
  migrate: refactor remote VM/tunnel start
  migrate: add remote migration handling
  api: add remote migrate endpoint
  qm: add remote-migrate command

pve-storage:
  storage_migrate_snapshot: skip for btrfs without snapshots
  storage_migrate: pull out import/export_prepare
  add volume_import/export_start helpers

pve-guest-common:
  migrate: add get_bwlimit helper
  add tunnel helper module
  add storage tunnel module