[pve-devel] [PATCH guest-common/qemu-server/manager/docs v7] implement experimental vgpu live migration

[Dominik Csapak]

== Summmary ==

This is implemented for mapped resources. This requires driver and
hardware support, but aside from nvidia vgpus there don't seem to be
many drivers (if any) that do support that.

qemu already supports that for vfio-pci devices, so nothing to be done
there besides actively enabling it.

Since we currently can't properly test it here and very much depends on
hardware/driver support, mark it as experimental everywhere
(docs/api/gui). (i tested it with a single server with multiple pve
"containers" that each got several virtual functions, so the migration
was actually to the same hardware but via our stack between two
different qemu processes)

i opted for marking them migratable at the mapping level, but we could
theoretically also put it in the hostpciX config instead. (though imho
it fits better in the cluster-wide resource mapping config)

also the naming/texts could probably be improved, but i think
'live-migration-capable' is very descriptive and i didn't want to use an
overly short name for it (which can be confusing, see the 'shared' flag
for storages)

== Dependencies ==

qemu-server depend on pve-guest-common and require a bumped pve-common

pve-manager depends on pve-guest-common and qemu-server

== Changelog ==

changes from v6:
* rebase on master
* include christophs and fioans reviewed-by where appropriate
* fixed some nits (see individual patches)
* don't add live migratable hostpci devices to 'local_resources' as blockers
* add a new patch to omit mapped hostpci devices to 'local_resources' as
  blockers (like it was intended in the first place)
  -> this make one of the bulk migration patches unnecessary so I dropped it
* don't change the check for bulk migration preconditions
  (was wrong for other local resources)

changes from v5:
* rebased on master
* new common patch that was missing last time
* dropped the move of find_on_current_node, since it only makes
  our lives harder and we don't gain much from it.
  -> this also fixed some bugs that were there in v5 due to
  the move.
* reordered some patches, so the dependencies are clearer
* added a patch that adds a state-migration summary for live
  migration (so we can see how much state was actually transferred)
* added missing colon in log output

changes from v4:
* rebased on master (some work due to the recent nvidia changes)
* incorporated thomas/alexanders feedback from v4

changes from v3:
* rebased on master
* split first guest-common patch into 3
* instead of merging keys, just write all expected keys in to expected_props
* made $cfg optional so it does not break callers that don't call it
* added patch to fix the cfg2cmd tests for mdev check
* added patch to show vfio state transferred for migration
* incorporated fionas feedback (mostly minor stuff)

changes from v2:
* rebased on master
* rework the rework of the properties check (pve-guest-common 1/4)
* properly check mdev in the gui (pve-manager 1/5)

pve-guest-common:

Dominik Csapak (2):
  mapping: pci: check the mdev configuration on the device too
  mapping: pci: add 'live-migration-capable' flag to mappings

 src/PVE/Mapping/PCI.pm | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

qemu-server:

Dominik Csapak (9):
  tests: cfg2cmd: fix mdev tests
  pci: mapping: check mdev config against hardware
  pci: set 'enable-migration' to on for live-migration marked mapped
    devices
  check_local_resources: add more info per mapped device and return as
    hash
  check_local_resources: allow mapped devices for offline migration
  api: enable live migration for marked mapped pci devices
  api: include not mapped resources for running vms in migrate
    preconditions
  migrate: show vfio state transferred too
  migrate: add transfer summary

 PVE/API2/Qemu.pm                 | 59 +++++++++++++++++++-------------
 PVE/QemuMigrate.pm               | 46 +++++++++++++++++++++----
 PVE/QemuServer.pm                | 24 +++++++++----
 PVE/QemuServer/PCI.pm            | 10 +++++-
 test/run_config2command_tests.pl |  2 +-
 5 files changed, 101 insertions(+), 40 deletions(-)

pve-manager:

Dominik Csapak (4):
  mapping: pci: include mdev in config checks
  bulk migrate: improve precondition checks
  ui: adapt migration window to precondition api change
  fix #5175: ui: allow configuring and live migration of mapped pci
    resources

 PVE/API2/Cluster/Mapping/PCI.pm   |  2 +-
 PVE/API2/Nodes.pm                 | 12 ++++++++
 www/manager6/dc/PCIMapView.js     |  8 +++++
 www/manager6/window/Migrate.js    | 51 ++++++++++++++++++++-----------
 www/manager6/window/PCIMapEdit.js | 20 +++++++++---
 5 files changed, 70 insertions(+), 23 deletions(-)

pve-docs:

Dominik Csapak (2):
  qm: resource mapping: add description for `mdev` option
  qm: resource mapping: document `live-migration-capable` setting

 qm.adoc | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

-- 
2.39.5