[pve-devel] [PATCH many v3] add cluster-wide hardware device mapping
Dominik Csapak
d.csapak at proxmox.com
Tue Sep 20 14:50:06 CEST 2022
this series aims to add a cluster-wide device mapping for pci and usb devices.
so that an admin can configure a device to be availble for migration and
configuring for uses that are non-root
this version is mostly the same as v2, aside from some bugfixes, rebase
and preventing from having mdev: 1 set for multifunction devices.
i would appreciate if somebody could take a look at this series
again ;) (below is the old cover letter + changelog)
built-in are some additional safety checks in contrast to current
passthrough, e.g. if pci addresses shift, with the mapping
we can detect that and prevent a vm to boot with the wrong device
(in most cases, there are some edge cases when one has multiple
of the same device, e.g. the same gpu, that we cannot detect)
new in this version is the ability to specify multiple devices for
each host mapping, such that we can select the first free one on
starting the vm (this fixes #3574). That makes using vGPUs and SR-IOV
much more useful as a user does not have to hardcode the pci ids anymore
i left that feature seperated in a patch for pve-common(3/3) and in
qemu-server ({12,13}/13) in the backend for easier review, but did not bother
to do it for the gui (if we really don't want it, i can just send a different
version for the ui)
also pve-common 1/3 and qemu-server 1-4/13 are general cleanups that
would even make sense without the remaining patches
(qemu-server 1/13 depends on pve-common 1/3)
changes from v2:
* some bug fixes (e.g use of unitialized variable)
* don't set mdev for multifunction devices
-> this should fix alexandres issue, since it's not possible anymore
to select a mediated device when having a multifunction device
selected
changes from v1:
* dropped 'check_hw_perm' (just use 'check_full' now)
* added some cleanups
* renamed the buttons in the ui (hopefully better now)
* added multi device mapping for each host
this includes a new 'multi pci' selector for that window, which
automatically adds entries for the whole slots which, when selected,
disabled the selection of the individual functions
* fixed some issues (e.g. missing entries in the 'caps' object, wrong
usb config parsing, etc.)
changes from the rfc:
* new cluster wide gui instead of node-local one (removed that, since
it's not necessary when we have a cluster-wide one)
* uses json instead of a section config
* api is quite different overall, i split the type into its own level
for configuring, similar to what we do in pbs
(e.g. /nodes/NODENAME/hardware/mapping/usb/)
* fixed quite some bugs the rfc had
* added patch for handling the gui with limited permissions better
* added a 'comment' field for mappings
dependencies:
pve-common (1) breaks current qemu-server
pve-common (2,3) depends on pve-cluster
qemu-server (1-4) depends on pve-common (1)
qemu-server (5-11) depends on qemu-server(<5), pve-access-control,pve-common (2)
qemu-server (12,13) depends on qemu-server(<12), pve-common (3)
manager depends on qemu-server,pve-access-control,pve-common
pve-cluster:
Dominik Csapak (1):
add nodes/hardware-map.conf
data/PVE/Cluster.pm | 1 +
data/src/status.c | 1 +
2 files changed, 2 insertions(+)
pve-access-control:
Dominik Csapak (1):
PVE/AccessControl: add Hardware.* privileges and /hardware/ paths
src/PVE/AccessControl.pm | 13 +++++++++++++
src/PVE/RPCEnvironment.pm | 3 ++-
2 files changed, 15 insertions(+), 1 deletion(-)
pve-common:
Dominik Csapak (3):
SysFSTools: make mdev cleanup independent of pciid
add PVE/HardwareMap
HardwareMap: add support for multiple pci device paths per mapping
src/Makefile | 1 +
src/PVE/HardwareMap.pm | 378 +++++++++++++++++++++++++++++++++++++++++
src/PVE/SysFSTools.pm | 6 +-
3 files changed, 381 insertions(+), 4 deletions(-)
create mode 100644 src/PVE/HardwareMap.pm
qemu-server:
Dominik Csapak (13):
cleanup pci devices in more situations
PCI: make mediated device path independent of pci id
PCI: refactor print_pci_device
PCI: reuse parsed info from print_hostpci_devices
PVE/QemuServer: allow mapped usb devices in config
PVE/QemuServer: allow mapped pci deviced in config
PVE/API2/Qemu: add permission checks for mapped usb devices
PVE/API2/Qemu: add permission checks for mapped pci devices
PVE/QemuServer: extend 'check_local_resources' for mapped resources
PVE/API2/Qemu: migrate preconditions: use new check_local_resources
info
PVE/QemuMigrate: check for mapped resources on migration
fix #3574: enable multi pci device mapping from config
add tests for mapped pci devices
PVE/API2/Qemu.pm | 109 +++++++++++--
PVE/QemuMigrate.pm | 13 +-
PVE/QemuServer.pm | 137 +++++++++++-----
PVE/QemuServer/PCI.pm | 149 +++++++++++++-----
PVE/QemuServer/USB.pm | 21 ++-
test/MigrationTest/Shared.pm | 7 +
test/cfg2cmd/q35-linux-hostpci-mapping.conf | 17 ++
.../q35-linux-hostpci-mapping.conf.cmd | 36 +++++
test/run_config2command_tests.pl | 76 +++++++++
9 files changed, 473 insertions(+), 92 deletions(-)
create mode 100644 test/cfg2cmd/q35-linux-hostpci-mapping.conf
create mode 100644 test/cfg2cmd/q35-linux-hostpci-mapping.conf.cmd
pve-manager:
Dominik Csapak (13):
PVE/API2/Hardware: add Mapping.pm
PVE/API2/Cluster: add Hardware mapping list api call
ui: form/USBSelector: make it more flexible with nodename
ui: form: add PCIMapSelector
ui: form: add USBMapSelector
ui: qemu/PCIEdit: rework panel to add a mapped configuration
ui: qemu/USBEdit: add 'mapped' device case
ui: form: add MultiPCISelector
ui: add window/PCIEdit: edit window for pci mappings
ui: add window/USBEdit: edit window for usb mappings
ui: add dc/HardwareView: a CRUD interface for hardware mapping
ui: window/Migrate: allow mapped devices
ui: improve permission handling for hardware
PVE/API2/Cluster.pm | 8 +
PVE/API2/Cluster/Hardware.pm | 117 +++++
PVE/API2/Cluster/Makefile | 1 +
PVE/API2/Hardware.pm | 6 +
PVE/API2/Hardware/Makefile | 1 +
PVE/API2/Hardware/Mapping.pm | 708 ++++++++++++++++++++++++++
www/css/ext6-pve.css | 4 +
www/manager6/Makefile | 6 +
www/manager6/data/PermPathStore.js | 1 +
www/manager6/dc/Config.js | 18 +-
www/manager6/dc/HardwareView.js | 324 ++++++++++++
www/manager6/form/MultiPCISelector.js | 289 +++++++++++
www/manager6/form/PCIMapSelector.js | 102 ++++
www/manager6/form/PCISelector.js | 18 +-
www/manager6/form/USBMapSelector.js | 73 +++
www/manager6/form/USBSelector.js | 33 +-
www/manager6/qemu/HardwareView.js | 17 +-
www/manager6/qemu/PCIEdit.js | 314 ++++++++----
www/manager6/qemu/USBEdit.js | 36 +-
www/manager6/window/Migrate.js | 37 +-
www/manager6/window/PCIEdit.js | 283 ++++++++++
www/manager6/window/USBEdit.js | 248 +++++++++
22 files changed, 2513 insertions(+), 131 deletions(-)
create mode 100644 PVE/API2/Cluster/Hardware.pm
create mode 100644 PVE/API2/Hardware/Mapping.pm
create mode 100644 www/manager6/dc/HardwareView.js
create mode 100644 www/manager6/form/MultiPCISelector.js
create mode 100644 www/manager6/form/PCIMapSelector.js
create mode 100644 www/manager6/form/USBMapSelector.js
create mode 100644 www/manager6/window/PCIEdit.js
create mode 100644 www/manager6/window/USBEdit.js
--
2.30.2
More information about the pve-devel
mailing list