[pve-devel] [PATCH/RFC cluster/common/... many] add cluster-wide hardware device mapping

[Dominik Csapak]

this series aims to add a cluster-wide device mapping for (atm) pci and
usb devices. so that an admin can configure a device to be availble
for migration and configuring for uses that are non-root

built-in are some additional safety checks in contrast to current
passthrough, e.g. if pci addresses shift, with the mapping
we can detect that and prevent a vm to boot with the wrong device
(in most cases, there are some edge cases when one has multiple
of the same device, e.g. the same gpu, that we cannot detect)

all in all the series should be functional, but there are some parts
that are not finished/rough:
* the config format
    for now i used a section config with 'hostname:id' as section
    header, but this is not optimal. after some discussion with several
    colleagues, i'll probably settle for a config file with pure json
    in it, since there is no good way to have the necessary nesting
    level (a mapping per node per id + properties per id) and have
    it reasonably integrate in the section config, while maintaining
    ease of use with a normal editor (ideally we would have anoter
    config format like json with comments/trailing commas, or
    rust-object-notation)

    if there are some better ideas, please tell!

* node api
    here the section config helps us, but we have the usual weirdness
    with all options everytime available. also currently the api is
    limited to the node level, and i have no good solution for
    distributing the current state of the devices yet
    (we can use the pvestatd as usual, but it does so many things
    already; the rewrite really becomes a blocker for many things)

* cluster-wide gui
    depends on the cluster wide api call for listing, which i omitted
    for now for the reasons i gave above

* vm editing gui
    not completely happy with the result, but could not come up
    with something better in the time i had until now

* config file location
    for now it lives in pve-common, though after looking at the
    dependency chain, it would be possible to move it to qemu-server
    does not really matter to me, and if we sometime want to extend
    it to containers it needs to be at least in pve-guest-common

* auto-generated roles:
    i did not find a way to have a 'PVEHardwareUser' role without an
    'PVEHardwareAdmin', but the 'Admin' would have the same privs.
    so i settled on only having an Admin. Not sure if that makes sense..

dependencies:
    manager depends on qemu-server,pve-access-control,pve-common
    qemu-server depends on pve-access-control,pve-common
    pve-common depends on pve-cluster

additioanl info:
pve-common 1/3 and 2/3 are necssary but could be independetly applied,
no breaking change


pve-cluster:

Dominik Csapak (1):
  add nodes/hardware-map.conf

 data/PVE/Cluster.pm | 1 +
 data/src/status.c   | 1 +
 2 files changed, 2 insertions(+)

pve-common:

Dominik Csapak (3):
  SysFSTools: add verbose flag to pci_device_info
  SysFSTools: change 'product' to 'device'
  add PVE/HardwareMap and Plugins

 src/Makefile                     |  4 ++
 src/PVE/HardwareMap.pm           | 54 ++++++++++++++++++++
 src/PVE/HardwareMap/PCIPlugin.pm | 87 ++++++++++++++++++++++++++++++++
 src/PVE/HardwareMap/Plugin.pm    | 82 ++++++++++++++++++++++++++++++
 src/PVE/HardwareMap/USBPlugin.pm | 69 +++++++++++++++++++++++++
 src/PVE/SysFSTools.pm            | 33 +++++++++---
 6 files changed, 323 insertions(+), 6 deletions(-)
 create mode 100644 src/PVE/HardwareMap.pm
 create mode 100644 src/PVE/HardwareMap/PCIPlugin.pm
 create mode 100644 src/PVE/HardwareMap/Plugin.pm
 create mode 100644 src/PVE/HardwareMap/USBPlugin.pm

pve-access-control:

Dominik Csapak (2):
  PVE/AccessControl: add Hardware.* privileges and /hardware/ paths
  PVE/RPCEnvironment: add helper for checking hw permissions

 src/PVE/API2/AccessControl.pm |  3 ++-
 src/PVE/AccessControl.pm      | 13 +++++++++++++
 src/PVE/RPCEnvironment.pm     |  8 ++++++++
 3 files changed, 23 insertions(+), 1 deletion(-)

qemu-server:

Dominik Csapak (7):
  PVE/QemuServer: allow mapped usb devices in config
  PVE/QemuServer: allow mapped pci deviced in config
  PVE/API2/Qemu: add permission checks for mapped usb devices
  PVE/API2/Qemu: add permission checks for mapped pci devices
  PVE/QemuServer: extend 'check_local_resources' for mapped resources
  PVE/API2/Qemu: migrate preconditions: use new check_local_resources
    info
  PVE/QemuMigrate: check for mapped resources on migration

 PVE/API2/Qemu.pm      | 108 ++++++++++++++++++++++++++++++++++++++----
 PVE/QemuMigrate.pm    |  13 ++++-
 PVE/QemuServer.pm     |  38 ++++++++++++++-
 PVE/QemuServer/PCI.pm |  22 ++++++++-
 PVE/QemuServer/USB.pm |  22 ++++++++-
 5 files changed, 188 insertions(+), 15 deletions(-)

pve-manager:

Dominik Csapak (8):
  PVE/API2/Hardware: add Mapping.pm
  ui: form/USBSelector: make it more flexible with nodename
  ui: form: add PCIMapSelector
  ui: form: add USBMapSelector
  ui: node: add HardwareView and relevant edit windows
  ui: qemu/PCIEdit: rework panel to add a mapped configuration
  ui: qemu/USBEdit: add 'mapped' device case
  ui: window/Migrate: allow mapped devices

 PVE/API2/Hardware.pm                |   6 +
 PVE/API2/Hardware/Makefile          |   1 +
 PVE/API2/Hardware/Mapping.pm        | 292 +++++++++++++
 www/manager6/Makefile               |   3 +
 www/manager6/form/PCIMapSelector.js |  95 +++++
 www/manager6/form/USBMapSelector.js |  73 ++++
 www/manager6/form/USBSelector.js    |  32 +-
 www/manager6/node/Config.js         |   8 +
 www/manager6/node/HardwareView.js   | 641 ++++++++++++++++++++++++++++
 www/manager6/qemu/PCIEdit.js        | 231 ++++++----
 www/manager6/qemu/USBEdit.js        |  34 +-
 www/manager6/window/Migrate.js      |  37 +-
 12 files changed, 1361 insertions(+), 92 deletions(-)
 create mode 100644 PVE/API2/Hardware/Mapping.pm
 create mode 100644 www/manager6/form/PCIMapSelector.js
 create mode 100644 www/manager6/form/USBMapSelector.js
 create mode 100644 www/manager6/node/HardwareView.js

-- 
2.20.1