[pve-devel] [PATCH v7 00/10] Add basics for custom CPU models

[Stefan Reiter]

Based on the RFC and following on- and off-list discussion about custom CPU
models [0].

In essence, this revised patch allows a user to specify custom CPU models in
/etc/pve/cpu-models.conf (section-config style [1]), where VMs using that CPU
model inherit details from the definition. This removes any fragile
"auto-magical" CPU flag detection, while still giving the user a way to create
VMs with the best possible subset of CPU features maintaining live-migration
compatibility.

Includes the infrastructure for broadcasting supported CPU flags for each
cluster-node via the key-value store - this is not necessary for the
custom-cpu feature in particular, but I think could prove useful for
implementing the GUI part (e.g. show the user which flags are supported on which
nodes).

I intentionally wanted to send this series before starting any GUI or new API
work, to get some feedback if this approach works better than the cluster-cpu
one.

v6 -> v7:

* rebased on master
* moved cpu-models.conf to /etc/pve/virtual-guest/cpu-models.conf (not entirely
  satisfied with the name 'virtual-guest' for the folder, but couldn't think of
  anything better...)
* squashed patches 4 and 5 (now patch 3)
* dropped applied patches
* added patch for snapshots and suspend to work with custom CPU models

v5 -> v6:

* rebased on latest masters and updated qemu-server patches to work with
  refactored interfaces (no cyclic dependencies introduced anymore!)
* fixed nits from Fabian's v5 review
* removed already merged pve-qemu patch, update file path to point to Thomas'
  updated version of _cpuid_ file
* added patch for live migration compatibility with custom CPU models (patch 11)

v4 -> v5:

* fix flag broadcasting error detection
* check for empty flag file on generation in pve-qemu
* get_model_by_name -> get_custom_model w/o defaults and inlined conf loading
* rename verify_vm_cpu_conf to parse_vm_cpu_conf and fix property checking
  (even custom models have restrictions on certain properties when used to
  verify a VM-specific configuration)
* add pve-cpu-conf format for future API additions
* smaller cleanups/renamings/commenting according to Fabian's review

v3 -> v4:

* Change "built-in"/"custom" flag to "custom-" prefix for namespacing
* Include missing pve-cluster patch for cfs_ functions

Previously forgot to mention for v3:
* Rebased to master to include Alexandre's Intel CPU model patches

v2 -> v3:

* Shuffle some code between patches to make sure nothing breaks in-between
* Change "query_understood_cpu_flags" to use static file generated at compile
  time of "pve-qemu"
* Fix "query_supported_cpu_flags" to accomodate tcg accelerator (instead of kvm
  only). Also fix broadcast to set both variants, and better failure handling.
* Useful minimum value for phys-bits
* Rename/Negate property: "custom" -> "built-in"
* more/better comments, minor code/commit fixes as suggested by Thomas' review

v1 -> v2:

Quite a big change to v1. Once I began reusing the $cpu_fmt hash for both custom
CPU models and VM-specific settings, I realized the approach mentioned by Fabian
- to move some helpers from QemuServer to the new CPUConfig
- would actually make a lot of sense.

* Re-query supported CPU flags when QEMU/KVM updates (without needing to
  restart pvestatd) using kvm_user_version
* Improve documentation and error handling on CPU flag querying helpers
* Rename CustomCPUConfig -> CPUConfig
* Extract some CPU helpers from QemuServer to CPUConfig
* Use cfs_ functions for IO, don't use 'bless'
* Merge $cpu_fmt for custom models and VM-specific CPU settings
* Rename 'basemodel' -> 'reported-model', only support default models as
  reported (avoids custom models depending on each other for now)
* Add new CPU flag resolving infrastructure
* Add and fix test cases

Does not include any fix for versioning/live-migration with custom models for
now, felt these changes were big enough. Also applies to CPU hotplug with custom
models.


[0]: https://pve.proxmox.com/pipermail/pve-devel/2019-July/038268.html
[1]: e.g.:
cpu-model: custom-cpu-name
    host-phys-bits 1
    flags +aes;+avx;+avx2
    reported-model kvm64


cluster: Stefan Reiter (1):
  Add "cpu-models.conf" to observed files

 data/PVE/Cluster.pm | 2 +-
 data/src/status.c   | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)

qemu-server: Stefan Reiter (9):
  Add CPUConfig file and migrate some helpers
  Adapt CPUConfig to handle custom models
  Verify VM-specific CPU configs seperately
  Add helpers to better structure CPU option handling
  Rework get_cpu_options and allow custom CPU models
  fix #2318: allow phys-bits and host-phys-bits CPU settings
  cfg2cmd: add test cases for custom CPU models
  Include "-cpu" parameter with live-migration
  Include "-cpu" parameter with snapshots/suspend

 PVE/API2/Qemu.pm                              |  10 +-
 PVE/QemuConfig.pm                             |  36 +-
 PVE/QemuMigrate.pm                            |  22 +
 PVE/QemuServer.pm                             | 261 +-------
 PVE/QemuServer/CPUConfig.pm                   | 579 ++++++++++++++++++
 PVE/QemuServer/Makefile                       |   1 +
 test/cfg2cmd/custom-cpu-model-defaults.conf   |   8 +
 .../custom-cpu-model-defaults.conf.cmd        |  24 +
 test/cfg2cmd/custom-cpu-model.conf            |   8 +
 test/cfg2cmd/custom-cpu-model.conf.cmd        |  27 +
 test/cfg2cmd/i440fx-win10-hostpci.conf.cmd    |   2 +-
 test/cfg2cmd/minimal-defaults.conf.cmd        |   2 +-
 test/cfg2cmd/pinned-version.conf.cmd          |   2 +-
 .../q35-linux-hostpci-multifunction.conf.cmd  |   2 +-
 test/cfg2cmd/q35-linux-hostpci.conf.cmd       |   2 +-
 test/cfg2cmd/q35-win10-hostpci.conf.cmd       |   2 +-
 test/cfg2cmd/simple1.conf.cmd                 |   2 +-
 test/cfg2cmd/spice-enhancments.conf.cmd       |   2 +-
 test/cfg2cmd/spice-linux-4.1.conf.cmd         |   2 +-
 test/cfg2cmd/spice-usb3.conf.cmd              |   2 +-
 test/cfg2cmd/spice-win.conf.cmd               |   2 +-
 test/run_config2command_tests.pl              |  23 +
 22 files changed, 771 insertions(+), 250 deletions(-)
 create mode 100644 PVE/QemuServer/CPUConfig.pm
 create mode 100644 test/cfg2cmd/custom-cpu-model-defaults.conf
 create mode 100644 test/cfg2cmd/custom-cpu-model-defaults.conf.cmd
 create mode 100644 test/cfg2cmd/custom-cpu-model.conf
 create mode 100644 test/cfg2cmd/custom-cpu-model.conf.cmd

-- 
2.20.1