[pve-devel] [PATCH v3 00/12] Add basics for custom CPU models

[Stefan Reiter]

Sorry, that's supposed to be v5 in the subject...

On 10/15/19 4:12 PM, Stefan Reiter wrote:
> Based on the RFC and following on- and off-list discussion about custom CPU
> models [0].
> 
> In essence, this revised patch allows a user to specify custom CPU models in
> /etc/pve/cpu-models.conf (section-config style [1]), where VMs using that CPU
> model inherit details from the definition. This removes any fragile
> "auto-magical" CPU flag detection, while still giving the user a way to create
> VMs with the best possible subset of CPU features maintaining live-migration
> compatibility.
> 
> Includes the infrastructure for broadcasting supported CPU flags for each
> cluster-node via the key-value store - this is not necessary for the
> custom-cpu feature in particular, but I think could prove useful for
> implementing the GUI part (e.g. show the user which flags are supported on which
> nodes).
> 
> I intentionally wanted to send this series before starting any GUI or new API
> work, to get some feedback if this approach works better than the cluster-cpu
> one.
> 
> v4 -> v5:
> 
> * fix flag broadcasting error detection
> * check for empty flag file on generation in pve-qemu
> * get_model_by_name -> get_custom_model w/o defaults and inlined conf loading
> * rename verify_vm_cpu_conf to parse_vm_cpu_conf and fix property checking
>    (even custom models have restrictions on certain properties when used to
>    verify a VM-specific configuration)
> * add pve-cpu-conf format for future API additions
> * smaller cleanups/renamings/commenting according to Fabian's review
> 
> v3 -> v4:
> 
> * Change "built-in"/"custom" flag to "custom-" prefix for namespacing
> * Include missing pve-cluster patch for cfs_ functions
> 
> Previously forgot to mention for v3:
> * Rebased to master to include Alexandre's Intel CPU model patches
> 
> v2 -> v3:
> 
> * Shuffle some code between patches to make sure nothing breaks in-between
> * Change "query_understood_cpu_flags" to use static file generated at compile
>    time of "pve-qemu"
> * Fix "query_supported_cpu_flags" to accomodate tcg accelerator (instead of kvm
>    only). Also fix broadcast to set both variants, and better failure handling.
> * Useful minimum value for phys-bits
> * Rename/Negate property: "custom" -> "built-in"
> * more/better comments, minor code/commit fixes as suggested by Thomas' review
> 
> v1 -> v2:
> 
> Quite a big change to v1. Once I began reusing the $cpu_fmt hash for both custom
> CPU models and VM-specific settings, I realized the approach mentioned by Fabian
> - to move some helpers from QemuServer to the new CPUConfig
> - would actually make a lot of sense.
> 
> * Re-query supported CPU flags when QEMU/KVM updates (without needing to
>    restart pvestatd) using kvm_user_version
> * Improve documentation and error handling on CPU flag querying helpers
> * Rename CustomCPUConfig -> CPUConfig
> * Extract some CPU helpers from QemuServer to CPUConfig
> * Use cfs_ functions for IO, don't use 'bless'
> * Merge $cpu_fmt for custom models and VM-specific CPU settings
> * Rename 'basemodel' -> 'reported-model', only support default models as
>    reported (avoids custom models depending on each other for now)
> * Add new CPU flag resolving infrastructure
> * Add and fix test cases
> 
> Does not include any fix for versioning/live-migration with custom models for
> now, felt these changes were big enough. Also applies to CPU hotplug with custom
> models.
> 
> 
> [0]: https://pve.proxmox.com/pipermail/pve-devel/2019-July/038268.html
> [1]: e.g.:
> cpu-model: custom-cpu-name
>      host-phys-bits 1
>      flags +aes;+avx;+avx2
>      reported-model kvm64
> 
> 
> manager: Stefan Reiter (1):
>    Broadcast supported CPU flags
> 
>   PVE/Service/pvestatd.pm | 47 +++++++++++++++++++++++++++++++++++++++--
>   1 file changed, 45 insertions(+), 2 deletions(-)
> 
> qemu: Stefan Reiter (1):
>    Write understood CPU flags into static file
> 
>   debian/rules | 14 ++++++++++++++
>   1 file changed, 14 insertions(+)
> 
> cluster: Stefan Reiter (1):
>    Add "cpu-models.conf" to observed files
> 
>   data/PVE/Cluster.pm | 1 +
>   data/src/status.c   | 1 +
>   2 files changed, 2 insertions(+)
> 
> qemu-server: Stefan Reiter (9):
>    Add QEMU CPU flag querying helpers
>    Add CPUConfig file and migrate some helpers
>    Adapt CPUConfig to handle custom models
>    Add overrides and convenience functions to CPUConfig
>    Verify VM-specific CPU configs seperately
>    Add helpers to better structure CPU option handling
>    Rework get_cpu_options and allow custom CPU models
>    fix #2318: allow phys-bits and host-phys-bits CPU settings
>    cfg2cmd: add test case for custom CPU model
> 
>   PVE/QemuServer.pm                          | 333 ++++--------
>   PVE/QemuServer/CPUConfig.pm                | 599 +++++++++++++++++++++
>   PVE/QemuServer/Makefile                    |   1 +
>   test/cfg2cmd/custom-cpu-model.conf         |   8 +
>   test/cfg2cmd/custom-cpu-model.conf.cmd     |  27 +
>   test/cfg2cmd/i440fx-win10-hostpci.conf.cmd |   2 +-
>   test/cfg2cmd/minimal-defaults.conf.cmd     |   2 +-
>   test/cfg2cmd/q35-linux-hostpci.conf.cmd    |   2 +-
>   test/cfg2cmd/q35-win10-hostpci.conf.cmd    |   2 +-
>   test/cfg2cmd/simple1.conf.cmd              |   2 +-
>   test/cfg2cmd/spice-usb3.conf.cmd           |   2 +-
>   test/run_config2command_tests.pl           |  21 +
>   12 files changed, 767 insertions(+), 234 deletions(-)
>   create mode 100644 PVE/QemuServer/CPUConfig.pm
>   create mode 100644 test/cfg2cmd/custom-cpu-model.conf
>   create mode 100644 test/cfg2cmd/custom-cpu-model.conf.cmd
>