[pve-devel] [PATCH docs/ha-manager/manager v5 00/29] HA Rules
Daniel Kral
d.kral at proxmox.com
Wed Jul 30 19:59:27 CEST 2025
Here's a (hopefully) final update on the core HA rules series.
@Michael and @Hannes Duerr have been a great help for testing this
series, thank you very much!
For maintainers: ha-manager patch #19 should be updated to the correct
pve-manager version that is dependent on the pve-ha-manager package
which can interpret the HA rules config.
Changelog since v4
------------------
- rebased on newest available master
- postponed persistent rules migration only if all ha groups migration
checks passed
- also check the lrm status of all nodes as additional ha groups
migration checks
- do not allow modifying ha rules api calls unless the ha groups have
been fully migrated
- do not allow reading/modifying ha group api calls after the ha groups
have been fully migrated (after thought: we could be more loose on the
reading api endpoints?)
- do not allow 'group' parameter for ha resources api calls after the ha
groups have been fully migrated
- the migrated ha groups' comment is not overwritten anymore
- fixed the ha-manager CLI for 'ha-manager rules config' to display an
overview of the ha rules on the CLI too
ha-manager:
Daniel Kral (23):
tree-wide: make arguments for select_service_node explicit
manager: improve signature of select_service_node
introduce rules base plugin
rules: introduce node affinity rule plugin
config, env, hw: add rules read and parse methods
config: delete services from rules if services are deleted from config
manager: read and update rules config
test: ha tester: add test cases for future node affinity rules
resources: introduce failback property in ha resource config
manager: migrate ha groups to node affinity rules in-memory
manager: apply node affinity rules when selecting service nodes
test: add test cases for rules config
api: introduce ha rules api endpoints
cli: expose ha rules api endpoints to ha-manager cli
sim: do not create default groups config
test: ha tester: migrate groups to service and rules config
test: ha tester: replace any reference to groups with node affinity
rules
env: add property delete for update_service_config
manager: persistently migrate ha groups to ha rules
api: groups: disallow calls to ha groups endpoints if fully migrated
api: resources: exclude group property in reading endpoints if
migrated
api: resources: disallow group prop in modifying endpoints if migrated
api: rules: disallow modifying api calls if ha groups not migrated
.gitignore | 1 +
debian/pve-ha-manager.install | 3 +
src/PVE/API2/HA/Groups.pm | 25 +-
src/PVE/API2/HA/Makefile | 2 +-
src/PVE/API2/HA/Resources.pm | 24 +-
src/PVE/API2/HA/Rules.pm | 394 +++++++++++++++
src/PVE/API2/HA/Status.pm | 11 +-
src/PVE/CLI/ha_manager.pm | 36 ++
src/PVE/HA/Config.pm | 67 ++-
src/PVE/HA/Env.pm | 34 +-
src/PVE/HA/Env/PVE2.pm | 45 +-
src/PVE/HA/Groups.pm | 49 ++
src/PVE/HA/Makefile | 3 +-
src/PVE/HA/Manager.pm | 321 ++++++++----
src/PVE/HA/Resources.pm | 9 +
src/PVE/HA/Resources/PVECT.pm | 1 +
src/PVE/HA/Resources/PVEVM.pm | 1 +
src/PVE/HA/Rules.pm | 455 ++++++++++++++++++
src/PVE/HA/Rules/Makefile | 6 +
src/PVE/HA/Rules/NodeAffinity.pm | 296 ++++++++++++
src/PVE/HA/Sim/Env.pm | 48 +-
src/PVE/HA/Sim/Hardware.pm | 70 ++-
src/PVE/HA/Tools.pm | 46 ++
src/test/Makefile | 4 +-
.../defaults-for-node-affinity-rules.cfg | 22 +
...efaults-for-node-affinity-rules.cfg.expect | 60 +++
...e-resource-refs-in-node-affinity-rules.cfg | 31 ++
...rce-refs-in-node-affinity-rules.cfg.expect | 63 +++
src/test/test-basic5/groups | 2 -
src/test/test-basic5/rules_config | 3 +
src/test/test-basic5/service_config | 2 +-
src/test/test-crs-static2/README | 3 +-
src/test/test-crs-static2/groups | 2 -
src/test/test-crs-static2/rules_config | 3 +
src/test/test-crs-static2/service_config | 2 +-
src/test/test-group-migrate1/README | 4 +
src/test/test-group-migrate1/cmdlist | 3 +
src/test/test-group-migrate1/groups | 7 +
src/test/test-group-migrate1/hardware_status | 5 +
src/test/test-group-migrate1/log.expect | 101 ++++
src/test/test-group-migrate1/manager_status | 1 +
src/test/test-group-migrate1/service_config | 5 +
src/test/test-group-migrate2/README | 3 +
src/test/test-group-migrate2/cmdlist | 3 +
src/test/test-group-migrate2/groups | 7 +
src/test/test-group-migrate2/hardware_status | 5 +
src/test/test-group-migrate2/log.expect | 50 ++
src/test/test-group-migrate2/manager_status | 1 +
src/test/test-group-migrate2/service_config | 5 +
src/test/test-node-affinity-nonstrict1/README | 11 +
.../test-node-affinity-nonstrict1/cmdlist | 4 +
.../hardware_status | 5 +
.../test-node-affinity-nonstrict1/log.expect | 40 ++
.../manager_status | 1 +
.../rules_config | 3 +
.../service_config | 3 +
src/test/test-node-affinity-nonstrict2/README | 12 +
.../test-node-affinity-nonstrict2/cmdlist | 4 +
.../hardware_status | 5 +
.../test-node-affinity-nonstrict2/log.expect | 35 ++
.../manager_status | 1 +
.../rules_config | 3 +
.../service_config | 3 +
src/test/test-node-affinity-nonstrict3/README | 10 +
.../test-node-affinity-nonstrict3/cmdlist | 4 +
.../hardware_status | 5 +
.../test-node-affinity-nonstrict3/log.expect | 56 +++
.../manager_status | 1 +
.../rules_config | 3 +
.../service_config | 5 +
src/test/test-node-affinity-nonstrict4/README | 14 +
.../test-node-affinity-nonstrict4/cmdlist | 4 +
.../hardware_status | 5 +
.../test-node-affinity-nonstrict4/log.expect | 54 +++
.../manager_status | 1 +
.../rules_config | 3 +
.../service_config | 5 +
src/test/test-node-affinity-nonstrict5/README | 16 +
.../test-node-affinity-nonstrict5/cmdlist | 5 +
.../hardware_status | 5 +
.../test-node-affinity-nonstrict5/log.expect | 66 +++
.../manager_status | 1 +
.../rules_config | 3 +
.../service_config | 3 +
src/test/test-node-affinity-nonstrict6/README | 14 +
.../test-node-affinity-nonstrict6/cmdlist | 5 +
.../hardware_status | 5 +
.../test-node-affinity-nonstrict6/log.expect | 52 ++
.../manager_status | 1 +
.../rules_config | 3 +
.../service_config | 3 +
src/test/test-node-affinity-strict1/README | 11 +
src/test/test-node-affinity-strict1/cmdlist | 4 +
.../hardware_status | 5 +
.../test-node-affinity-strict1/log.expect | 40 ++
.../test-node-affinity-strict1/manager_status | 1 +
.../test-node-affinity-strict1/rules_config | 4 +
.../test-node-affinity-strict1/service_config | 3 +
src/test/test-node-affinity-strict2/README | 11 +
src/test/test-node-affinity-strict2/cmdlist | 4 +
.../hardware_status | 5 +
.../test-node-affinity-strict2/log.expect | 40 ++
.../test-node-affinity-strict2/manager_status | 1 +
.../test-node-affinity-strict2/rules_config | 4 +
.../test-node-affinity-strict2/service_config | 3 +
src/test/test-node-affinity-strict3/README | 10 +
src/test/test-node-affinity-strict3/cmdlist | 4 +
.../hardware_status | 5 +
.../test-node-affinity-strict3/log.expect | 74 +++
.../test-node-affinity-strict3/manager_status | 1 +
.../test-node-affinity-strict3/rules_config | 4 +
.../test-node-affinity-strict3/service_config | 5 +
src/test/test-node-affinity-strict4/README | 14 +
src/test/test-node-affinity-strict4/cmdlist | 4 +
.../hardware_status | 5 +
.../test-node-affinity-strict4/log.expect | 54 +++
.../test-node-affinity-strict4/manager_status | 1 +
.../test-node-affinity-strict4/rules_config | 4 +
.../test-node-affinity-strict4/service_config | 5 +
src/test/test-node-affinity-strict5/README | 16 +
src/test/test-node-affinity-strict5/cmdlist | 5 +
.../hardware_status | 5 +
.../test-node-affinity-strict5/log.expect | 66 +++
.../test-node-affinity-strict5/manager_status | 1 +
.../test-node-affinity-strict5/rules_config | 4 +
.../test-node-affinity-strict5/service_config | 3 +
src/test/test-node-affinity-strict6/README | 15 +
src/test/test-node-affinity-strict6/cmdlist | 5 +
.../hardware_status | 5 +
.../test-node-affinity-strict6/log.expect | 52 ++
.../test-node-affinity-strict6/manager_status | 1 +
.../test-node-affinity-strict6/rules_config | 4 +
.../test-node-affinity-strict6/service_config | 3 +
src/test/test-recovery1/groups | 4 -
src/test/test-recovery1/rules_config | 4 +
src/test/test-recovery1/service_config | 2 +-
src/test/test-recovery2/README | 4 +-
src/test/test-recovery2/groups | 4 -
src/test/test-recovery2/rules_config | 4 +
src/test/test-recovery2/service_config | 2 +-
src/test/test-recovery3/groups | 4 -
src/test/test-recovery3/rules_config | 4 +
src/test/test-recovery3/service_config | 2 +-
src/test/test-recovery4/groups | 4 -
src/test/test-recovery4/rules_config | 4 +
src/test/test-recovery4/service_config | 2 +-
.../test-relocate-policy-default-group/README | 6 +-
src/test/test-resource-failure2/groups | 2 -
src/test/test-resource-failure2/rules_config | 3 +
.../test-resource-failure2/service_config | 2 +-
.../test-resource-failure3/service_config | 2 +-
src/test/test-resource-failure6/README | 6 +-
src/test/test-shutdown2/groups | 2 -
src/test/test-shutdown2/rules_config | 3 +
src/test/test-shutdown2/service_config | 4 +-
src/test/test-shutdown3/groups | 2 -
src/test/test-shutdown3/rules_config | 3 +
src/test/test-shutdown3/service_config | 4 +-
src/test/test_failover1.pl | 27 +-
src/test/test_rules_config.pl | 100 ++++
160 files changed, 3374 insertions(+), 173 deletions(-)
create mode 100644 src/PVE/API2/HA/Rules.pm
create mode 100644 src/PVE/HA/Rules.pm
create mode 100644 src/PVE/HA/Rules/Makefile
create mode 100644 src/PVE/HA/Rules/NodeAffinity.pm
create mode 100644 src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg
create mode 100644 src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg.expect
create mode 100644 src/test/rules_cfgs/multiple-resource-refs-in-node-affinity-rules.cfg
create mode 100644 src/test/rules_cfgs/multiple-resource-refs-in-node-affinity-rules.cfg.expect
delete mode 100644 src/test/test-basic5/groups
create mode 100644 src/test/test-basic5/rules_config
delete mode 100644 src/test/test-crs-static2/groups
create mode 100644 src/test/test-crs-static2/rules_config
create mode 100644 src/test/test-group-migrate1/README
create mode 100644 src/test/test-group-migrate1/cmdlist
create mode 100644 src/test/test-group-migrate1/groups
create mode 100644 src/test/test-group-migrate1/hardware_status
create mode 100644 src/test/test-group-migrate1/log.expect
create mode 100644 src/test/test-group-migrate1/manager_status
create mode 100644 src/test/test-group-migrate1/service_config
create mode 100644 src/test/test-group-migrate2/README
create mode 100644 src/test/test-group-migrate2/cmdlist
create mode 100644 src/test/test-group-migrate2/groups
create mode 100644 src/test/test-group-migrate2/hardware_status
create mode 100644 src/test/test-group-migrate2/log.expect
create mode 100644 src/test/test-group-migrate2/manager_status
create mode 100644 src/test/test-group-migrate2/service_config
create mode 100644 src/test/test-node-affinity-nonstrict1/README
create mode 100644 src/test/test-node-affinity-nonstrict1/cmdlist
create mode 100644 src/test/test-node-affinity-nonstrict1/hardware_status
create mode 100644 src/test/test-node-affinity-nonstrict1/log.expect
create mode 100644 src/test/test-node-affinity-nonstrict1/manager_status
create mode 100644 src/test/test-node-affinity-nonstrict1/rules_config
create mode 100644 src/test/test-node-affinity-nonstrict1/service_config
create mode 100644 src/test/test-node-affinity-nonstrict2/README
create mode 100644 src/test/test-node-affinity-nonstrict2/cmdlist
create mode 100644 src/test/test-node-affinity-nonstrict2/hardware_status
create mode 100644 src/test/test-node-affinity-nonstrict2/log.expect
create mode 100644 src/test/test-node-affinity-nonstrict2/manager_status
create mode 100644 src/test/test-node-affinity-nonstrict2/rules_config
create mode 100644 src/test/test-node-affinity-nonstrict2/service_config
create mode 100644 src/test/test-node-affinity-nonstrict3/README
create mode 100644 src/test/test-node-affinity-nonstrict3/cmdlist
create mode 100644 src/test/test-node-affinity-nonstrict3/hardware_status
create mode 100644 src/test/test-node-affinity-nonstrict3/log.expect
create mode 100644 src/test/test-node-affinity-nonstrict3/manager_status
create mode 100644 src/test/test-node-affinity-nonstrict3/rules_config
create mode 100644 src/test/test-node-affinity-nonstrict3/service_config
create mode 100644 src/test/test-node-affinity-nonstrict4/README
create mode 100644 src/test/test-node-affinity-nonstrict4/cmdlist
create mode 100644 src/test/test-node-affinity-nonstrict4/hardware_status
create mode 100644 src/test/test-node-affinity-nonstrict4/log.expect
create mode 100644 src/test/test-node-affinity-nonstrict4/manager_status
create mode 100644 src/test/test-node-affinity-nonstrict4/rules_config
create mode 100644 src/test/test-node-affinity-nonstrict4/service_config
create mode 100644 src/test/test-node-affinity-nonstrict5/README
create mode 100644 src/test/test-node-affinity-nonstrict5/cmdlist
create mode 100644 src/test/test-node-affinity-nonstrict5/hardware_status
create mode 100644 src/test/test-node-affinity-nonstrict5/log.expect
create mode 100644 src/test/test-node-affinity-nonstrict5/manager_status
create mode 100644 src/test/test-node-affinity-nonstrict5/rules_config
create mode 100644 src/test/test-node-affinity-nonstrict5/service_config
create mode 100644 src/test/test-node-affinity-nonstrict6/README
create mode 100644 src/test/test-node-affinity-nonstrict6/cmdlist
create mode 100644 src/test/test-node-affinity-nonstrict6/hardware_status
create mode 100644 src/test/test-node-affinity-nonstrict6/log.expect
create mode 100644 src/test/test-node-affinity-nonstrict6/manager_status
create mode 100644 src/test/test-node-affinity-nonstrict6/rules_config
create mode 100644 src/test/test-node-affinity-nonstrict6/service_config
create mode 100644 src/test/test-node-affinity-strict1/README
create mode 100644 src/test/test-node-affinity-strict1/cmdlist
create mode 100644 src/test/test-node-affinity-strict1/hardware_status
create mode 100644 src/test/test-node-affinity-strict1/log.expect
create mode 100644 src/test/test-node-affinity-strict1/manager_status
create mode 100644 src/test/test-node-affinity-strict1/rules_config
create mode 100644 src/test/test-node-affinity-strict1/service_config
create mode 100644 src/test/test-node-affinity-strict2/README
create mode 100644 src/test/test-node-affinity-strict2/cmdlist
create mode 100644 src/test/test-node-affinity-strict2/hardware_status
create mode 100644 src/test/test-node-affinity-strict2/log.expect
create mode 100644 src/test/test-node-affinity-strict2/manager_status
create mode 100644 src/test/test-node-affinity-strict2/rules_config
create mode 100644 src/test/test-node-affinity-strict2/service_config
create mode 100644 src/test/test-node-affinity-strict3/README
create mode 100644 src/test/test-node-affinity-strict3/cmdlist
create mode 100644 src/test/test-node-affinity-strict3/hardware_status
create mode 100644 src/test/test-node-affinity-strict3/log.expect
create mode 100644 src/test/test-node-affinity-strict3/manager_status
create mode 100644 src/test/test-node-affinity-strict3/rules_config
create mode 100644 src/test/test-node-affinity-strict3/service_config
create mode 100644 src/test/test-node-affinity-strict4/README
create mode 100644 src/test/test-node-affinity-strict4/cmdlist
create mode 100644 src/test/test-node-affinity-strict4/hardware_status
create mode 100644 src/test/test-node-affinity-strict4/log.expect
create mode 100644 src/test/test-node-affinity-strict4/manager_status
create mode 100644 src/test/test-node-affinity-strict4/rules_config
create mode 100644 src/test/test-node-affinity-strict4/service_config
create mode 100644 src/test/test-node-affinity-strict5/README
create mode 100644 src/test/test-node-affinity-strict5/cmdlist
create mode 100644 src/test/test-node-affinity-strict5/hardware_status
create mode 100644 src/test/test-node-affinity-strict5/log.expect
create mode 100644 src/test/test-node-affinity-strict5/manager_status
create mode 100644 src/test/test-node-affinity-strict5/rules_config
create mode 100644 src/test/test-node-affinity-strict5/service_config
create mode 100644 src/test/test-node-affinity-strict6/README
create mode 100644 src/test/test-node-affinity-strict6/cmdlist
create mode 100644 src/test/test-node-affinity-strict6/hardware_status
create mode 100644 src/test/test-node-affinity-strict6/log.expect
create mode 100644 src/test/test-node-affinity-strict6/manager_status
create mode 100644 src/test/test-node-affinity-strict6/rules_config
create mode 100644 src/test/test-node-affinity-strict6/service_config
delete mode 100644 src/test/test-recovery1/groups
create mode 100644 src/test/test-recovery1/rules_config
delete mode 100644 src/test/test-recovery2/groups
create mode 100644 src/test/test-recovery2/rules_config
delete mode 100644 src/test/test-recovery3/groups
create mode 100644 src/test/test-recovery3/rules_config
delete mode 100644 src/test/test-recovery4/groups
create mode 100644 src/test/test-recovery4/rules_config
delete mode 100644 src/test/test-resource-failure2/groups
create mode 100644 src/test/test-resource-failure2/rules_config
delete mode 100644 src/test/test-shutdown2/groups
create mode 100644 src/test/test-shutdown2/rules_config
delete mode 100644 src/test/test-shutdown3/groups
create mode 100644 src/test/test-shutdown3/rules_config
create mode 100755 src/test/test_rules_config.pl
docs:
Daniel Kral (2):
ha: add documentation about ha rules and ha node affinity rules
ha: crs: add effects of ha node affinity rule on the crs scheduler
Makefile | 2 +
gen-ha-rules-node-affinity-opts.pl | 20 +++++
gen-ha-rules-opts.pl | 17 +++++
ha-manager.adoc | 113 +++++++++++++++++++++++++++++
ha-rules-node-affinity-opts.adoc | 18 +++++
ha-rules-opts.adoc | 12 +++
pmxcfs.adoc | 1 +
7 files changed, 183 insertions(+)
create mode 100755 gen-ha-rules-node-affinity-opts.pl
create mode 100755 gen-ha-rules-opts.pl
create mode 100644 ha-rules-node-affinity-opts.adoc
create mode 100644 ha-rules-opts.adoc
manager:
Daniel Kral (4):
api: ha: add ha rules api endpoints
ui: ha: remove ha groups from ha resource components
ui: ha: show failback flag in resources status view
ui: ha: replace ha groups with ha node affinity rules
PVE/API2/HAConfig.pm | 8 +-
www/manager6/Makefile | 8 +-
www/manager6/StateProvider.js | 2 +-
www/manager6/dc/Config.js | 8 +-
www/manager6/ha/GroupSelector.js | 71 -------
www/manager6/ha/Groups.js | 117 -----------
www/manager6/ha/ResourceEdit.js | 16 +-
www/manager6/ha/Resources.js | 17 +-
www/manager6/ha/RuleEdit.js | 146 +++++++++++++
www/manager6/ha/RuleErrorsModal.js | 50 +++++
www/manager6/ha/Rules.js | 196 ++++++++++++++++++
www/manager6/ha/StatusView.js | 5 +-
.../NodeAffinityRuleEdit.js} | 105 ++--------
www/manager6/ha/rules/NodeAffinityRules.js | 36 ++++
14 files changed, 481 insertions(+), 304 deletions(-)
delete mode 100644 www/manager6/ha/GroupSelector.js
delete mode 100644 www/manager6/ha/Groups.js
create mode 100644 www/manager6/ha/RuleEdit.js
create mode 100644 www/manager6/ha/RuleErrorsModal.js
create mode 100644 www/manager6/ha/Rules.js
rename www/manager6/ha/{GroupEdit.js => rules/NodeAffinityRuleEdit.js} (67%)
create mode 100644 www/manager6/ha/rules/NodeAffinityRules.js
Summary over all repositories:
181 files changed, 4038 insertions(+), 477 deletions(-)
--
Generated by git-murpp 0.8.0
More information about the pve-devel
mailing list