Locking HA during UPS shutdown
Stefan Radman
stefan.radman at me.com
Thu Mar 10 12:10:09 CET 2022
Hi
I am configuring a 3 node PVE cluster with integrated Ceph storage.
It is powered by 2 UPS that are monitored by NUT (Network UPS Tools).
HA is configured with 3 groups:
group pve1 nodes pve1:1,pve2,pve3
group pve2 nodes pve1,pve2:1,pve3
group pve3 nodes pve1,pve2,pve3:1
That will normally place the VMs in each group on the corresponding node, unless that node fails.
The cluster is configured to migrate VMs away from a node before shutting it down (Cluster=>Options=>HA Settings: shutdown_policy=migrate).
NUT is configured to shut down the serves once the last of the two UPS is running low on battery.
My problem:
When NUT starts shutting down the 3 nodes, HA will first try to live-migrate them to another node.
That live migration process gets stuck because all the nodes are shutting down simultaneously.
It seems that the whole process runs into a timeout, finally “powers off” all the VMs and shuts down the nodes.
My question:
Is there a way to “lock” or temporarily de-activate HA before shutting down a node to avoid that deadlock?
Thank you
Stefan
More information about the pve-user
mailing list