[pve-devel] [PATCH ha-manager 8/9] manager: make online node usage computation granular

[Daniel Kral]

The HA Manager builds $online_node_usage in every FSM iteration in
manage(...) and at every HA resource state change in
change_service_state(...). This becomes quite costly with a high HA
resource count and a lot of state changes happening at once, e.g.
starting up multiple nodes with rebalance_on_request_start set or a
failover of a node with many configured HA resources.

To improve this situation, make the changes to the $online_node_usage
more granular by building $online_node_usage only once per call to
manage(...) and changing the nodes a HA resource uses individually on
every HA resource state transition.

The change in service usage "freshness" should be negligible here as the
static service usage data is cached anyway (except if the cache fails
for some reason).

Signed-off-by: Daniel Kral <d.kral at proxmox.com>
---
The add_service_usage(...) helper is added in anticipation for the next
patch, we don't need a helper if we don't go for #9.

 src/PVE/HA/Manager.pm | 22 +++++++++++++++++++---
 1 file changed, 19 insertions(+), 3 deletions(-)

diff --git a/src/PVE/HA/Manager.pm b/src/PVE/HA/Manager.pm
index d0d4d0a5..253deba9 100644
--- a/src/PVE/HA/Manager.pm
+++ b/src/PVE/HA/Manager.pm
@@ -77,6 +77,21 @@ sub new {
     return $self;
 }
 
+sub add_service_usage {
+    my ($self, $sid, $sd) = @_;
+
+    my $online_node_usage = $self->{online_node_usage};
+
+    my $online_nodes = { map { $_ => 1 } $online_node_usage->list_nodes() };
+    my $nodes = PVE::HA::Tools::get_used_service_nodes($sd, $online_nodes);
+
+    my ($current, $target) = $nodes->@{qw(current target)};
+    $online_node_usage->add_service_usage_to_node($current, $sid, $sd->{node}, $sd->{target})
+        if $current;
+    $online_node_usage->add_service_usage_to_node($target, $sid, $sd->{node}, $sd->{target})
+        if $target;
+}
+
 sub update_crs_scheduler_mode {
     my ($self) = @_;
 
@@ -314,7 +329,8 @@ my $change_service_state = sub {
         $sd->{$k} = $v;
     }
 
-    $self->recompute_online_node_usage();
+    $self->{online_node_usage}->remove_service_usage($sid);
+    $self->add_service_usage($sid, $sd);
 
     $sd->{uid} = compute_new_uuid($new_state);
 
@@ -706,6 +722,8 @@ sub manage {
         delete $ss->{$sid};
     }
 
+    $self->recompute_online_node_usage();
+
     my $new_rules = $haenv->read_rules_config();
 
     # TODO PVE 10: Remove group migration when HA groups have been fully migrated to rules
@@ -735,8 +753,6 @@ sub manage {
     for (;;) {
         my $repeat = 0;
 
-        $self->recompute_online_node_usage();
-
         foreach my $sid (sort keys %$ss) {
             my $sd = $ss->{$sid};
             my $cd = $sc->{$sid} || { state => 'disabled' };
-- 
2.47.3