[pve-devel] [PATCH ha-manager 3/4] send also email on hardware fence failure

[Fabian Grünbichler]

On March 27, 2019 5:42 pm, Thomas Lamprecht wrote:
> We introduced sending an email when Fencing fails/succeeds in
> general some time ago, send now also an email if a HW fence
> fails

missing S-O-B

> ---
>  src/PVE/HA/NodeStatus.pm | 7 ++++---
>  1 file changed, 4 insertions(+), 3 deletions(-)
> 
> diff --git a/src/PVE/HA/NodeStatus.pm b/src/PVE/HA/NodeStatus.pm
> index ca13f2f..c4d147c 100644
> --- a/src/PVE/HA/NodeStatus.pm
> +++ b/src/PVE/HA/NodeStatus.pm
> @@ -2,6 +2,7 @@ package PVE::HA::NodeStatus;
>  
>  use strict;
>  use warnings;
> +
>  use PVE::HA::Fence;
>  
>  use JSON;
> @@ -204,7 +205,6 @@ EOF
>      $haenv->sendmail($mail_subject, $mail_text);
>  };
>  
> -
>  # start fencing
>  sub fence_node {
>      my ($self, $node) = @_;
> @@ -230,8 +230,9 @@ sub fence_node {
>  
>  	# bad fence.cfg or no devices and only hardware fencing configured
>  	if ($hw_fence_success < 0 && $fencing_mode eq 'hardware') {
> -	    $haenv->log('err', "Fencing of node '$node' failed and needs " .
> -			"manual intervention!");
> +	    my $msg = "Fencing of node '$node' failed and needs manual intervention!";
> +	    $haenv->log('err', $msg);
> +	    &$send_fence_state_email($self, 'FAILED', $msg, $node);

seems like a good idea in general, but see patch #2 - this part gets 
called over and over again if hardware fencing fails? e.g.,

./ha-tester.pl test-hw-fence1 --nodiff | grep emai
emai    160    node1/crm: FENCE: Try to fence node 'node3'
emai    160    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    160    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    180    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    200    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    220    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    240    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    260    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    280    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    300    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    320    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    340    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    360    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    380    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    400    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    420    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    440    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    460    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    480    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    500    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    520    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    540    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    560    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    580    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    600    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    620    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    640    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    660    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    680    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!
emai    700    node1/crm: FAILED: Fencing of node 'node3' failed and needs manual intervention!

if I hardcode $hw_success to -1

or is this just an artifact of some simulator limitation?

>  	    return 0;
>  	}
>  
> -- 
> 2.20.1
> 
> 
> _______________________________________________
> pve-devel mailing list
> pve-devel at pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>