[pve-devel] [PATCH ha-manager 0/4] Add inital HW based fencing

[Fabian Grünbichler]

On March 27, 2019 5:42 pm, Thomas Lamprecht wrote:
> Actually this'd be v8, but it's to old and to unreviewed for anybody do
> remember anyway, so best seen as new series.
> 
> Thomas Lamprecht (4):
>   allow LRM lock stealing for fenced nodes
>   allow use of external fencing devices
>   send also email on hardware fence failure
>   add some infos about HW fencing to README

first off, sorry for the long delay..

low-level nits and questions as replies to individual patches, some 
stuff I noticed while testing, mostly regarding the already merged 
Fence/FenceConfig code:

1) missing 'use FenceConfig;' in PVE::HA::Config.pm

2) as discussed off-list, it would be great if the fork could be moved 
into exec_fence_agent to get rid of the test/ha env distinction in 
Fence.pm (possibly as follow-up, since it does not change the 
semantics/functionality)

3) the simulator does not support fencing_mode 'both' (since it just 
decides beside on whether a fence config exists)

4) I wonder whether there is a use-case for an actual 'both' mode (since 
IMHO the current 'both' mode is more like an 'either' mode, we are just 
waiting for hardware or regular fencing to go through, not both ;)):

something like a hardware fence device that is optional and helpful, but 
alone is not enough to ensure that the node is actually fully fenced.

e.g., fence the network of the node via switch (to prevent further 
client access), but we still need to wait for the regular fencing to go 
through since all nodes have some non-network shared storage,
so the not-yet-stopped services on the fenced node can still write to 
the shared disks until the node is fully fenced.

or some other hacky use cases, like marking OSDs as down & out via 
fencing, but that is more abuse case than use case ;)

possible this could also be done via fence.cfg, by marking certain 
devices as 'not enough to consider fenced'? obviously with fencing_mode 
hardware you'd need at least one proper device ;)

5) the config parser/writer pair is broken:
- writer: does not re-quote the node args
- parser: both [args] are not actually optional (like the README says)

(the writer is not yet used anywhere)

$ cat /etc/pve/ha/fence.cfg

device pve_nina:1 fence_pve 'ip=192.168.15.38' 'username=fäöµ€' 'password=12"345'
connect pve_nina:1 node=node1 plug=500 x
connect pve_nina:1 node=node2 'username=fäöµ€' 'password=12"345'

$ perl -e  'use strict; use warnings; use Data::Dumper; use PVE::Cluster; use PVE::HA::FenceConfig; use PVE::HA::Config; PVE::Cluster::cfs_update(); my $cfg = PVE::HA::Config::read_fence_config(); print Dumper($cfg), "\n"; PVE::HA::Config::write_fence_config($cfg);'
$VAR1 = {
  'pve_nina' => {
    'priority' => 0,
    'sub_devs' => {
      '1' => {
        'node_args' => {
          'node2' => [
            'username=fäöµ€',
            'password=12"345'
          ],
          'node1' => [
            'plug=500',
            'x'
          ]
        },
        'args' => [
          'ip=192.168.15.38',
          'username=fäöµ€',
          'password=12"345'
        ],
        'agent' => 'fence_pve'
      }
    }
  }
};

quotes from node arguments were dropped by the writer:

$ cat /etc/pve/ha/fence.cfg

device pve_nina:1 fence_pve 'ip=192.168.15.38' 'username=fäöµ€' 'password=12"345'
connect pve_nina:1 node=node1 plug=500 x
connect pve_nina:1 node=node2 username=fäöµ€ password=12"345

$ perl -e  'use strict; use warnings; use Data::Dumper; use PVE::Cluster; use PVE::HA::FenceConfig; use PVE::HA::Config; PVE::Cluster::cfs_update(); my $cfg = PVE::HA::Config::read_fence_config(); print Dumper($cfg), "\n"; PVE::HA::Config::write_fence_config($cfg);'
$VAR1 = {
  'pve_nina' => {
    'sub_devs' => {
      '1' => {
        'agent' => 'fence_pve',
        'node_args' => {
          'node2' => [],
          'node1' => [
            'plug=500',
            'x'
          ]
        },
        'args' => [
          'ip=192.168.15.38',
          'username=fäöµ€',
          'password=12"345'
        ]
      }
    },
    'priority' => 0
  }
};

which makes the parser fail since the input is not properly quoted, 
dropping the args altogether!?

$ cat /etc/pve/ha/fence.cfg

device pve_nina:1 fence_pve 'ip=192.168.15.38' 'username=fäöµ€' 'password=12"345'
connect pve_nina:1 node=node1 plug=500 x
connect pve_nina:1 node=node2 

$ perl -e  'use strict; use warnings; use Data::Dumper; use PVE::Cluster; use PVE::HA::FenceConfig; use PVE::HA::Config; PVE::Cluster::cfs_update(); my $cfg = PVE::HA::Config::read_fence_config(); print Dumper($cfg), "\n"; PVE::HA::Config::write_fence_config($cfg);'
/etc/pve/ha/fence.cfg ignore line 4: connect pve_nina:1 node=node2
$VAR1 = {
  'pve_nina' => {
    'priority' => 0,
    'sub_devs' => {
      '1' => {
        'args' => [
          'ip=192.168.15.38',
          'username=fäöµ€',
          'password=12"345'
        ],
        'node_args' => {
          'node1' => [
            'plug=500',
            'x'
          ]
        },
        'agent' => 'fence_pve'
      }
    }
  }
};

and finally, node2 gets dropped altogether since it does not have any 
arguments and the parser fails to parse its line

$ cat /etc/pve/ha/fence.cfg

device pve_nina:1 fence_pve 'ip=192.168.15.38' 'username=fäöµ€' 'password=12"345'
connect pve_nina:1 node=node1 plug=500 x