[PVE-User] Changing votes and quorum

[Dewangga Alam]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hello!

On 30/10/18 15.48, Thomas Lamprecht wrote:
> Hi!
> 
> Am 10/29/2018 um 05:36 PM schrieb Dewangga Alam:
>> On 29/10/18 16.14, Thomas Lamprecht wrote:
>>> Am 10/28/2018 um 02:54 PM schrieb Dewangga Alam: Hello!
>>> 
>>> I was new in proxmox and am trying to build large scale
>>> proxmox 5.2 cluster (>128 nodes). My `/etc/pve/corosync.conf`
>>> configuration like :
>>> 
>>> ``` nodelist { node { name: node1 nodeid: 1 quorum_votes: 1 
>>> ring0_addr: 192.168.24.2 } ... skip till nodes 28 ... node {
>>> name: node28 nodeid: 28 quorum_votes: 1 ring0_addr:
>>> 192.168.24.110 } } quorum { provider: corosync_votequorum
>>> expected_votes: 16
>>> 
>>>> expected_votes must be your real highest expected votes.
>> 
>> [..]
>>> 
>>> last_man_standing: 1 last_man_standing_window: 20000
>>> 
>>> 
>>> 
>>> } totem { cluster_name: px-cluster1 config_version: 90
>>> window_size: 300 interface { ringnumber: 0 } ip_version: ipv4
>>> secauth: on transport: udpu version: 2 } ```
>>> 
>>> My cluster have 28 nodes in each rack. and total nodes will 28 
>>> nodes*5 racks. So it will 140 nodes in a cluster. From the 
>>> adjustment above, I wonder there's affected to
>>> pve/corosync.conf configuration, but in fact, it didn't.
>>> 
>>> So my basic question, when I invoke `pvecm status` in a node,
>>> the result wasn't as I expect. Then, is it possible to change 
>>> votequorum configuration?
>>> 
>>>> What wasn't as expected? That your set expected_votes is not
>>>>  "accepted" by corosync? That is expected behaviour.
>>> 
>>>> What is the real problem you want to solve?
>> 
>> I want to build > 32 nodes in one cluster,
> 
> cool!

I am really scary right now. LoL.

> 
>> and I am expect that expected_votes can be controlled lower than
>> real votes. I thought, it should be make a quorum if the 50%+1
>> formulas aren't met.
> 
> No, that isn't needed. Highest expected can not be smaller then
> the actual online quorate nodes. That's like saying you have a
> election in your small country with 10 people (and thus 10 expected
> votes) but you receive (as example) 16 votes - something is fishy.
> Corosync here just thinks that the census (in this case you) was
> wrong and uses the higher number.
> 
> Although, if you set expected votes, but have a higher node count 
> but equal or less than your manual set expected_votes count are 
> online/quorate then you will also see that it stays at your set
> number and you actually can have quorum with "less" nodes. But as
> soon as more nodes get online corosync expected vote number will
> rise.
> 
> When enabling last_man_standing you can achieve that the highest 
> expecting also scales down, if some nodes go down (or have another 
> reason to lose cluster communication) but there are enough nodes
> left to have a working, quorate, cluster then after a specified
> time window - when all stays working - corosync recalculates it's
> expected votes and you can then loose additional nodes.

OTOH, with last_man_standing it should be safer than wait_for_all,
isn't it? (eg. If I got real nodes online 56 nodes in a cluster, then
30 nodes are down, it will trigger expected_votes and re-calculate
automatically, right?)

If it isn't, is there any best practice to adjust corosync.conf in
large scale deployment?

> 
> Hope that helps a bit.
> 
> cheers, Thomas
> 
>> 
>> Is it a best practice?
>> 
> 
> 
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEZpxiw/Jg6pEte5xQ5X/SIKAozXAFAlvYJOQACgkQ5X/SIKAo
zXADzQ/9FdNCA0OrqXWSDPe3LWWSnmETiABxYYZR7N3WXLenMXBd3CiK5G7ncAxO
wIJZL0Jq+TTxL720eyQWM7coKp73BJi2elJPt7iJpUfProU9J6hpd1g0aEEBLy4h
vxJZpCLAtrIYINuwxSgvVRvLuyOV1IgyE4M5SCyoL21OIuo07vNt3u1J3JNCCBbE
o+NlumXlDfAzt8XKkdICrFGXxZL/kQT2jwsxWxKoh1Bz+TGQH332dqINiqRkdhOB
tMJ59tWjSLjIo/M+BDqCGMzbQM3PWzXdBg6JSS7cPHD/nuS2uuEoI2KHofKofNh1
vo/Ip6HR46kLy77ZDVD2N8W9wsotAS1vWsDmveJ7qAQC8OjAsqjlyhoSv9N4eI3a
CZFdjkzOvsAhRXgM6izjUDNEyKJtYpnQBvmnlcJFMeoLxNQro68eo3KP/vc5QuSS
f4KgmdXHjerB/MgNcD1BeUytf+ornUofYWOK7UqkCBF/Lpk5K9lT4roWoqUz/np3
1gaxKyV0q6+tCorFdvaae5mYvk77m/n3MItMIs2djj60/D3nSDD4Oq6N9D2TC7Tl
lRzPoXJiuiarUv+xT3Lm/8KGTin05ZFNv4ZZkKNN6HlsohxVyLJ3q3xMf9J5vM1j
nu3W64zf6gurRmEYIEig+Z88Y/6mBMniFZhKiCoHW07SqQcnNWo=
=YehX
-----END PGP SIGNATURE-----