[PVE-User] Changing votes and quorum

[Thomas Lamprecht]

Hi!

Am 10/29/2018 um 05:36 PM schrieb Dewangga Alam:
> On 29/10/18 16.14, Thomas Lamprecht wrote:
>> Am 10/28/2018 um 02:54 PM schrieb Dewangga Alam: Hello!
>>
>> I was new in proxmox and am trying to build large scale proxmox
>> 5.2 cluster (>128 nodes). My `/etc/pve/corosync.conf` configuration
>> like :
>>
>> ``` nodelist { node { name: node1 nodeid: 1 quorum_votes: 1 
>> ring0_addr: 192.168.24.2 } ... skip till nodes 28 ... node { name:
>> node28 nodeid: 28 quorum_votes: 1 ring0_addr: 192.168.24.110 } } 
>> quorum { provider: corosync_votequorum expected_votes: 16
>>
>>> expected_votes must be your real highest expected votes.
> 
> [..]
>>
>> last_man_standing: 1 last_man_standing_window: 20000
>>
>>
>>
>> } totem { cluster_name: px-cluster1 config_version: 90 window_size:
>> 300 interface { ringnumber: 0 } ip_version: ipv4 secauth: on 
>> transport: udpu version: 2 } ```
>>
>> My cluster have 28 nodes in each rack. and total nodes will 28
>> nodes*5 racks. So it will 140 nodes in a cluster. From the
>> adjustment above, I wonder there's affected to pve/corosync.conf
>> configuration, but in fact, it didn't.
>>
>> So my basic question, when I invoke `pvecm status` in a node, the 
>> result wasn't as I expect. Then, is it possible to change
>> votequorum configuration?
>>
>>> What wasn't as expected? That your set expected_votes is not 
>>> "accepted" by corosync? That is expected behaviour.
>>
>>> What is the real problem you want to solve?
> 
> I want to build > 32 nodes in one cluster,

cool!

> and I am expect that
> expected_votes can be controlled lower than real votes. I thought, it
> should be make a quorum if the 50%+1 formulas aren't met.

No, that isn't needed. Highest expected can not be smaller then the
actual online quorate nodes. That's like saying you have a election
in your small country with 10 people (and thus 10 expected votes) but
you receive (as example) 16 votes - something is fishy. Corosync here
just thinks that the census (in this case you) was wrong and uses the
higher number.

Although, if you set expected votes, but have a higher node count
but equal or less than your manual set expected_votes count are
online/quorate then you will also see that it stays at your set number
and you actually can have quorum with "less" nodes. But as soon as more
nodes get online corosync expected vote number will rise.

When enabling last_man_standing you can achieve that the highest
expecting also scales down, if some nodes go down (or have another
reason to lose cluster communication) but there are enough nodes left
to have a working, quorate, cluster then after a specified time window
- when all stays working - corosync recalculates it's expected votes
and you can then loose additional nodes.

Hope that helps a bit.

cheers,
Thomas

> 
> Is it a best practice?
>