[PVE-User] BIG cluster questions
elacunza at binovo.es
Thu Jul 1 08:55:16 CEST 2021
El 26/6/21 a las 13:16, aderumier at odiso.com escribió:
> Le jeudi 24 juin 2021 à 16:30 +0200, Eneko Lacunza via pve-user a écrit :
>> Now for more general questions; do you think a 88-node Proxmox VE
>> cluster is feasible?
> Well, corosync is not really done to this amount of node. (I think the
> hardcoded limit is around 100),
> but in practice, I have seen a lot of users having problem with
> corosync communication starting around 30 nodes
> (Maybe with low-latency switches + fast frequencies cpu, it's possible
> to keep latency enough low to get it working)
Thanks for this info. Documentation isn't helpful when it says "There’s
no explicit limit for the number of nodes in a cluster. In practice, the
actual possible node count may be limited by the host and network
performance. Currently (2021), there are reports of clusters (using
high-end enterprise hardware) with over 50 nodes in production." in
https://pve.proxmox.com/wiki/Cluster_Manager (it's the same wording in
>> Those 88 nodes will host about 14.000 VMs. Will HA manager be able to
>> manage them, or are they too many? (HA for those VMs doesn't seem to be
>> a requirement right know).
> I have cluster with 2500vms, it's working fine. (on a 15 nodes cluster
> with around 200vms on each node).
> I don't known with 15000vms, maybe the main pve-crm loop will take
> more time, I'm not sure about the timeouts.
> At work, I'm doing nodes 20 nodes cluster (1 by rack) to avoid to have
> big cluster.
> Multi-cluster central management is not yet on the roadmap, but they
> are some tool like
> https://maas.io/ <https://maas.io/>
> which allow to manage multi-cluster in a central way.
Thanks for this data. In 20 nodes cluster, can you share whether cluster
network is physically different from VM traffic/storage, and it's speed?
Zuzendari teknikoa | Director técnico
Binovo IT Human Project
Tel. +34 943 569 206 | https://www.binovo.es
Astigarragako Bidea, 2 - 2º izda. Oficina 10-11, 20180 Oiartzun
More information about the pve-user