[PVE-User] BIG cluster questions

[Eneko Lacunza]

Hi JR,

El 26/6/21 a las 14:59, JR Richardson escribió:
> That is a big cluster, I like it, hope it works out. You should separate the
> corosync/heartbeat network on its own physical Ethernet link. This is
> probably where you are getting latency from. Even though you are using 25Gig
> NICs, pushing all your data/migration traffic/heartbeat traffic, across one
> physical link bonded or not, you can experience situations with a busy link
> where your corosync traffic is queued, even for a few milli seconds, this
> will add up across many nodes. Think about jumbo frames as well, slamming a
> NIC with 9000 byte packets for storage, and poor little heartbeat packets
> start queueing up in the waiting pool.
>
> In the design notes for proxmox, it's highly recommended to separate all
> needed networks on physical NICs and switches as well.

Yes I'm aware of that, but we don't have VM traffic yet on this cluster, 
we're testing it. Also, I was involved on the project too late, when 
node hardware was already purchased ;)

Thanks

>
> Good luck.
>
> JR Richardson
> Engineering for the Masses
> Chasing the Azeotrope
> JRx DistillCo
> 1'st Place Brisket
> 1'st Place Chili
>
> This is anecdotal but I have never seen one cluster that big. You might want
> to inquire about professional support which would give you a better
> perspective for that kind of scale.
>
> On Thu, Jun 24, 2021 at 10:30 AM Eneko Lacunza via pve-user <
> pve-user at lists.proxmox.com> wrote:
>
>>
>>
>> ---------- Forwarded message ----------
>> From: Eneko Lacunza <elacunza at binovo.es>
>> To: "pve-user at pve.proxmox.com" <pve-user at pve.proxmox.com>
>> Cc:
>> Bcc:
>> Date: Thu, 24 Jun 2021 16:30:31 +0200
>> Subject: BIG cluster questions
>> Hi all,
>>
>> We're currently helping a customer to configure a virtualization
>> cluster with 88 servers for VDI.
>>
>
>
> _______________________________________________
> pve-user mailing list
> pve-user at lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>

Eneko Lacunza
Zuzendari teknikoa | Director técnico
Binovo IT Human Project

Tel. +34 943 569 206 | https://www.binovo.es
Astigarragako Bidea, 2 - 2º izda. Oficina 10-11, 20180 Oiartzun

https://www.youtube.com/user/CANALBINOVO
https://www.linkedin.com/company/37269706/