[pve-devel] has somebody already tested corosync3 alpha et new knet transport ?

[Alexandre DERUMIER]

> do you see so much traffic on your proxmox test clusters ? 

>>We need to compare multicast vs unicast ... 


yes sure, it's 1mbit/s vs 50kb/s. but if I can run 200 nodes with 10mbits, I really don't care ;).
It's more about latency

>>Multicast is known to work with 32 nodes (some reports exists with up to 64 
>>nodes), 
>>and some users even want bigger clusters. So any such test/benchmark 
>>should use up to 32 nodes, maybe more ... 

Seem that they have begin to test with bigger clusters

https://docs.google.com/document/d/1Hb0sJVVGcnlc_ry2SJnIy22ct34QVAXz9I3AmetBNcY/edit#heading=h.99ommvng5zql

"24th Dec
Executed the first batch of tests with 9 nodes.
Cpghum tests confirm that knet is still much faster (much higher bw and much lower latency) than udpu.


25th Dec
Fixed knet_bench template generation
Started testing with 16 nodes over 2 networks (still applying several workaround for PMTUd)
16 nodes testing + hacks still shows knet being a lot faster than UDPU
Testing 32 nodes (so far membership join/keep is stable)

"

So, it seem to be on the right way. 


----- Mail original -----
De: "dietmar" <dietmar at proxmox.com>
À: "aderumier" <aderumier at odiso.com>
Cc: "Thomas Lamprecht" <t.lamprecht at proxmox.com>, "pve-devel" <pve-devel at pve.proxmox.com>
Envoyé: Mercredi 27 Juin 2018 10:02:52
Objet: Re: [pve-devel] has somebody already tested corosync3 alpha et new knet transport ?

> >>for such small clusters, but it is essential if you want bigger clusters - 
> >>else 
> >>you will 
> >>produce too much traffic on the net. 
> 
> do you see so much traffic on your proxmox test clusters ? 

We need to compare multicast vs unicast ... 

> Currently, on my 20 proxmox nodes clusters, with unicast, I'm around 1mbit/s 
> rx/tx on each node. 
> 
> 
> I would like to see latency benchmark with knet and more nodes. 

Multicast is known to work with 32 nodes (some reports exists with up to 64 
nodes), 
and some users even want bigger clusters. So any such test/benchmark 
should use up to 32 nodes, maybe more ...