[PVE-User] Shared storage recommendations

[Alexandre DERUMIER]

>>Not sure what documentation exist. 40gbps links are internally 4x 10gbps 
>>waves so a single io will flow on one of the 10gbps links at 10gbps 
>>latency.  a 25 Gbps link is a faster single wave, and 100Gbps is 
>>4x25Gbps waves.  since IOPS is usually much more important then 
>>bandwidth (for VM's) you may get more iops with 25Gbps then with 
>>40Gbps   basically it depends on the workload, if the load is mostly 
>>bandwidth you might need the thru put of 40Gbps more.

Yes, that's true. 

But you have also the latency from the cpu (as ceph is cpu intensive),
so try to have fast frequencies cpus first (both client side, and cluster nodes side)


----- Mail original -----
De: "Ronny Aasen" <ronny+pve-user at aasen.cx>
À: "proxmoxve" <pve-user at pve.proxmox.com>
Envoyé: Mercredi 27 Février 2019 19:03:58
Objet: Re: [PVE-User] Shared storage recommendations

On 27.02.2019 16:49, Frederic Van Espen wrote: 
> Hi, 
> 
>> tips: you want 5 monitors, so you can have dual failures or one fail 
>> while another is under maintenance/reboot; Good networking is 
>> important, 25gbps is better then 40gbps due to lower latency; 
> Thanks for these tips! Do you have any reference documentation for the 
> 25gbps vs 40gbps statement? I find no immediate results for that on 
> google. In any case, we were thinking of starting out with a 10gbps 
> network. Are there any solid reasons not to do that for a light 
> workload? 

Not sure what documentation exist. 40gbps links are internally 4x 10gbps 
waves so a single io will flow on one of the 10gbps links at 10gbps 
latency. a 25 Gbps link is a faster single wave, and 100Gbps is 
4x25Gbps waves. since IOPS is usually much more important then 
bandwidth (for VM's) you may get more iops with 25Gbps then with 
40Gbps basically it depends on the workload, if the load is mostly 
bandwidth you might need the thru put of 40Gbps more. 

quote from 
https://ceph.com/planet/making-ceph-faster-lessons-from-performance-testing/ 
"20 HDDs or 2-3 SSDs can exceed the bandwidth of single 10GbE link for 
read throughput. " 
so if your nodes are on this size, a single 10gbps will be the 
bottleneck. but if the load is below the bottleneck it is not a problem. 

even if you design network with 2x10gbps links for HA you may need need 
to run the cluster on a single 10gbps in case of a failure on part of 
the network. workload and SLA's you provide to your clients determine if 
that is acceptable or not. monitoring is important 


good luck! 

Ronny Aasen 




_______________________________________________ 
pve-user mailing list 
pve-user at pve.proxmox.com 
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user