[pve-devel] less cores more iops / speed

[Alexandre DERUMIER]

Hi Eric,

>>I have been testing IOPS with Areca 1882 pcie 3.0 with SSD disks. 
>>A single VM on my system seems to be limited to about 30K IOPS writing 
>>and about 70K reading. 
>>I can get more total IOPS running benchmarks on two VMs at the same time. 

Great Results ! 
What benchmark do you use ? fio ? can you post the bench command line to compare ?

Also what is your cluster hardware (cpu,ram,disk model, how many nodes, switchs models ?).

Your results seem to be a lot higher than stephan. (This is the best results I ever see with rbd ;)




>>Instructions here: 
>>http://forum.proxmox.com/threads/11508-Intel-Turbo-Boost-not-working?p=63058#post63058 

I agree with this settings, I always set this options in bios.


Also maybe irqbalance daemon can help:

#apt-get install irqbalance



----- Mail original ----- 

De: "Eric Blevins" <eric at netwalk.com> 
À: pve-devel at pve.proxmox.com 
Envoyé: Vendredi 9 Novembre 2012 21:53:21 
Objet: Re: [pve-devel] less cores more iops / speed 

I have been testing IOPS with Areca 1882 pcie 3.0 with SSD disks. 
A single VM on my system seems to be limited to about 30K IOPS writing 
and about 70K reading. 
I can get more total IOPS running benchmarks on two VMs at the same time. 

Xeon E5-2650 2.0Ghz 8 core 

These newer Xeon with turboboost make it a little difficult to benchmark 
if frequencies change all the time. 
With a few teaks I got all the cores in my 2650s to run at 2.4Ghz all 
the time. 
Instructions here: 
http://forum.proxmox.com/threads/11508-Intel-Turbo-Boost-not-working?p=63058#post63058 




On 11/09/2012 11:02 AM, Alexandre DERUMIER wrote: 
> Oh great, 
> so it seem that you can get more ios with more vm. (It's become interesting for me :) 
> 
> What are you exact xeon models ? same server model,bios options ? 
> 
> what happen on the dual xeon, if you shutdown some cores ? 
> 
> echo 0 > /sys/devices/system/cpu/cpuX/online 
> 
> 
> also, does pining help on dual xeon ? 
> 
> 
> 
> ----- Mail original ----- 
> 
> De: "Stefan Priebe - Profihost AG" <s.priebe at profihost.ag> 
> À: "Alexandre DERUMIER" <aderumier at odiso.com> 
> Cc: pve-devel at pve.proxmox.com 
> Envoyé: Vendredi 9 Novembre 2012 16:42:09 
> Objet: Re: [pve-devel] less cores more iops / speed 
> 
> Am 09.11.2012 15:49, schrieb Alexandre DERUMIER: 
>> Maybe one last question (sorry ;) 
>> 
>> Do you have tried to launch 2 fio test from 2 differents kvm host at the same time ? 
>> 
>> If yes, do you have more io or do you have half of speed on each fio test ? 
> Only one VM active on a Single Xeon 3.6Ghz: 
> write: io=6024MB, bw=68519KB/s, iops=17129, runt= 90025msec 
> read : io=6057MB, bw=68891KB/s, iops=17222, runt= 90026msec 
> write: io=71540MB, bw=808308KB/s, iops=197, runt= 90630msec 
> read : io=147096MB, bw=1631MB/s, iops=407, runt= 90162msec 
> 
> 
> Only one VM active on a Dual Xeon 2.5Ghz: 
> write: io=2224MB, bw=25285KB/s, iops=6321, runt= 90070msec 
> read : io=2033MB, bw=23108KB/s, iops=5777, runt= 90078msec 
> write: io=50900MB, bw=574235KB/s, iops=140, runt= 90767msec 
> read : io=84228MB, bw=954499KB/s, iops=233, runt= 90361msec 
> 
> 
> Both VMs active (doing each test in parallel): 
> 
> VM 1: 
> write: io=5376MB, bw=61112KB/s, iops=15277, runt= 90086msec 
> read : io=6092MB, bw=69292KB/s, iops=17323, runt= 90028msec 
> write: io=67724MB, bw=766012KB/s, iops=187, runt= 90533msec 
> read : io=160200MB, bw=1776MB/s, iops=444, runt= 90187msec 
> 
> VM 2: 
> write: io=2351MB, bw=26719KB/s, iops=6679, runt= 90083msec 
> read : io=2190MB, bw=24899KB/s, iops=6224, runt= 90081msec 
> write: io=48320MB, bw=545652KB/s, iops=133, runt= 90680msec 
> read : io=88136MB, bw=999571KB/s, iops=244, runt= 90290msec 
> 
> Right now i've no idea why the Dual Xeon is slower at all / in general. 
> 
> Stefan 
> _______________________________________________ 
> pve-devel mailing list 
> pve-devel at pve.proxmox.com 
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 

_______________________________________________ 
pve-devel mailing list 
pve-devel at pve.proxmox.com 
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel