[pve-devel] less cores more iops / speed

[Stefan Priebe]

Am 13.11.2012 09:01, schrieb Alexandre DERUMIER:
> Ok, I hope Intank will look at librbd and find the bottleneck !
I hope that too. Sadly i'm not able to produce a callgraph with google 
profiler as the kvm proces just segfaults when it is loaded.


> ----- Mail original -----
>
> De: "Stefan Priebe" <s.priebe at profihost.ag>
> À: "Alexandre DERUMIER" <aderumier at odiso.com>
> Cc: pve-devel at pve.proxmox.com, "Dietmar Maurer" <dietmar at proxmox.com>
> Envoyé: Mardi 13 Novembre 2012 08:59:41
> Objet: Re: [pve-devel] less cores more iops / speed
>
> Not sure which component limits the speed. There's no measurable
> component running at 100% CPU load. Or 100% network load...
>
> Stefan
>
> Am 13.11.2012 08:46, schrieb Alexandre DERUMIER:
>>> so around 3000 iops by node, I hope It'll improve in time.
>>>> Yes at least it scales. Sadly only 3000 iops... but each node has a
>>
>>>> capacity of 360.000 iops... perhaps a bit overkill... but SSDs are near
>>>> cheap as SAS drives...
>>
>> So, If you launch fio from differents kvm guest on differents kvm host, you are limited by the ceph cluster speed ?
>>
>>
>> ----- Mail original -----
>>
>> De: "Stefan Priebe" <s.priebe at profihost.ag>
>> À: "Alexandre DERUMIER" <aderumier at odiso.com>
>> Cc: pve-devel at pve.proxmox.com, "Dietmar Maurer" <dietmar at proxmox.com>
>> Envoyé: Lundi 12 Novembre 2012 23:37:14
>> Objet: Re: [pve-devel] less cores more iops / speed
>>
>> Am 09.11.2012 15:43, schrieb Alexandre DERUMIER:
>>>>> To me yes. With 3 Nodes i had a max of 10.000 iops.
>>>
>>> Ah, great to known !
>>>
>>> so around 3000 iops by node, I hope It'll improve in time.
>> Yes at least it scales. Sadly only 3000 iops... but each node has a
>> capacity of 360.000 iops... perhaps a bit overkill... but SSDs are near
>> cheap as SAS drives...
>>
>>> Did you have tested with gigabit link with bigger latencies ?
>> No. But i hope i can compare Arista 10G against HP 10G soon.
>>
>> Stefan
>>
>>
>>> ----- Mail original -----
>>>
>>> De: "Stefan Priebe - Profihost AG" <s.priebe at profihost.ag>
>>> À: "Alexandre DERUMIER" <aderumier at odiso.com>
>>> Cc: pve-devel at pve.proxmox.com, "Dietmar Maurer" <dietmar at proxmox.com>
>>> Envoyé: Vendredi 9 Novembre 2012 15:28:12
>>> Objet: Re: [pve-devel] less cores more iops / speed
>>>
>>> Am 09.11.2012 11:31, schrieb Alexandre DERUMIER:
>>>>>> Yes. CPU Load in Ceph for rand. 4k writes can be dropped by 50% when
>>>>>> disabling all debug setting (see my last post on ceph mailinglist).
>>>>
>>>> I just see it, great,I'll test it on my side :)
>>>>
>>>> (how many ssds - nodes ?) I hope that Intank is working on random io performance.
>>> I have 5 nodes with 4 SSDs each.
>>>
>>>> I don't understand why we can't get more iops, if they are no cpu or disks bottleneck... (or they are locks somewhere ....)
>>>>
>>>> I really would like to known if iops scale with the number of nodes...
>>>
>>> To me yes. With 3 Nodes i had a max of 10.000 iops.
>>>
>>> Stefan
>>>