[pve-devel] Limited IOP/s on Dual Xeon KVM Host

[Stefan Priebe]

Got it fixed by Bios update... crazy.

Am 10.11.2012 17:00, schrieb Andrey Korolyov:
> On Sat, Nov 10, 2012 at 5:49 PM, Stefan Priebe <s.priebe at profihost.ag> wrote:
>> Am 10.11.2012 14:41, schrieb Mark Nelson:
>>
>>> On 11/10/2012 02:03 AM, Stefan Priebe wrote:
>>>>
>>>> Hello lists,
>>>>
>>>> on a dual Xeon KVM Host i get max 6000 IOP/s random 4k writes AND reads.
>>>> On a Single Xeon KVM Host i get 17.000-18.000 IOP/s write and read. I
>>>> already tried to pin the kvm process using numactl and also the fio
>>>> process but it doesn't help on the dual xeon.
>>>>
>>>> 10GBE Network is fine. I get 9.8Gbit/s on both hosts. Kernel is also he
>>>> same on both.
>>>>
>>>> Anybody an idea?
>>>
>>>
>>> When you say KVM host, do you mean the underlying node or the virtual
>>> machine instance?
>>
>> Sorry i'm talking about the vm host regarding the HW. The vm instance is
>> always the same.
>>
>>
>>> If you mean underlying node, it could be remote memory access or if you
>>> are on a last gen xeon if you have dual io hubs, you could be hitting a
>>> remote io hub for the network card.  I wouldn't think that would cause
>>> such a big hit, but those are things to look into.
>>
>>
>> I'm on E5-Xeon. What means io hub?
>
> QPI path length, in other terms, numa distance(hope Mark means the
> same). Yes, it is impossible to have such degradation even in worst
> case on two-head node. I assume two possible things - you have pinned
> many processes on the core set which including default core for the
> network card` irq, please check it via /proc/interrupts, or you have
> not really did pinning and qemu process losing ticks by switching
> cores - it may be checked, say, by top and guest cpu bencmark. For the
> network card, it may be generally recommended to move its irq affinity
> to entire numa node to which it belongs.
>
>>
>>
>> Stefan
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo at vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>