[PVE-User] vm's disk io lower with local highspeed disk?

Fri Jun 17 09:38:27 CEST 2016

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Your test is completely useless. It does not proof anything.

On June 17, 2016 9:06:45 AM GMT+02:00, haoyun <haoyun-kgq at 163.com> wrote:
>First of all,thank you for answering.
>the result is : it is not vm's problem,it is raid card problem,raid
>card without cache.
>I test again ,the data as below:
>dd if=/dev/zero of=dd.img bs=1M count=2000
>-------------------------------------------------------------------------
>use single thread:
>/root/fio -filename=/mnt/dd.img -direct=1 -iodepth 1 -thread
>-rw=randwrite -ioengine=psync -bs=16k -size=1G -numjobs=1 -runtime=60
>-group_reporting -name=mytest
>R330(H330 0MB-cache,SAS 10K 600G * 6, Raid-10)  : bw=3028.7KB/s,
>iops=189
>NX3200 (H710 Mini 512MB-cache,SAS 7.2K 4T * 10,Raid-5) : bw=14555KB/s,
>iops=909
>MD3200i (2GB-cache,SAS 15K 600G * 12 ,Raid-5) : bw=24120KB/s, iops=1507
>R710 (PERC 6/i Integrated RAID Controller 256MB-cache,SAS 15K 600G * 2,
>Raid-1) : bw=16183KB/s, iops=1011
>R410 (SAS 6/iR Integrated Controller 0MB-cache,SAS 15K 300G * 2,Raid-1)
>: bw=3397.6KB/s, iops=212
>
>
>
>
>
>At 2016-06-15 12:22:35, "Alexandre DERUMIER" <aderumier at odiso.com>
>wrote:
>>raid10 is physical host's Raid-10 with 6 disks(SAS 10k 600G)
>>
>>
>>>>local-highspeed is LogicalVolume on Raid-10
>>
>>lvm volume ?  classic lvm or thin lvm ?
>>
>>
>>>>raid10 is physical host's Raid-10 with 6 disks(SAS 10k 600G)
>>>>"do you have cache on your raid controller ?" I don't know what's
>means?
>>
>>Well, generally (for hdd mainly), hardware raid controller have memory
>to handle writes fast, then flush them to disk.
>>what is your model ?
>>
>>
>>
>>>>fio command is : ./fio -filename=/dev/vda -direct=1 -iodepth 1
>-thread -rw=randwrite -ioengine=psync -bs=16k -size=4G -numjobs=30
>-runtime=100 -group_reporting -name=mytest
>>
>>So you are doing sync write,
>>that mean that it should bypass all caches (including raid controller
>cache, disk cache) to be sure that data are correctly written to the
>disk platters.
>>
>>I'm surprised than you can reach iops=1344 with 6 SAS 10K in raid10.
>(should be 250 iops by disk , x3 because of raid10, so around 750iops).
>>
>>
>>Now, some raid hardware controller, can handle the sync writes,
>without transmit them to disk platter (They need to have a batery, or
>persistent memory).
>>Could explain why you have better performance with your san.
>>
>>
>>can you do fio benchmark with ioengin : libaio , and increase iodepth
>to 32
>>
>>./fio -filename=/dev/vda -direct=1 -iodepth 32 -rw=randwrite
>-ioengine=libaiO -bs=16k -size=4G -numjobs=30 -runtime=100
>-group_reporting -name=mytest
>>
>>
>>then compare with --sync=1 option
>>
>>./fio -filename=/dev/vda -direct=1 -iodepth 32 -thread -rw=randwrite
>-ioengine=libaiO -bs=16k -size=4G -numjobs=30 -runtime=100
>-group_reporting -name=mytest
>>
>>----- Mail original -----
>>De: "haoyun" <haoyun-kgq at 163.com>
>>À: "proxmoxve" <pve-user at pve.proxmox.com>
>>Envoyé: Mercredi 15 Juin 2016 03:51:03
>>Objet: Re: [PVE-User] vm's disk io lower with local highspeed disk?
>>
>>fio command is : ./fio -filename=/dev/vda -direct=1 -iodepth 1 -thread
>-rw=randwrite -ioengine=psync -bs=16k -size=4G -numjobs=30 -runtime=100
>-group_reporting -name=mytest
>>
>>
>>vmconfig:
>>balloon: 1024
>>bootdisk: virtio2
>>cores: 2
>>ide2: none,media=cdrom
>>memory: 4096
>>name: tt
>>net0: bridge=vmbr0,e1000=66:37:31:32:34:37
>>numa: 0
>>ostype: l26
>>smbios1: uuid=ce775a6f-6652-43c4-aa92-69ffd3d20013
>>sockets: 1
>>virtio0: local-highspeed:vm-255-disk-2,size=60G
>>virtio2: local-highspeed:vm-255-disk-1,size=8G
>>
>>
>>raid10 is physical host's Raid-10 with 6 disks(SAS 10k 600G)
>>
>>
>>local-highspeed is LogicalVolume on Raid-10
>>
>>"do you have cache on your raid controller ?" I don't know what's
>means?
>>
>>
>>
>>At 2016-06-14 17:59:17, "Alexandre DERUMIER" <aderumier at odiso.com>
>wrote:
>>>>>Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await
>svctm %util
>>>>>vda 0.00 71.00 8.00 226.00 64.00 2368.00 10.39 1.37 5.85 4.24 99.10
>
>>>>>but I test disk random write is better with fio tools,
>>>>>bw=21518KB/s iops=1344
>>>
>>>
>>>do you have cache on your raid controller ?
>>>
>>>can you send the fio job config?
>>>
>>>can you send vmconfig ?
>>>
>>>what is the local raid10 filesystem ?
>>>
>>>
>>>
>>>
>>>----- Mail original -----
>>>De: "haoyun" <haoyun-kgq at 163.com>
>>>À: "proxmoxve" <pve-user at pve.proxmox.com>
>>>Envoyé: Mardi 14 Juin 2016 07:53:29
>>>Objet: [PVE-User] vm's disk io lower with local highspeed disk?
>>>
>>>hello everyone~
>>>
>>>
>>>I have a problem,
>>>I have a new host is Dell R330,
>>>run pve4.2 os
>>>disk with 2 raid,one is raid1,another is raid10(sas 10k 600G * 6)
>>>raid1 as system disk,raid10 as vm images
>>>my mysql slave run on this vm with raid10 local disk(lvm).
>>>the question is this vm's disk io is very slow,
>>>that's why?
>>>Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await
>svctm %util
>>>vda 0.00 71.00 8.00 226.00 64.00 2368.00 10.39 1.37 5.85 4.24 99.10
>>>but I test disk random write is better with fio tools,
>>>bw=21518KB/s iops=1344
>>>same vm,I use iscsi remote storage better than it,
>>>Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await
>svctm %util
>>>vda 0.00 0.00 0.00 3.00 0.00 24.00 8.00 0.00 0.33 0.33 0.10
>>>that's why?
>>>who can help me?
>>>What information is needed？please tell me?
>>>thanks!
>>>_______________________________________________
>>>pve-user mailing list
>>>pve-user at pve.proxmox.com
>>>http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>>>_______________________________________________
>>>pve-user mailing list
>>>pve-user at pve.proxmox.com
>>>http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>>_______________________________________________
>>pve-user mailing list
>>pve-user at pve.proxmox.com
>>http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>>_______________________________________________
>>pve-user mailing list
>>pve-user at pve.proxmox.com
>>http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>_______________________________________________
>pve-user mailing list
>pve-user at pve.proxmox.com
>http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

- --
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
-----BEGIN PGP SIGNATURE-----
Version: APG v1.1.1

iHsEAREKADsFAldjqPM0HE1pY2hhZWwgUmFzbXVzc2VuIChNaWNoYWVsIEdQRy1r
ZXkpIDxtaXJAbWlyYXMub3JnPgAKCRBUSthV4+gJF507AJ91hgEodeZxNq0+z1He
4QqW/cur7QCdECdxXroBw+e1mhwFWA9xySDcaRA=
=5S0m
-----END PGP SIGNATURE-----