[PVE-User] vm's disk io lower with local highspeed disk?

haoyun haoyun-kgq at 163.com
Fri Jun 17 09:06:45 CEST 2016 

First of all,thank you for answering.
the result is : it is not vm's problem,it is raid card problem,raid card without cache.
I test again ,the data as below:
dd if=/dev/zero of=dd.img bs=1M count=2000
-------------------------------------------------------------------------
use single thread:
/root/fio -filename=/mnt/dd.img -direct=1 -iodepth 1 -thread -rw=randwrite -ioengine=psync -bs=16k -size=1G -numjobs=1 -runtime=60 -group_reporting -name=mytest
R330(H330 0MB-cache,SAS 10K 600G * 6, Raid-10)  : bw=3028.7KB/s, iops=189
NX3200 (H710 Mini 512MB-cache,SAS 7.2K 4T * 10,Raid-5) : bw=14555KB/s, iops=909
MD3200i (2GB-cache,SAS 15K 600G * 12 ,Raid-5) : bw=24120KB/s, iops=1507
R710 (PERC 6/i Integrated RAID Controller 256MB-cache,SAS 15K 600G * 2, Raid-1) : bw=16183KB/s, iops=1011
R410 (SAS 6/iR Integrated Controller 0MB-cache,SAS 15K 300G * 2,Raid-1) : bw=3397.6KB/s, iops=212





At 2016-06-15 12:22:35, "Alexandre DERUMIER" <aderumier at odiso.com> wrote:
>raid10 is physical host's Raid-10 with 6 disks(SAS 10k 600G) 
>
>
>>>local-highspeed is LogicalVolume on Raid-10 
>
>lvm volume ?  classic lvm or thin lvm ?
>
>
>>>raid10 is physical host's Raid-10 with 6 disks(SAS 10k 600G) 
>>>"do you have cache on your raid controller ?" I don't know what's means? 
>
>Well, generally (for hdd mainly), hardware raid controller have memory to handle writes fast, then flush them to disk.
>what is your model ?
>
>
>
>>>fio command is : ./fio -filename=/dev/vda -direct=1 -iodepth 1 -thread -rw=randwrite -ioengine=psync -bs=16k -size=4G -numjobs=30 -runtime=100 -group_reporting -name=mytest 
>
>So you are doing sync write, 
>that mean that it should bypass all caches (including raid controller cache, disk cache) to be sure that data are correctly written to the disk platters.
>
>I'm surprised than you can reach iops=1344 with 6 SAS 10K in raid10.  (should be 250 iops by disk , x3 because of raid10, so around 750iops).
>
>
>Now, some raid hardware controller, can handle the sync writes, without transmit them to disk platter (They need to have a batery, or persistent memory).
>Could explain why you have better performance with your san.
>
>
>can you do fio benchmark with ioengin : libaio , and increase iodepth to 32
>
>./fio -filename=/dev/vda -direct=1 -iodepth 32 -rw=randwrite -ioengine=libaiO -bs=16k -size=4G -numjobs=30 -runtime=100 -group_reporting -name=mytest 
>
>
>then compare with --sync=1 option
>
>./fio -filename=/dev/vda -direct=1 -iodepth 32 -thread -rw=randwrite -ioengine=libaiO -bs=16k -size=4G -numjobs=30 -runtime=100 -group_reporting -name=mytest 
>
>----- Mail original -----
>De: "haoyun" <haoyun-kgq at 163.com>
>À: "proxmoxve" <pve-user at pve.proxmox.com>
>Envoyé: Mercredi 15 Juin 2016 03:51:03
>Objet: Re: [PVE-User] vm's disk io lower with local highspeed disk?
>
>fio command is : ./fio -filename=/dev/vda -direct=1 -iodepth 1 -thread -rw=randwrite -ioengine=psync -bs=16k -size=4G -numjobs=30 -runtime=100 -group_reporting -name=mytest 
>
>
>vmconfig: 
>balloon: 1024 
>bootdisk: virtio2 
>cores: 2 
>ide2: none,media=cdrom 
>memory: 4096 
>name: tt 
>net0: bridge=vmbr0,e1000=66:37:31:32:34:37 
>numa: 0 
>ostype: l26 
>smbios1: uuid=ce775a6f-6652-43c4-aa92-69ffd3d20013 
>sockets: 1 
>virtio0: local-highspeed:vm-255-disk-2,size=60G 
>virtio2: local-highspeed:vm-255-disk-1,size=8G 
>
>
>raid10 is physical host's Raid-10 with 6 disks(SAS 10k 600G) 
>
>
>local-highspeed is LogicalVolume on Raid-10 
>
>"do you have cache on your raid controller ?" I don't know what's means? 
>
>
>
>At 2016-06-14 17:59:17, "Alexandre DERUMIER" <aderumier at odiso.com> wrote: 
>>>>Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util 
>>>>vda 0.00 71.00 8.00 226.00 64.00 2368.00 10.39 1.37 5.85 4.24 99.10 
>>>>but I test disk random write is better with fio tools, 
>>>>bw=21518KB/s iops=1344 
>> 
>> 
>>do you have cache on your raid controller ? 
>> 
>>can you send the fio job config? 
>> 
>>can you send vmconfig ? 
>> 
>>what is the local raid10 filesystem ? 
>> 
>> 
>> 
>> 
>>----- Mail original ----- 
>>De: "haoyun" <haoyun-kgq at 163.com> 
>>À: "proxmoxve" <pve-user at pve.proxmox.com> 
>>Envoyé: Mardi 14 Juin 2016 07:53:29 
>>Objet: [PVE-User] vm's disk io lower with local highspeed disk? 
>> 
>>hello everyone~ 
>> 
>> 
>>I have a problem, 
>>I have a new host is Dell R330, 
>>run pve4.2 os 
>>disk with 2 raid,one is raid1,another is raid10(sas 10k 600G * 6) 
>>raid1 as system disk,raid10 as vm images 
>>my mysql slave run on this vm with raid10 local disk(lvm). 
>>the question is this vm's disk io is very slow, 
>>that's why? 
>>Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util 
>>vda 0.00 71.00 8.00 226.00 64.00 2368.00 10.39 1.37 5.85 4.24 99.10 
>>but I test disk random write is better with fio tools, 
>>bw=21518KB/s iops=1344 
>>same vm,I use iscsi remote storage better than it, 
>>Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util 
>>vda 0.00 0.00 0.00 3.00 0.00 24.00 8.00 0.00 0.33 0.33 0.10 
>>that's why? 
>>who can help me? 
>>What information is needed？please tell me? 
>>thanks! 
>>_______________________________________________ 
>>pve-user mailing list 
>>pve-user at pve.proxmox.com 
>>http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user 
>>_______________________________________________ 
>>pve-user mailing list 
>>pve-user at pve.proxmox.com 
>>http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user 
>_______________________________________________ 
>pve-user mailing list 
>pve-user at pve.proxmox.com 
>http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user 
>_______________________________________________
>pve-user mailing list
>pve-user at pve.proxmox.com
>http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

More information about the pve-user mailing list