[PVE-User] vm's disk io lower with local highspeed disk?

[Alexandre DERUMIER]

raid10 is physical host's Raid-10 with 6 disks(SAS 10k 600G) 


>>local-highspeed is LogicalVolume on Raid-10 

lvm volume ?  classic lvm or thin lvm ?


>>raid10 is physical host's Raid-10 with 6 disks(SAS 10k 600G) 
>>"do you have cache on your raid controller ?" I don't know what's means? 

Well, generally (for hdd mainly), hardware raid controller have memory to handle writes fast, then flush them to disk.
what is your model ?



>>fio command is : ./fio -filename=/dev/vda -direct=1 -iodepth 1 -thread -rw=randwrite -ioengine=psync -bs=16k -size=4G -numjobs=30 -runtime=100 -group_reporting -name=mytest 

So you are doing sync write, 
that mean that it should bypass all caches (including raid controller cache, disk cache) to be sure that data are correctly written to the disk platters.

I'm surprised than you can reach iops=1344 with 6 SAS 10K in raid10.  (should be 250 iops by disk , x3 because of raid10, so around 750iops).


Now, some raid hardware controller, can handle the sync writes, without transmit them to disk platter (They need to have a batery, or persistent memory).
Could explain why you have better performance with your san.


can you do fio benchmark with ioengin : libaio , and increase iodepth to 32

./fio -filename=/dev/vda -direct=1 -iodepth 32 -rw=randwrite -ioengine=libaiO -bs=16k -size=4G -numjobs=30 -runtime=100 -group_reporting -name=mytest 


then compare with --sync=1 option

./fio -filename=/dev/vda -direct=1 -iodepth 32 -thread -rw=randwrite -ioengine=libaiO -bs=16k -size=4G -numjobs=30 -runtime=100 -group_reporting -name=mytest 

----- Mail original -----
De: "haoyun" <haoyun-kgq at 163.com>
À: "proxmoxve" <pve-user at pve.proxmox.com>
Envoyé: Mercredi 15 Juin 2016 03:51:03
Objet: Re: [PVE-User] vm's disk io lower with local highspeed disk?

fio command is : ./fio -filename=/dev/vda -direct=1 -iodepth 1 -thread -rw=randwrite -ioengine=psync -bs=16k -size=4G -numjobs=30 -runtime=100 -group_reporting -name=mytest 


vmconfig: 
balloon: 1024 
bootdisk: virtio2 
cores: 2 
ide2: none,media=cdrom 
memory: 4096 
name: tt 
net0: bridge=vmbr0,e1000=66:37:31:32:34:37 
numa: 0 
ostype: l26 
smbios1: uuid=ce775a6f-6652-43c4-aa92-69ffd3d20013 
sockets: 1 
virtio0: local-highspeed:vm-255-disk-2,size=60G 
virtio2: local-highspeed:vm-255-disk-1,size=8G 


raid10 is physical host's Raid-10 with 6 disks(SAS 10k 600G) 


local-highspeed is LogicalVolume on Raid-10 

"do you have cache on your raid controller ?" I don't know what's means? 



At 2016-06-14 17:59:17, "Alexandre DERUMIER" <aderumier at odiso.com> wrote: 
>>>Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util 
>>>vda 0.00 71.00 8.00 226.00 64.00 2368.00 10.39 1.37 5.85 4.24 99.10 
>>>but I test disk random write is better with fio tools, 
>>>bw=21518KB/s iops=1344 
> 
> 
>do you have cache on your raid controller ? 
> 
>can you send the fio job config? 
> 
>can you send vmconfig ? 
> 
>what is the local raid10 filesystem ? 
> 
> 
> 
> 
>----- Mail original ----- 
>De: "haoyun" <haoyun-kgq at 163.com> 
>À: "proxmoxve" <pve-user at pve.proxmox.com> 
>Envoyé: Mardi 14 Juin 2016 07:53:29 
>Objet: [PVE-User] vm's disk io lower with local highspeed disk? 
> 
>hello everyone~ 
> 
> 
>I have a problem, 
>I have a new host is Dell R330, 
>run pve4.2 os 
>disk with 2 raid,one is raid1,another is raid10(sas 10k 600G * 6) 
>raid1 as system disk,raid10 as vm images 
>my mysql slave run on this vm with raid10 local disk(lvm). 
>the question is this vm's disk io is very slow, 
>that's why? 
>Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util 
>vda 0.00 71.00 8.00 226.00 64.00 2368.00 10.39 1.37 5.85 4.24 99.10 
>but I test disk random write is better with fio tools, 
>bw=21518KB/s iops=1344 
>same vm,I use iscsi remote storage better than it, 
>Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util 
>vda 0.00 0.00 0.00 3.00 0.00 24.00 8.00 0.00 0.33 0.33 0.10 
>that's why? 
>who can help me? 
>What information is needed？please tell me? 
>thanks! 
>_______________________________________________ 
>pve-user mailing list 
>pve-user at pve.proxmox.com 
>http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user 
>_______________________________________________ 
>pve-user mailing list 
>pve-user at pve.proxmox.com 
>http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user 
_______________________________________________ 
pve-user mailing list 
pve-user at pve.proxmox.com 
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user