[pve-devel] [PATCH] Increases the dd read/write speed when migrating LVM disks. This should bring LVM migration speed up to match that of rsyncing qcows.

[Alexandre DERUMIER]

>>64k got similar speeds.

Do you use 1gigabit network link ? (as it seem to be around 100MB/S max)

----- Mail original -----
De: "jlavoy" <jalavoy at gmail.com>
À: "pve-devel" <pve-devel at pve.proxmox.com>
Envoyé: Mardi 23 Août 2016 19:54:31
Objet: Re: [pve-devel] [PATCH] Increases the dd read/write speed when migrating LVM disks. This should bring LVM migration speed up to match that of rsyncing qcows.

I don't feel super strongly about what bs is used, from where I'm sitting, 
whatever the community wants is fine, provided it's higher than default. 

64k got similar speeds. If we wanna go that low that's fine by me: 

abrams Code # time dd if=1GB.file bs=64K |ssh mary "dd of=/root/1GB.file 
bs=64K" 
16384+0 records in 
16384+0 records out 
1073741824 bytes (1.1 GB) copied, 10.2513 s, 105 MB/s 
7+61998 records in 
7+61998 records out 
1073741824 bytes (1.1 GB) copied, 10.1363 s, 106 MB/s 

real 0m10.257s 
user 0m7.270s 
sys 0m2.364s 


On Tue, Aug 23, 2016 at 5:47 AM, Michael Rasmussen <mir at datanom.net> wrote: 

> Default block equals logical block size so either 512B or 4k. 
> 
> On August 23, 2016 12:16:49 PM GMT+02:00, Alexandre DERUMIER < 
> aderumier at odiso.com> wrote: 
> >>>I think a value which can be handle by disk cache is ok. (1M should 
> >be large enough, if the disk can do 200iops, it's around 200MB/S) 
> > 
> >Another thing to check , is the sparse option. 
> > 
> >If the block size is too high, the are more chance that they are datas 
> >inside the block, so it'll be full copied. 
> > 
> >What is the result with a 64k block size instead 1M ? (I think the 
> >default bs is 512b, so 64k is already a huge improvement) 
> > 
> >----- Mail original ----- 
> >De: "aderumier" <aderumier at odiso.com> 
> >À: "dietmar" <dietmar at proxmox.com>, "pve-devel" 
> ><pve-devel at pve.proxmox.com> 
> >Cc: "jlavoy" <jalavoy at gmail.com> 
> >Envoyé: Mardi 23 Août 2016 12:12:17 
> >Objet: Re: [pve-devel] [PATCH] Increases the dd read/write speed when 
> >migrating LVM disks. This should bring LVM migration speed up to match 
> >that of rsyncing qcows. 
> > 
> >>>IMHO a block size of 64MB is a bit too large. I guess you can see the 
> >same 
> >>>speed improvements with much smaller sizes - mabe 1M is large enough? 
> > 
> > 
> >I think a value which can be handle by disk cache is ok. (1M should be 
> >large enough, if the disk can do 200iops, it's around 200MB/S) 
> > 
> >----- Mail original ----- 
> >De: "dietmar" <dietmar at proxmox.com> 
> >À: "jlavoy" <jalavoy at gmail.com>, "pve-devel" 
> ><pve-devel at pve.proxmox.com> 
> >Envoyé: Mardi 23 Août 2016 10:26:38 
> >Objet: Re: [pve-devel] [PATCH] Increases the dd read/write speed when 
> >migrating LVM disks. This should bring LVM migration speed up to match 
> >that of rsyncing qcows. 
> > 
> >> + run_command([["dd", "if=$src", "bs=64M"],["/usr/bin/ssh", 
> >> "root\@${target_host}", 
> >> + "dd", 'conv=sparse', "of=$dst", "bs=64M"]]); 
> > 
> >IMHO a block size of 64MB is a bit too large. I guess you can see the 
> >same 
> >speed improvements with much smaller sizes - mabe 1M is large enough? 
> > 
> >_______________________________________________ 
> >pve-devel mailing list 
> >pve-devel at pve.proxmox.com 
> >http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
> > 
> >_______________________________________________ 
> >pve-devel mailing list 
> >pve-devel at pve.proxmox.com 
> >http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
> 
> -- 
> Sent from my Android phone with K-9 Mail. Please excuse my brevity. 
> _______________________________________________ 
> pve-devel mailing list 
> pve-devel at pve.proxmox.com 
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
> 
_______________________________________________ 
pve-devel mailing list 
pve-devel at pve.proxmox.com 
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel