[pve-devel] [PATCH] Increases the dd read/write speed when migrating LVM disks. This should bring LVM migration speed up to match that of rsyncing qcows.

[Michael Rasmussen]

I can test on infiniband when I get home today.

On August 24, 2016 9:35:35 AM GMT+02:00, Alexandre DERUMIER <aderumier at odiso.com> wrote:
>>>64k got similar speeds.
>
>Do you use 1gigabit network link ? (as it seem to be around 100MB/S
>max)
>
>----- Mail original -----
>De: "jlavoy" <jalavoy at gmail.com>
>À: "pve-devel" <pve-devel at pve.proxmox.com>
>Envoyé: Mardi 23 Août 2016 19:54:31
>Objet: Re: [pve-devel] [PATCH] Increases the dd read/write speed when
>migrating LVM disks. This should bring LVM migration speed up to match
>that of rsyncing qcows.
>
>I don't feel super strongly about what bs is used, from where I'm
>sitting, 
>whatever the community wants is fine, provided it's higher than
>default. 
>
>64k got similar speeds. If we wanna go that low that's fine by me: 
>
>abrams Code # time dd if=1GB.file bs=64K |ssh mary "dd
>of=/root/1GB.file 
>bs=64K" 
>16384+0 records in 
>16384+0 records out 
>1073741824 bytes (1.1 GB) copied, 10.2513 s, 105 MB/s 
>7+61998 records in 
>7+61998 records out 
>1073741824 bytes (1.1 GB) copied, 10.1363 s, 106 MB/s 
>
>real 0m10.257s 
>user 0m7.270s 
>sys 0m2.364s 
>
>
>On Tue, Aug 23, 2016 at 5:47 AM, Michael Rasmussen <mir at datanom.net>
>wrote: 
>
>> Default block equals logical block size so either 512B or 4k. 
>> 
>> On August 23, 2016 12:16:49 PM GMT+02:00, Alexandre DERUMIER < 
>> aderumier at odiso.com> wrote: 
>> >>>I think a value which can be handle by disk cache is ok. (1M
>should 
>> >be large enough, if the disk can do 200iops, it's around 200MB/S) 
>> > 
>> >Another thing to check , is the sparse option. 
>> > 
>> >If the block size is too high, the are more chance that they are
>datas 
>> >inside the block, so it'll be full copied. 
>> > 
>> >What is the result with a 64k block size instead 1M ? (I think the 
>> >default bs is 512b, so 64k is already a huge improvement) 
>> > 
>> >----- Mail original ----- 
>> >De: "aderumier" <aderumier at odiso.com> 
>> >À: "dietmar" <dietmar at proxmox.com>, "pve-devel" 
>> ><pve-devel at pve.proxmox.com> 
>> >Cc: "jlavoy" <jalavoy at gmail.com> 
>> >Envoyé: Mardi 23 Août 2016 12:12:17 
>> >Objet: Re: [pve-devel] [PATCH] Increases the dd read/write speed
>when 
>> >migrating LVM disks. This should bring LVM migration speed up to
>match 
>> >that of rsyncing qcows. 
>> > 
>> >>>IMHO a block size of 64MB is a bit too large. I guess you can see
>the 
>> >same 
>> >>>speed improvements with much smaller sizes - mabe 1M is large
>enough? 
>> > 
>> > 
>> >I think a value which can be handle by disk cache is ok. (1M should
>be 
>> >large enough, if the disk can do 200iops, it's around 200MB/S) 
>> > 
>> >----- Mail original ----- 
>> >De: "dietmar" <dietmar at proxmox.com> 
>> >À: "jlavoy" <jalavoy at gmail.com>, "pve-devel" 
>> ><pve-devel at pve.proxmox.com> 
>> >Envoyé: Mardi 23 Août 2016 10:26:38 
>> >Objet: Re: [pve-devel] [PATCH] Increases the dd read/write speed
>when 
>> >migrating LVM disks. This should bring LVM migration speed up to
>match 
>> >that of rsyncing qcows. 
>> > 
>> >> + run_command([["dd", "if=$src", "bs=64M"],["/usr/bin/ssh", 
>> >> "root\@${target_host}", 
>> >> + "dd", 'conv=sparse', "of=$dst", "bs=64M"]]); 
>> > 
>> >IMHO a block size of 64MB is a bit too large. I guess you can see
>the 
>> >same 
>> >speed improvements with much smaller sizes - mabe 1M is large
>enough? 
>> > 
>> >_______________________________________________ 
>> >pve-devel mailing list 
>> >pve-devel at pve.proxmox.com 
>> >http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
>> > 
>> >_______________________________________________ 
>> >pve-devel mailing list 
>> >pve-devel at pve.proxmox.com 
>> >http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
>> 
>> -- 
>> Sent from my Android phone with K-9 Mail. Please excuse my brevity. 
>> _______________________________________________ 
>> pve-devel mailing list 
>> pve-devel at pve.proxmox.com 
>> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
>> 
>_______________________________________________ 
>pve-devel mailing list 
>pve-devel at pve.proxmox.com 
>http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
>
>_______________________________________________
>pve-devel mailing list
>pve-devel at pve.proxmox.com
>http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.