[pve-devel] Speed up PVE Backup

[Alexandre DERUMIER]

>>There is no reason to flush a restored disk until just the end, really. 
>>Issuing flushes every x MB could hurt other storages without need. 

I'm curious to see host memory usage of a big local file storage restore (100GB), with writeback without any flush ?


----- Mail original -----
De: "Eneko Lacunza" <elacunza at binovo.es>
À: "pve-devel" <pve-devel at pve.proxmox.com>
Envoyé: Mardi 26 Juillet 2016 10:13:59
Objet: Re: [pve-devel] Speed up PVE Backup

Hi, 

El 26/07/16 a las 10:04, Alexandre DERUMIER escribió: 
>>> I think qmrestore isn't issuing any flush request (until maybe the end), 
> Need to be checked! (but if I think we open restore block storage with writeback, so I hope we send flush) 
> 
>>> so for ceph storage backend we should set 
>>> rbd_cache_writethrough_until_flush=false for better performance. 
> I think it's possible to pass theses flag in qemu block driver option, when opening the rbd storage 
> 
> 
> http://docs.ceph.com/docs/hammer/rbd/qemu-rbd/ 
> 
> qemu-img {command} [options] rbd:{pool-name}/{image-name}[@snapshot-name][:option1=value1][:option2=value2...] 
> 
> 
> for qemu-img or with qemu drive option, I think it's possible to send as option, ":rbd_cache_writethrough_until_flush=false" 
I developed a small patch to do this, waiting to test it in our setup 
(today or tomorrow) 
> But if missing flush if really the problem, it should be added to restore command directly. (maybe 1 flush each 4MB for example) 
This flush is needed only for Ceph RBD, so I think using the flag above 
would be more correct. 

There is no reason to flush a restored disk until just the end, really. 
Issuing flushes every x MB could hurt other storages without need. 

In fact all this is because Ceph trying to "fix" broken virtio drivers... :) 

Thanks 
Eneko 

-- 
Zuzendari Teknikoa / Director Técnico 
Binovo IT Human Project, S.L. 
Telf. 943493611 
943324914 
Astigarraga bidea 2, planta 6 dcha., ofi. 3-2; 20180 Oiartzun (Gipuzkoa) 
www.binovo.es 

_______________________________________________ 
pve-devel mailing list 
pve-devel at pve.proxmox.com 
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel