[pve-devel] [PATCH pve-storage 0/2] move qemu_img_create to common helpers and enable preallocation on backed images

[Fabian Grünbichler]

> Alexandre Derumier via pve-devel <pve-devel at lists.proxmox.com> hat am 19.05.2025 12:23 CEST geschrieben:
> This is part of my work on qcow2 external snapshot, but could improve current qcow2 linked clone
> 
> This patch serie move qemu_img_create to common helpers,
> and enable preallocation on backed_image to increase performance

did you maybe forget to send the actual patches? I can't find them
in my inbox, and they aren't on lore either:

https://lore.proxmox.com/pve-devel/mailman.556.1747917864.394.pve-devel@lists.proxmox.com/t/#u

> 
> This require l2_extended=on on the backed image
> 
> I have done some benchmarks on localssd with 100gb qcow2, the performance in randwrite 4k is 5x faster
> 
> some presentation of l2_extended=on are available here
> https://www.youtube.com/watch?v=zJetcfDVFNw
> https://www.youtube.com/watch?v=NfgLCdtkRus
> 
> I don't have enabled it for base image, as I think that Fabian see performance regression some month ago.
> but I don't see performance difference in my bench. (can you could test on your side again ?)
> 
> It could help to reduce qcow2 overhead on disk,
> and allow to keep more metadatas in memory for bigger image, as qemu default memory l2_cache_size=1MB)
> https://www.ibm.com/products/tutorials/how-to-tune-qemu-l2-cache-size-and-qcow2-cluster-size
> Maybe more test with bigger image (>1TB) could be done too to see if it's help
> 
> 
> 
> bench on 100G qcow2 file:
> 
> fio --filename=/dev/sdb --direct=1 --rw=randwrite --bs=4k --iodepth=32 --ioengine=libaio --name=test
> fio --filename=/dev/sdb --direct=1 --rw=randread --bs=4k --iodepth=32 --ioengine=libaio --name=test
> 
> base image:
> 
> randwrite 4k: prealloc=metadata, l2_extended=off, cluster_size=64k: 20215
> randread 4k: prealloc=metadata, l2_extended=off, cluster_size=64k: 22219
> randwrite 4k: prealloc=metadata, l2_extended=on, cluster_size=64k: 20217
> randread 4k: prealloc=metadata, l2_extended=on, cluster_size=64k: 21742
> randwrite 4k: prealloc=metadata, l2_extended=on, cluster_size=128k: 21599
> randread 4k: prealloc=metadata, l2_extended=on, cluster_size=128k: 22037
> 
> linked clone image with backing file:
> 
> randwrite 4k: prealloc=metadata, l2_extended=off, cluster_size=64k: 3912
> randread 4k: prealloc=metadata, l2_extended=off, cluster_size=64k: 21476
> randwrite 4k: prealloc=metadata, l2_extended=on, cluster_size=64k: 20563
> randread 4k: prealloc=metadata, l2_extended=on, cluster_size=64k: 22265
> randwrite 4k: prealloc=metadata, l2_extended=on, cluster_size=128k: 18016
> randread 4k: prealloc=metadata, l2_extended=on, cluster_size=128k: 21611
> 
> Alexandre Derumier (2):
>   common: add qemu_img_create an preallocation_cmd_option
>   common: qemu_img_create: add backing_file support
> 
>  src/PVE/Storage/Common.pm          | 62 ++++++++++++++++++++++++++++++
>  src/PVE/Storage/GlusterfsPlugin.pm |  2 +-
>  src/PVE/Storage/Plugin.pm          | 52 +------------------------
>  3 files changed, 65 insertions(+), 51 deletions(-)
> 
> -- 
> 2.39.5