[pve-devel] applied: [PATCH http-server] file upload: don't calculate MD5

Thomas Lamprecht t.lamprecht at proxmox.com
Thu Apr 13 12:52:52 CEST 2023


Am 12/04/2023 um 16:22 schrieb Matthias Heiserer:
> Until now, we calculated the MD5 hash of any uploaded file during the upload, regardless
> of whether the user chose to provide a hash sum and algorithm.
> The hash was only logged in the syslog.
> 
> As the user can provide a hash algorithm and a checksum when uploading a file,
> which gets automatically checked (after the upload), this is not needed anymore.
> Instead, the file name is logged.
> 
> Depending on the speed of the network and the cpu, upload speed or CPU usage might improve:
> All tests were made by uploading a 3.6GB iso from the PVE host to a local VM.
> First line is with md5, second without.
> 
> no networklimit
> multipart upload complete (size: 3826831360B time: 20.310s rate: 179.69MiB/s md5sum: 8c651682056205967d530697c98d98c3)
> multipart upload complete (size: 3826831360B time: 16.169s rate: 225.72MiB/s filename: ubuntu-22.04.1-desktop-amd64.iso)
> 
> 125MB/s network
> In this test, pveproxy worker used x % CPU during the upload. As you can see, the reduced CPU usage is noticable in slower networks.
> ~75% CPU: multipart upload complete (size: 3826831360B time: 30.764s rate: 118.63MiB/s md5sum: 8c651682056205967d530697c98d98c3)
> ~60% CPU: multipart upload complete (size: 3826831360B time: 30.763s rate: 118.64MiB/s filename: ubuntu-22.04.1-desktop-amd64.iso)
> 
> qemu64 cpu, no network limit
> multipart upload complete (size: 3826831360B time: 46.113s rate: 79.14MiB/s md5sum: 8c651682056205967d530697c98d98c3)
> multipart upload complete (size: 3826831360B time: 41.492s rate: 87.96MiB/s filename: ubuntu-22.04.1-desktop-amd64.iso)
> 
> qemu64, -aes, 1 core, 0.7 cpu
> multipart upload complete (size: 3826831360B time: 79.875s rate: 45.69MiB/s md5sum: 8c651682056205967d530697c98d98c3)
> multipart upload complete (size: 3826831360B time: 66.364s rate: 54.99MiB/s filename: ubuntu-22.04.1-desktop-amd64.iso)
> 
> Signed-off-by: Matthias Heiserer <m.heiserer at proxmox.com>
> ---
>  src/PVE/APIServer/AnyEvent.pm | 8 +++-----
>  1 file changed, 3 insertions(+), 5 deletions(-)
> 
>

applied, thanks!





More information about the pve-devel mailing list