[pbs-devel] [RFC proxmox-backup 1/2] datastore: Allow encoding with a set compression level

[Maximiliano Sandoval]

Christian Ebner <c.ebner at proxmox.com> writes:

> I would suggest to perform all tests with the binaries compiled in release mode again, otherwise you might loose out on a lot of compile time optimizations.

Indeed, running them again I got the following.

```
$ ./target/release/proxmox-backup-client backup --compression=none root.pxar:PATH/linux --repository=pbs:linux-none
Starting backup: host/hostname/2023-11-06T10:52:00Z
Client name: hostname
Starting backup protocol: Mon Nov  6 11:52:00 2023
No previous manifest available.
Upload directory 'PATH/linux' to 'pbs:linux-none' as root.pxar.didx
root.pxar: had to backup 1.55 GiB of 1.55 GiB (compressed 1.55 GiB) in 8.47s
root.pxar: average backup speed: 187.38 MiB/s
Uploaded backup catalog (1.802 MiB)
Duration: 26.24s
End Time: Mon Nov  6 11:52:27 2023

$ ./target/release/proxmox-backup-client backup --compression=zstd,1 root.pxar:PATH/linux --repository=pbs:linux-zstd1
Starting backup: host/hostname/2023-11-06T10:52:44Z
Client name: hostname
Starting backup protocol: Mon Nov  6 11:52:44 2023
No previous manifest available.
Upload directory 'PATH/linux' to 'pbs:linux-zstd1' as root.pxar.didx
root.pxar: had to backup 1.55 GiB of 1.55 GiB (compressed 475.468 MiB) in 7.84s
root.pxar: average backup speed: 202.58 MiB/s
Uploaded backup catalog (1.802 MiB)
Duration: 14.57s
End Time: Mon Nov  6 11:52:58 2023

$ ./target/release/proxmox-backup-client backup --compression=zstd,3 root.pxar:PATH/linux --repository=pbs:linux-zstd3
Starting backup: host/hostname/2023-11-06T10:53:23Z
Client name: hostname
Starting backup protocol: Mon Nov  6 11:53:23 2023
No previous manifest available.
Upload directory 'PATH/linux' to 'pbs:linux-zstd3' as root.pxar.didx
root.pxar: had to backup 1.55 GiB of 1.55 GiB (compressed 449.656 MiB) in 9.34s
root.pxar: average backup speed: 169.891 MiB/s
Uploaded backup catalog (1.802 MiB)
Duration: 15.80s
End Time: Mon Nov  6 11:53:39 2023

$ ./target/release/proxmox-backup-client backup --compression=zstd,5 root.pxar:PATH/linux --repository=pbs:linux-zstd5
Starting backup: host/hostname/2023-11-06T10:53:50Z
Client name: hostname
Starting backup protocol: Mon Nov  6 11:53:50 2023
No previous manifest available.
Upload directory 'PATH/linux' to 'pbs:linux-zstd5' as root.pxar.didx
root.pxar: had to backup 1.55 GiB of 1.55 GiB (compressed 429.919 MiB) in 16.78s
root.pxar: average backup speed: 94.617 MiB/s
Uploaded backup catalog (1.802 MiB)
Duration: 22.83s
End Time: Mon Nov  6 11:54:13 2023

$ ./target/release/proxmox-backup-client backup --compression=none root.pxar:PATH/val2017 --repository=pbs:coco-none
Starting backup: host/hostname/2023-11-06T10:54:42Z
Client name: hostname
Starting backup protocol: Mon Nov  6 11:54:42 2023
No previous manifest available.
Upload directory 'PATH/val2017' to 'pbs:coco-none' as root.pxar.didx
root.pxar: had to backup 777.579 MiB of 777.579 MiB (compressed 777.581 MiB) in 4.09s
root.pxar: average backup speed: 190.242 MiB/s
Uploaded backup catalog (126.99 KiB)
Duration: 14.70s
End Time: Mon Nov  6 11:54:56 2023

$ ./target/release/proxmox-backup-client backup --compression=zstd,1 root.pxar:PATH/val2017 --repository=pbs:coco-zstd1
Starting backup: host/hostname/2023-11-06T10:55:06Z
Client name: hostname
Starting backup protocol: Mon Nov  6 11:55:06 2023
No previous manifest available.
Upload directory 'PATH/val2017' to 'pbs:coco-zstd1' as root.pxar.didx
root.pxar: had to backup 777.579 MiB of 777.579 MiB (compressed 772.769 MiB) in 4.52s
root.pxar: average backup speed: 172.011 MiB/s
Uploaded backup catalog (126.99 KiB)
Duration: 14.98s
End Time: Mon Nov  6 11:55:21 2023

$ ./target/release/proxmox-backup-client backup --compression=zstd,3 root.pxar:PATH/val2017 --repository=pbs:coco-zstd3
Starting backup: host/hostname/2023-11-06T10:55:27Z
Client name: hostname
Starting backup protocol: Mon Nov  6 11:55:27 2023
No previous manifest available.
Upload directory 'PATH/val2017' to 'pbs:coco-zstd3' as root.pxar.didx
root.pxar: had to backup 777.579 MiB of 777.579 MiB (compressed 770.425 MiB) in 5.15s
root.pxar: average backup speed: 150.85 MiB/s
Uploaded backup catalog (126.99 KiB)
Duration: 14.58s
End Time: Mon Nov  6 11:55:42 2023

$ ./target/release/proxmox-backup-client backup --compression=zstd,5 root.pxar:PATH/val2017 --repository=pbs:coco-zstd5
Starting backup: host/hostname/2023-11-06T10:55:48Z
Client name: hostname
Starting backup protocol: Mon Nov  6 11:55:48 2023
No previous manifest available.
Upload directory 'PATH/val2017' to 'pbs:coco-zstd5' as root.pxar.didx
root.pxar: had to backup 777.579 MiB of 777.579 MiB (compressed 770.388 MiB) in 8.05s
root.pxar: average backup speed: 96.551 MiB/s
Uploaded backup catalog (126.99 KiB)
Duration: 18.82s
End Time: Mon Nov  6 11:56:07 2023
```

Do note that the Proxmox Backup Server used for the test is running in a
VM stored in a Samsung 860 QVO.

--
Maximiliano