[pbs-devel] superseded: [PATCH proxmox{, -backup} 0/5] parallelize chunk reads in verification

[Nicolas Frey]

Superseded-by: https://lore.proxmox.com/pbs-devel/20251106161316.528349-1-n.frey@proxmox.com/T/#t
On 11/5/25 4:51 PM, Nicolas Frey wrote:
> This patch series aims to expand on Dominik's series [0] written for
> pbs 3, parallelizing chunk reads in `VerifyWorker` using a seperate
> thread pool from the verification.
> 
> The number of threads was previously hard-coded, but is now
> configurable via the API and GUI with a new property called
> `worker-threads`, similarly to tape backups.
> 
> The number of `worker-threads` only controls the thread pool for
> reading, though if it makes sense to reuse this for the verification
> pool, it could be adjusted to do so too.
> 
> In my local tests I measured the following speed difference:
> verified a single snapshot with ~32 GiB (4x the RAM size) with 4 cores
> 
> 1 thread:    ~440MiB/s
> 2 threads:   ~780MiB/s
> 4 threads:   ~1140MiB/s
> 
> [0] https://lore.proxmox.com/pbs-devel/20250707132706.2854973-1-d.csapak@proxmox.com/#t
> 
> proxmox:
> 
> Nicolas Frey (1):
>   pbs-api-types: jobs: verify: add worker-threads to
>     VerificationJobSetup
> 
>  pbs-api-types/src/jobs.rs | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> 
> proxmox-backup:
> 
> Nicolas Frey (4):
>   api: verify: move chunk loading into parallel handler
>   api: verify: use worker-threads to determine the number of threads to
>     use
>   api: verify: add worker-threads to update endpoint
>   ui: verify: add option to set number of threads for job
> 
>  src/api2/admin/datastore.rs    |  13 +++-
>  src/api2/backup/environment.rs |   2 +-
>  src/api2/config/verify.rs      |   8 +++
>  src/backup/verify.rs           | 123 +++++++++++++++++++++------------
>  src/server/verify_job.rs       |   3 +-
>  www/window/VerifyAll.js        |  12 ++++
>  www/window/VerifyJobEdit.js    |  13 ++++
>  7 files changed, 125 insertions(+), 49 deletions(-)
> 
> 
> Summary over all repositories:
>   8 files changed, 135 insertions(+), 49 deletions(-)
>