[pbs-devel] [PATCH proxmox{, -backup} v3 0/9] parallelize chunk reads in verification

[Christian Ebner]

On 11/10/25 9:44 AM, Nicolas Frey wrote:
> This patch series aims to expand on Dominik's series [0] written for
> pbs 3, parallelizing chunk reads in `VerifyWorker` using a seperate
> thread pool from the verification.
> 
> The number of threads was previously hard-coded, but is now
> configurable via the API and GUI with new properties called
> `{read,verify}-threads`, similarly to tape backups.
> 
> The number of threads should also be configurable through tuning
> options or datastore config (as discussed by Chris & Thomas on list),
> which can be added in a follow up patch series.
> 
> In my local tests I measured the following speed difference:
> verified a single snapshot with ~32 GiB (4x the RAM size) with 4
> cores (just changing the read threads)
> 
> 1 thread:    ~440MiB/s
> 2 threads:   ~780MiB/s
> 4 threads:   ~1140MiB/s
> 
> [0] https://lore.proxmox.com/pbs-devel/20250707132706.2854973-1-d.csapak@proxmox.com/#t
> 
> Changes since v2:
> * split move to parallel handler into 2 seperate commits
> * add constructor to `IndexVerifyState` and clean up nits
> 
> Changes since v1, thanks to Chris:
> * define dedicated schema for {worker,read,verify}-threads
> * rebase proxmox-backup
> * introduce new state struct `IndexVerifyState` to reduce the amount
>    of Arc clones and overall better bundling
> * adjust update endpoint and UI to the new properties
Series is looking good already, only a few smaller thing need some 
adaption (see individual patches).

One major thing which should be however included is a short 
documentation of these parameters and what they do.

Tested the configs are set as expected via cli/ui, default values are 
cleared, the logs expose the number of used reader/verify threads, 
parameters are actually used for the verification job.

Consider the whole series:

Tested-by: Christian Ebner <c.ebner at proxmox.com>