[pbs-devel] [PATCH proxmox-backup 0/3] reduce GC S3 locking

[Christian Ebner]

On 11/21/25 10:06 AM, Fabian Grünbichler wrote:
> this patch series tries to reduce the number of open locks held by GC,
> in particular in case most objects returned by the S3 backend are
> garbage that need deletion.
> 
> the first patch reduces the number of open locks by at least a factor of
> 10 in the worst case (from up to 1000 to up to 100).
> 
> the second patch just refactors some now common code.
> 
> the third patch tries to reduce the number of delete calls for regular
> GC runs by batching deletes more efficiently.
> 
> Fabian Grünbichler (3):
>    GC: S3: reduce number of open FDs for to-be-deleted objects
>    GC: S3: factor out batch object deletion
>    GC: S3: phase2: delete last partial batch of objects at the very end
> 
>   pbs-datastore/src/datastore.rs | 49 +++++++++++++++++++++++-----------
>   1 file changed, 33 insertions(+), 16 deletions(-)
> 

Apart from the off by one error the patches look good to me and 
significantly reduce the likelihood to run into the soft limit for open 
files. And as a positive side effect, this makes garbage collection even 
more efficient when only a limited number of chunks per list batch has 
to be removed, collecting them into a reduced number of API calls.

With the one issue addressed, consider:

Reviewed-by: Christian Ebner <c.ebner at proxmox.com>
Tested-by: Christian Ebner <c.ebner at proxmox.com>