[pbs-devel] [PATCH v4 proxmox-backup 4/5] garbage collection: generate index file list via datastore iterators

[Thomas Lamprecht]

Am 21.03.25 um 10:32 schrieb Christian Ebner:
> Instead of iterating over all index files found in the datastore in
> an unstructured manner, use the datastore iterators to logically
> iterate over them as other datastore operations will.
> 
> This allows to better distinguish index files in unexpected locations
> from ones in their expected location, warning the user of unexpected
> ones to allow to act on possible missconfigurations. Further, this
> will allow to integrate marking of snapshots with missing chunks as
> incomplete/corrupt more easily.
> 
> This now iterates twice over the index files, as indices in
> unexpected locations are still considered by generating the list of
> all index files to be found in the datastore and removing regular
> index files from that list, leaving unexpected ones behind.

Can you also describe the rough changes in terms of time and (peak)
memory used? As replied to patch 5/5 it's probably not too important
to get the perfect reference setup for evaluating this, but to have
one that isn't an obvious extreme (no snapshots, the same snapshots
a hundred times, ...) and describe that setup.

It might be also nice to mention that this is in preparation to get
better cache reuse due to processing snapshots from the same group
closely together.