[pve-devel] Further improvement to GC

[Christian Ebner]

Hi,

thanks for sharing your ideas!

On 6/8/25 16:25, Kamil Trzciński wrote:
> Thanks for introducing LRU Cache.
> 
> I have another suggestion that we further the sweep phase could be improved:
> 
> 1. The sweep_unused_chunks could use LRU cache to avoid lstatat if
> object is found in cache.

That is indeed true and a good further potential optimization, given 
that cached chunk digest can be considered to have an atime newer than 
the cutoff time. Cached items must and could be accounted differently in 
the GC stats then.

> The lstatat would only be required for potential objects that might
> have been added while running GC cycle, or that might have been
> evicted from LRU cache.
> For big enough LRU caches and the fact that only small amount of
> objects are usually removed this should make the GC almost in-memory.
> 
> 2. The process could add a warning that LRU cache is too small, and
> propose the value big enough to avoid cache eviction based on number
> of chunks observed.

The number of in use chunks can be rather volatile, so just defining a 
suggested cache capacity based on that does not seem to be that ideal to 
me. Also, memory usage by the cache must be taken into account.

But I agree with your points for further improvements and kindly ask you 
to open a new issue for this in our bugtracker [0].

As side note: please use the PBS developer mailing list [1] for 
discussions regarding Proxmox Backup Server.

Best regards,
Chris

[0] https://bugzilla.proxmox.com/
[1] https://lore.proxmox.com/pbs-devel/