[pve-devel] Further improvement to GC

[Kamil Trzciński]

Thanks. I created in https://bugzilla.proxmox.com/show_bug.cgi?id=6452.

Kamil

On Tue, 10 Jun 2025 at 09:20, Christian Ebner <c.ebner at proxmox.com> wrote:
>
> Hi,
>
> thanks for sharing your ideas!
>
> On 6/8/25 16:25, Kamil Trzciński wrote:
> > Thanks for introducing LRU Cache.
> >
> > I have another suggestion that we further the sweep phase could be improved:
> >
> > 1. The sweep_unused_chunks could use LRU cache to avoid lstatat if
> > object is found in cache.
>
> That is indeed true and a good further potential optimization, given
> that cached chunk digest can be considered to have an atime newer than
> the cutoff time. Cached items must and could be accounted differently in
> the GC stats then.
>
> > The lstatat would only be required for potential objects that might
> > have been added while running GC cycle, or that might have been
> > evicted from LRU cache.
> > For big enough LRU caches and the fact that only small amount of
> > objects are usually removed this should make the GC almost in-memory.
> >
> > 2. The process could add a warning that LRU cache is too small, and
> > propose the value big enough to avoid cache eviction based on number
> > of chunks observed.
>
> The number of in use chunks can be rather volatile, so just defining a
> suggested cache capacity based on that does not seem to be that ideal to
> me. Also, memory usage by the cache must be taken into account.
>
> But I agree with your points for further improvements and kindly ask you
> to open a new issue for this in our bugtracker [0].
>
> As side note: please use the PBS developer mailing list [1] for
> discussions regarding Proxmox Backup Server.
>
> Best regards,
> Chris
>
> [0] https://bugzilla.proxmox.com/
> [1] https://lore.proxmox.com/pbs-devel/
>