[pbs-devel] Verify, Prune & GC sequence

[Thomas Lamprecht]

Hi,

On 30.10.20 10:31, Graeme Seaton wrote:
> If I have verify, prune & GC all set to run daily do the jobs run concurrently or in sequence?

Well, you could just try out ;-)

But yes, they can run in parallel. 

Since PBS allows multiple potentially interfering operations at the same time (e.g., garbage collect,
prune, multiple backup creations (only in separate groups), forget, ...), these need to lock against
each other in certain scenarios. There is no overarching global lock though, instead always the finest
grained lock possible is used, because running these operations concurrently is treated as a feature
on its own.

Still, while we handle locking fine grained an admin could try to optimize the order of scheduling
such things.

1. Prune, to increase the chance that GC can sweep more data and verify has less to look at.
   This is rather quick, so one could schedule the next probably half an hour afterwards and
   be confident it finishes before 2.

2. GC, as chunks are deleted, which may make verify also faster as it can make the underlying
   FS faster - but that's rather theory.

3. Verify

I think the order of 2. and 3. is not to important, but personally, I'd always schedule pruning
first, can only help.

cheers,
Thomas