[pbs-devel] [RFC proxmox-backup 0/4] implement trash can for snapshots

[Christian Ebner]

In an effort to simplify the GC phase 1 logic introduced by commit
cb9814e3 ("garbage collection: fix rare race in chunk marking phase")
this patch series implement a trash can functionality for snapshots.

The main intention is to allow snapshot's index files, pruned while
ongoing phase 1 of garbage collection, to be read and their chunks
marked as in use as well. This will allow to get rid of the currently
implemented and rather complex retry looping logic, which could in
theory lead to failing GC or backups when trying to lock the whole
group exclusively following the 10-th retry.

To achieve this, pruning of snapshots does not remove them
immediately, but rather moves them to a `.trash` subfolder in the
datastores base directory. This directory will then be cleared before
starting of GC phase 1, meaning that any index file could be restored
until the next GC run.

This however comes with it's own set of issues, therefore sending
these patches as RFC for now. Open questions and known limitations
are:
- Pruning does not cleanup any space, on the contrary it might
  require additional space on COW filesystem. Should there be a flag
  to bypass the trash, also given that sometimes users truly want to
  remove a snapshot immediately? Although that would re-introduce the
  issue with new snapshot ceration and concurrent GC on a last
  snapshot.
- Prune + sync + prune might lead to the same snapshot being pruned
  multiple times, currently any second prune on a snapshot would
  fail. Should this overwrite the trashed snapshot?
- GC might now read the same index twice, once before it was pruned
  followed by a prune while phase 1 is still ongoing and the second
  time as read from the trash. Not really an issue, but rather a
  limitation.
- Further issues I'm currently overlooking

Christian Ebner (4):
  datastore: always skip over base directory when listing index files
  datastore: allow to specify sub-directory for index file listing
  datastore: move snapshots to trash folder on destroy
  garbage collection: read pruned snapshot index files from trash

 pbs-datastore/src/backup_info.rs |  14 ++-
 pbs-datastore/src/datastore.rs   | 158 +++++++++++++++----------------
 2 files changed, 89 insertions(+), 83 deletions(-)

-- 
2.39.5