[pve-devel] [PATCH-SERIES v3] fix #2649: introduce prune-backups property for storages supporting backups

[Fabian Ebner]

Patch #12 is the 'real' change needed for the manager to get
'prune-backups' working. Patch #13 gets rid of 'maxfiles' internally,
re-using 'prune-backups' instead.
Most other new patches/changes are in preparation for those two,
to make it possible to translate maxfiles to keep-last and not have
too much special case handling for the '--dumpdir' case.

Changes from v2:
    * Introduce checks in archive_remove
    * Improve readability with ctimes in tests

archive_info has been extended even further to include an untainted filename
and for backups with standard names, the expected log filename.

A few tests for the prune logic are included.

Non-standard backups are ignored when pruning. I chose 'protected'
to mark those, 'ignore' could also be used. Backups are grouped
by type+ID.

The backup type from PBS is translated: ct->lxc, vm->qemu, so that the
possible values for the API result don't depend on the backend.

Regarding the API endpoints: there's a regex below '{storage}/content',
namely '{volume}', so it's not possible to create endpoints like
'{storage}/content/prunebackups'. Instead, I introduced '{storage}/prunebackups'.


A dependency bump 'manager -> storage' is needed for patches #11-#13.


storage:

Fabian Ebner (7):
  Introduce prune-backups property for directory-based storages
  Extend archive_info to include filename and logfilename
  Add archive_remove
  PBSPlugin: Add print_volid helper
  Add prune_backups to storage API
  Whitespace cleanup
  Add API and pvesm calls for prune_backups

 PVE/API2/Storage/Makefile        |   2 +-
 PVE/API2/Storage/PruneBackups.pm | 153 ++++++++++++++++++++
 PVE/API2/Storage/Status.pm       |  72 +++++-----
 PVE/CLI/pvesm.pm                 |  27 ++++
 PVE/Storage.pm                   | 133 ++++++++++++++++-
 PVE/Storage/CIFSPlugin.pm        |   1 +
 PVE/Storage/CephFSPlugin.pm      |   1 +
 PVE/Storage/DirPlugin.pm         |   5 +-
 PVE/Storage/GlusterfsPlugin.pm   |   5 +-
 PVE/Storage/NFSPlugin.pm         |   5 +-
 PVE/Storage/PBSPlugin.pm         |  64 ++++++++-
 PVE/Storage/Plugin.pm            | 108 +++++++++++++-
 test/archive_info_test.pm        |  22 +++
 test/prune_backups_test.pm       | 239 +++++++++++++++++++++++++++++++
 test/run_plugin_tests.pl         |   1 +
 15 files changed, 786 insertions(+), 52 deletions(-)
 create mode 100644 PVE/API2/Storage/PruneBackups.pm
 create mode 100644 test/prune_backups_test.pm


manager:

Fabian Ebner (6):
  Die if dumpdir and storage are both defined
  storage_info: avoid duplication
  Simplify how maxfiles is determined
  make use of archive_info and archive_remove
  Use prune-backups instead of maxfiles when configured
  Always use prune-backups instead of maxfiles internally

 PVE/VZDump.pm | 117 ++++++++++++++++++++++++++++++--------------------
 1 file changed, 70 insertions(+), 47 deletions(-)

-- 
2.20.1