[pbs-devel] applied-series: [PATCH v2 proxmox-backup 0/9] drop catalog encoding for split pxar archives

[Fabian Grünbichler]

with two minor follow-ups (using schema for achive-name, and cargo fmt)

On June 7, 2024 1:37 pm, Christian Ebner wrote:
> The catalog stores metadata needed for fast lookups via e.g. the file
> browser for single file restore.
> Since it is now possible to create snapshots with split pxar archives,
> the same metadata can be accessed in a reasonable performat way also via
> the metadata archive, effectively making the catalog unneeded duplicate
> data for this case.
> 
> This patch series therefore allows to lookup directory entries where
> needed via the metadata archive instead of the catalog and conditionally
> drops the catalog encoding for snapshots using split archive encoding.
> 
> The patches adapt not only the `catalog` api endpoint, used by the
> Proxmox Backup WebUI to access the snapshots contents but also the
> proxomox-file-restore, which is used by the Proxmox Virtual Environments
> PBS storage plugin to access and restore contents.
> 
> Tested that regular backups can still be inspected and restored and that
> the same functionality is still available for split archive backups,
> without now having a catalog encoded.
> 
> Functionally was tested with the following path series applied to PVE:
> https://lists.proxmox.com/pipermail/pve-devel/2024-May/064024.html
> 
> Please note: For directories with may entries a noticeable performance
> drop is present, which might be improved upon by optimizing decoding.
> 
> Changes since version 1:
> - Do not base64 encode archive name
> - Split off file restore archive name listing
> 
> The rest of the comments will be addressed as followups as discussed
> off-list.
> 
> Christian Ebner (9):
>   api: datastore: factor out path decoding for catalog
>   api: datastore: move reusable code out of thread
>   client: tools: add helper to lookup `ArchiveEntry`s via pxar
>   api: datastore: conditional lookup for catalog endpoint
>   api: datastore: add optional archive-name to file-restore
>   file-restore: never list ppxar as archive
>   file-restore: fallback to mpxar if catalog not present
>   www: content: lookup via metadata archive instead of catalog
>   client: backup: conditionally write catalog for file level backups
> 
>  pbs-client/src/pxar_backup_stream.rs |  11 ++-
>  pbs-client/src/tools/mod.rs          |  84 +++++++++++++++++++
>  proxmox-backup-client/src/main.rs    |  21 +++--
>  proxmox-file-restore/src/main.rs     |  72 ++++++++++++----
>  src/api2/admin/datastore.rs          | 121 ++++++++++++++++++---------
>  www/datastore/Content.js             |   3 +
>  6 files changed, 243 insertions(+), 69 deletions(-)
> 
> -- 
> 2.39.2
> 
> From 8dffaab3ce2c20b0c376d6c0dc8b3d561499fa24 Mon Sep 17 00:00:00 2001
> From: Christian Ebner <c.ebner at proxmox.com>
> Date: Fri, 7 Jun 2024 11:25:09 +0200
> 
> Christian Ebner (8):
>   api: datastore: factor out path decoding for catalog
>   api: datastore: move reusable code out of thread
>   client: tools: add helper to lookup `ArchiveEntry`s via pxar
>   api: datastore: conditional lookup for catalog endpoint
>   api: datastore: add optional archive-name to file-restore
>   file-restore: fallback to mpxar if catalog not present
>   www: content: lookup via metadata archive instead of catalog
>   client: backup: conditionally write catalog for file level backups
> 
>  pbs-client/src/pxar_backup_stream.rs |  11 ++-
>  pbs-client/src/tools/mod.rs          |  84 +++++++++++++++++
>  proxmox-backup-client/src/main.rs    |  21 +++--
>  proxmox-file-restore/src/main.rs     |  72 +++++++++++----
>  src/api2/admin/datastore.rs          | 129 ++++++++++++++++++---------
>  www/datastore/Content.js             |   3 +
>  6 files changed, 251 insertions(+), 69 deletions(-)
> 
> -- 
> 2.39.2
> 
> 
> 
> _______________________________________________
> pbs-devel mailing list
> pbs-devel at lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
> 
> 
>