[pbs-devel] applied-series: [PATCH vma-to-pbs v6 0/4] add support for bulk import of a dump directory

[Fabian Grünbichler]

thanks!

Quoting Filip Schauer (2024-11-13 16:57:58)
> When a path to a directory is provided in the vma_file argument, try to
> upload all VMA backups in the directory. This also handles compressed
> VMA files, notes and logs. If a vmid is specified with --vmid, only the
> backups of that particular vmid are uploaded.
> 
> Also improve the readability of the log messages to keep track of all
> imported backups.
> 
> Changed since v5:
> * Extend the help text by seperate usages for single VMA import and bulk
>   import
> * Move variables into format strings where possible
> 
> Changed since v4:
> * Switch grouped_vmas from Vec<Vec<>> to HashMap<String, Vec<>>
> * Remove dependency on itertools
> * bail when no backups were found
> * Default to yes on the bulk import confirmation prompt
> * bail on invalid input to the bulk import confirmation prompt
> 
> Changed since v3:
> * Mention in the description of the --vmid argument, that it is required
>   if a single VMA file is provided
> * Construct grouped_vmas in place
> * Add debug logs when gathering files for bulk import
> * Log a summary of the files gathered for bulk import
> * Remove the "confusing VMA file path" error message in the second
>   commit
> * Switch chunk_stats from Arc<Mutex<[u64; 256]>> to
>   Arc<[AtomicU64; 256]> and use fetch_add to atomically increment and
>   fetch the chunk stat
> * Ask for confirmation before bulk import
> * Add --yes option to skip the confirmation prompt
> 
> Changed since v2:
> * Make skipping a VMID on error optional with the --skip-failed option
> * Switch log output from stderr to stdout
> * Bump itertools to 0.13
> 
> Changed since v1:
> * Do not recurse through dump directory
> * Compile regex once before iterating over the files in the dump
>   directory
> * Use extract on regex capture groups
> * Do not use deprecated method `chrono::NaiveDateTime::timestamp`
> * Use proxmox_sys::fs::file_read_optional_string
> * Group VMA files by VMID and continue with next VMID on error
> * Move the BackupVmaToPbsArgs split into its own commit
> * Remove hard coded occurences of 255
> * Use level-based logging instead of println
> 
> Filip Schauer (4):
>   add support for bulk import of a dump directory
>   add option to skip vmids whose backups failed to upload
>   use level-based logging instead of println
>   log device upload progress as a percentage
> 
>  Cargo.toml     |   4 +
>  src/main.rs    | 198 +++++++++++++++++++++++++++++++++++++++++++++----
>  src/vma2pbs.rs | 108 +++++++++++++++++++++------
>  3 files changed, 271 insertions(+), 39 deletions(-)
> 
> -- 
> 2.39.5
> 
> 
> 
> _______________________________________________
> pbs-devel mailing list
> pbs-devel at lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
> 
>