[pve-devel] [RFC storage v2 10/25] plugin: introduce new_backup_provider() method

[Fabian Grünbichler]

high-level comment: nicely documented! but wow is POD annoying to read
in source form ;)

On August 13, 2024 3:28 pm, Fiona Ebner wrote:
> The new_backup_provider() method can be used by storage plugins for
> external backup providers. If the method returns a provider, Proxmox
> VE will use callbacks to that provider for backups and restore instead
> of using its usual backup/restore mechanisms.
> 
> API age and version are both bumped.
> 
> The backup provider API is split into two parts, both of which again
> need different implementations for VM and LXC guests:
> 
> 1. Backup API
> 
> There are two hook callback functions, namely:
> 1. job_hook() is called during the start/end/abort phases of the
>    whole backup job.
> 2. backup_hook() is called during the start/end/abort phases of the
>    backup of an individual guest.
> 
> The backup_get_mechanism() method is used to decide on the backup
> mechanism. Currently, 'block-device' or 'nbd' for VMs, and 'directory'
> for containers is possible. The method also let's the plugin indicate
> whether to use a bitmap for incremental VM backup or not. It is enough
> to implement one mechanism for VMs and one mechanism for containers.
> 
> Next, there are methods for backing up the guest's configuration and
> data, backup_vm() for VM backup and backup_container() for container
> backup.
> 
> Finally, some helpers like getting the provider name or volume ID for
> the backup target, as well as for handling the backup log.
> 
> 1.1 Backup Mechanisms
> 
> VM:
> 
> Access to the data on the VM's disk from the time the backup started
> is made available via a so-called "snapshot access". This is either
> the full image, or in case a bitmap is used, the dirty parts of the
> image since the last time the bitmap was used for a successful backup.
> Reading outside of the dirty parts will result in an error. After
> backing up each part of the disk, it should be discarded in the export
> to avoid unnecessary space usage on the Proxmox VE side (there is an
> associated fleecing image).
> 
> VM mechanism 'block-device':
> 
> The snapshot access is exposed as a block device. If used, a bitmap is
> passed along.
> 
> VM mechanism 'nbd':
> 
> The snapshot access and, if used, bitmap are exported via NBD.
> 
> Container mechanism 'directory':
> 
> A copy or snapshot of the container's filesystem state is made
> available as a directory.
> 
> 2. Restore API
> 
> The restore_get_mechanism() method is used to decide on the restore
> mechanism. Currently, 'qemu-img' for VMs, and 'directory' or 'tar' for
> containers are possible. It is enough to implement one mechanism for
> VMs and one mechanism for containers.
> 
> Next, methods for extracting the guest and firewall configuration and
> the implementations of the restore mechanism via a pair of methods: an
> init method, for making the data available to Proxmox VE and a cleanup
> method that is called after restore.
> 
> For VMs, there also is a restore_vm_get_device_info() helper required,
> to get the disks included in the backup and their sizes.
> 
> 2.1. Restore Mechanisms
> 
> VM mechanism 'qemu-img':
> 
> The backup provider gives a path to the disk image that will be
> restored. The path needs to be something 'qemu-img' can deal with,
> e.g. can also be an NBD URI or similar.
> 
> Container mechanism 'directory':
> 
> The backup provider gives the path to a directory with the full
> filesystem structure of the container.
> 
> Container mechanism 'directory':
> 
> The backup provider gives the path to a (potentially compressed) tar
> archive with the full filesystem structure of the container.

same as in the cover letter ;) base on the code here I assume the second
one should be tar. I wonder whether just tar wouldn't be enough (easier
to not get ACLs/xattrs/ownership/.. right)?

> 
> See the PVE::BackupProvider::Plugin module for the full API
> documentation.
> 
> Signed-off-by: Fiona Ebner <f.ebner at proxmox.com>
> ---
> 

[snip..]

> diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
> index 6444390..d5b76ae 100644
> --- a/src/PVE/Storage/Plugin.pm
> +++ b/src/PVE/Storage/Plugin.pm
> @@ -1755,6 +1755,21 @@ sub rename_volume {
>      return "${storeid}:${base}${target_vmid}/${target_volname}";
>  }
>  
> +# Used by storage plugins for external backup providers. See PVE::BackupProvider::Plugin for the API
> +# the provider needs to implement.
> +#
> +# $scfg - the storage configuration
> +# $storeid - the storage ID
> +# $log_function($log_level, $message) - this log function can be used to write to the backup task
> +#   log in Proxmox VE. $log_level is 'info', 'warn' or 'err', $message is the message to be printed.
> +#
> +# Returns a blessed reference to the backup provider class.
> +sub new_backup_provider {
> +    my ($class, $scfg, $storeid, $log_function) = @_;
> +
> +    return;
> +}

would it maybe make sense to make this a "die implement me" and make the
opt-in via the storage plugin features? it would be more in line with
what we do in other parts and less subtle..

> +
>  sub config_aware_base_mkdir {
>      my ($class, $scfg, $path) = @_;
>  
> -- 
> 2.39.2
> 
> 
> 
> _______________________________________________
> pve-devel mailing list
> pve-devel at lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
> 
>