[pve-devel] [PATCH storage v4 09/27] plugin: introduce new_backup_provider() method

[Andreas Rogge]

Hi everyone,

sorry for digging up that ancient mail, but I feel that's the best 
starting point for me.
Bareos investigates how to better integrate with Proxmox VE, so users 
can take backups of virtual machines and restore them using Bareos.

Am 14.11.24 um 16:07 schrieb Fiona Ebner:
> The new_backup_provider() method can be used by storage plugins for
> external backup providers. If the method returns a provider, Proxmox
> VE will use callbacks to that provider for backups and restore instead
> of using its usual backup/restore mechanisms.
So we as a backup provider need to provide plugin code that will run as 
part of PVE. If I understand correctly this must be implemented as a 
Perl module. Also, PVE wants to trigger the backup itself.

This introduces a few hurdles for us:
First of all, we don't usually program in Perl, so that's going to be a 
problem for us. We can probably implement something, but we probably 
don't have any developer that can write Perl code that meets any kind of 
professional standard.

To do its job[1], Bareos must be able to schedule backup and restore 
jobs all by itself. If I understood correctly, backups utilizing one of 
the plugins can be run by calling `vzdump`, so that this is at least 
possible.

However, what we've seen other vendors do is that they provide an API 
where you just tell the system what VM you want to back up and then you 
are provided with access to metadata (i.e. VM configuration, disk 
layouts, nvram content, etc) and disk contents.
For restore it is mostly the same: you provide the metadata, get a set 
of raw disks that you can write your data to and finally you just tell 
the system that the VM restore is done.


> The backup provider API is split into two parts, both of which again
> need different implementations for VM and LXC guests:
[...]

> 1.1 Backup Mechanisms
> 
[...]
> VM mechanism 'block-device':
> The snapshot access is exposed as a block device. If used, a bitmap is
> passed along.
> 
> VM mechanism 'nbd':
> The snapshot access and, if used, bitmap are exported via NBD.

That looks reasonable.

> 2. Restore API
> 
> 2.1. Restore Mechanisms
> 
> VM mechanism 'qemu-img':
> 
> The backup provider gives a path to the disk image that will be
> restored. The path needs to be something 'qemu-img' can deal with,
> e.g. can also be an NBD URI or similar.
Um... that has nothing to do with what you provided when we take the 
backup. Is there a reason PVE cannot provide a writeable block device to 
restore to?
For Bareos this requirement would imply that we need an unpleasantly 
large staging area on the PVE node to facilitate a restore: As we can 
only stream the data we'd have to create a temporary disk image that PVE 
can then read. This sounds pretty inefficient - especially when 
comparing with qmrestore's ability to just read read from stdin.

Best Regards,
Andreas


[1] As a backup software it is Bareos' job to make sure the production 
system is as strictly separated from the backup data as possible. While 
we understand how nice tight integration with PVE would be, we see this 
primarily as unnecessary attack surface.
For a nicely integrated solution that can bring back last week's VM 
snapshot with a few clicks, there's Proxmox Backup Server, which works 
great.




-- 
Andreas Rogge                             andreas.rogge at bareos.com
   Bareos GmbH & Co. KG                      Phone: +49 221-630693-86
   http://www.bareos.com

   Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646
   Komplementär: Bareos Verwaltungs-GmbH
   Geschäftsführer: Stephan Dühr, Jörg Steffens, Philipp Storz