[pve-devel] [PATCH 00/11] live-restore for PBS snapshots

[Stefan Reiter]

On 1/11/21 4:50 PM, aderumier at odiso.com wrote:
> Hi,
> that's great !
> 
> I'm not sure it's related, but do you have any plan to add support to
> restore/extract files from the vm filesystem directly ?
> if we could make some kind of qemu-nbd + mount loop of the backup
> volume for example.
> 

Unrelated, but yes, we're working on it.

At the moment you can already map snapshots via loop devices using the 
map command, e.g.:
proxmox-backup-client map vm/115/2020-10-13T15:03:12Z drive-scsi0.img
# mapped as /dev/loopN
proxmox-backup-client unmap ...

What's planned is a fully featured file-restore, including a GUI in PVE 
(similar to what's already available for CTs in the PBS interface).

~ Stefan

> 
> This is the main blocking feature for me, instead using my rbd
> import/export for backups.
> 
> Regards,
> 
> Alexandre
>  > Le lundi 11 janvier 2021 à 12:13 +0100, Stefan Reiter a écrit :
>> "live-restore" allows starting a VM immediately from a backup
>> snapshot, no
>> waiting for a long restore process. This is made possible with QEMU
>> backing
>> images, i.e. data is read from the backup which is attached to the VM
>> as a
>> drive, but new data is written to the destination, while a background
>> process
>> ('block-stream') copies over data in a linear fashion as well.
>>
>> QEMU backing images are normally only supported for qcow2 images, but
>> since the
>> destination always starts out empty, we can use a dirty bitmap to
>> achieve the
>> same effect - this is implemented as the 'alloc-track' driver in the
>> 'qemu' part
>> of the series.
>>
>> The Rust part of the equation is adjusted to provide (quiet a lot)
>> more caching,
>> as mixing random read/write from the guest with the linear reads from
>> the
>> background process (both of which may use read sizes smaller or
>> bigger than a
>> single chunk) would thrash performance without large buffers.
>>
>> I've marked the feature as 'experimental' in the GUI for now, as I'm
>> sure there
>> are a lot of edge cases I've missed to test, and there's also the
>> possibility of
>> data loss, since anything the VM writes during the restore is removed
>> if the
>> restore fails.
>>
>>
>> qemu: Stefan Reiter (3):
>>    PVE: explicitly add libuuid as linking dependency
>>    PVE: block/pbs: fast-path reads without allocation if possible
>>    block: add alloc-track driver
>>
>>   Makefile.objs       |   2 +
>>   block/Makefile.objs |   1 +
>>   block/alloc-track.c | 319
>> ++++++++++++++++++++++++++++++++++++++++++++
>>   block/pbs.c         |  17 ++-
>>   4 files changed, 336 insertions(+), 3 deletions(-)
>>   create mode 100644 block/alloc-track.c
>>
>> proxmox-backup: Stefan Reiter (1):
>>    RemoteChunkReader: add LRU cached variant
>>
>>   src/bin/proxmox_backup_client/mount.rs |  4 +-
>>   src/client/remote_chunk_reader.rs      | 77 ++++++++++++++++++++----
>> --
>>   2 files changed, 62 insertions(+), 19 deletions(-)
>>
>> proxmox-backup-qemu: Stefan Reiter (1):
>>    access: use bigger cache and LRU chunk reader
>>
>>   src/restore.rs | 5 +++--
>>   1 file changed, 3 insertions(+), 2 deletions(-)
>>
>> qemu-server: Stefan Reiter (5):
>>    make qemu_drive_mirror_monitor more generic
>>    cfg2cmd: allow PBS snapshots as backing files for drives
>>    enable live-restore for PBS
>>    extract register_qmeventd_handle to QemuServer.pm
>>    live-restore: register qmeventd handle
>>
>>   PVE/API2/Qemu.pm         |  15 ++-
>>   PVE/QemuServer.pm        | 263 ++++++++++++++++++++++++++++++++-----
>> --
>>   PVE/VZDump/QemuServer.pm |  32 +----
>>   3 files changed, 233 insertions(+), 77 deletions(-)
>>
>> manager: Stefan Reiter (1):
>>    ui: restore: add live-restore checkbox
>>
>>   www/manager6/grid/BackupView.js    |  8 +++++--
>>   www/manager6/storage/BackupView.js |  7 ++++--
>>   www/manager6/window/Restore.js     | 38
>> +++++++++++++++++++++++++++++-
>>   3 files changed, 48 insertions(+), 5 deletions(-)
>>
> 
> 
> 
> _______________________________________________
> pve-devel mailing list
> pve-devel at lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>