[pve-devel] Improve container backup speed dramatically (factor 100-1000)

[Carsten Härle]

Container backup is very slow compared to VM backup. I have a 500 GB container (sftp server) with minimal changing files, but even the incremental bakcups take 2 hours with heavy disk activity. Almost nothing is transfered to the backup server. It seems that it it reads the whole container everytime, without any optimization. Before I did backup with zfs send it there it took only a couple of seconds or minutes for every didfferencal backup. 
 
See discussion here: https://forum.proxmox.com/threads/no-differantial-container-backup-with-big-containers.75676/#post-338868 <https://forum.proxmox.com/threads/no-differantial-container-backup-with-big-containers.75676/#post-338868> 
 
PBS is not storage agnostic but uses underlying snapshot feature according to the documentation: For container, the underlying snapshot feature of the file system ARE used, it already uses ZFS feature. 
https://pve.proxmox.com/wiki/Backup_and_Restore <https://pve.proxmox.com/wiki/Backup_and_Restore> 
 
For zfs file systems the set of changed file between snapshots can easy be displayed with "zfs diff", so PBS should use this feature to speed up large container backups dramatically. In my case it would be faktor about factor 1000! Alternatively the snapshots data can be accessed via the hidden .zfs directories, so PBS has knows which files changed and has access to the old and the new data.

 

https://bugzilla.proxmox.com/show_bug.cgi?id=3138