[PVE-User] Backup server: Backup not finished because of network outage, can failing backup potentially cause read-only root FS?

[Quenten]

I had this a while back as well! The remote server I backed up to was 
offline for a while because of an internet outage there, meaning the 
backups failed naturally, that's fine. But like you said it somehow 
caused read-only disks! Super annoying and very disruptive.

My backup provider and I also noticed that after I filled up the disk on 
her backup server, the entire backup server just crashed and died. She 
had to completely reformat the entire drive and we had to back our stuff 
up again.

On 01-12-2023 05:38 PM, Benjamin Hofer wrote:
> Dear Proxmox community,
> 
> we had a 13 minutes firewall outage. During that time, there was an 
> hourly
> VM backup running.
> The backup didn't finish as the Proxmox VE node couldn't reach the 
> backup
> server.
> I see the "GET" API request (return code 101) that starts the backup in 
> the
> PBS API logs. The HTTP "PUT" to transfer the backup is missing.
> Unfortunately the backup's task log is missing. It obviously got 
> rotated.
> During that time, the root filesystem of the VM changed to read-only.
> 
> To find out if things are related or not, it'll be helpful for me to 
> answer
> the following questions:
> 
> 1) What happens when a Proxmox backup is started and PBS is not 
> reachable
> anymore to transfer the backup?
> 2) In general, is there a (small) chance of a (failing) VM backup 
> causing
> the filesystem to be read-only afterwards? I read that qemu-guest-agent
> calls fsfreeze during Proxmox VM backup. What are the impacts of 
> fsfreeze?
> How are errors handled? Are there other potential risks?
> 
> Here's the VM config:
> 
> agent: 1
> bios: seabios
> boot: cdn
> bootdisk: scsi0
> cipassword: **********
> ciuser: cloudadmin
> cores: 8
> cpu: Skylake-Server-noTSX-IBRS
> description: VM Description
> hotplug: network,disk,usb
> kvm: 1
> machine: q35
> memory: 32768
> name: vm-name
> net0: virtio=26:4B:29:B1:71:89,bridge=vmbr0,tag=37
> numa: 0
> onboot: 0
> ostype: l26
> protection: 1
> rng0: source=/dev/urandom
> scsi0: ceph:vm-108-disk-0,size=255180M,ssd=1
> scsihw: virtio-scsi-single
> serial0: socket
> smbios1: uuid=e8736d61-9a25-4f89-9f6d-a84ef25c42cc
> sockets: 1
> tablet: 1
> vga: qxl
> vmgenid: 9d3233e9-6bc1-43e8-be50-39597b6a5034
> 
> 
> I highly appreciate any answers and thoughts!
> 
> All the best
> 
> Benjamin
> _______________________________________________
> pve-user mailing list
> pve-user at lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user