[pve-devel] Serious problems with the PVE Cluster

[Cesar Peschiera]

Thanks Eric for your answers.

But Dietmar said:
Well, this was a known issue, but should be fixed now

@for some developer:
On any case, is possible modify the "backup code" and the "PVE GUI" 
tag/backup for that the PVE GUI shows "pre-backup" and "post-backup", in 
this mode will be very easy add additionals scripts, and them will run in 
mode consecutive (Veeam Backup for VMware and Hyper-V have these options)? 
.. if I were a developer, would gladly do that contribution. :-(

Thanks again for all.

Best regards
Cesar

> ----- Original Message ----- 
> From: "Eric Blevins" <eric at netwalk.com>
> To: <pve-devel at pve.proxmox.com>
> Sent: Thursday, January 30, 2014 11:47 AM
> Subject: Re: [pve-devel] Serious problems with the PVE Cluster
>
>
>> What you describe does not seem like a Proxmox specific problem to me.
>>
>> You turned off the NFS server without dismounting the volumes from the
>> nodes.
>> This causes IO to those NFS volumes to stall.
>>
>> Proxmox does periodically check the backup directories, so there is
>> consistent IO to them.
>>
>> Since that IO cannot complete, it causes processes to hang.
>> I've even seen Linux not perform IO to local disks when IO to NFS is
>> stalled for a period of time.
>> I am sure you can envision the horrible problems this can cause.
>>
>> Using NFS soft mount might help prevent this problem but that can also
>> cause corrupted data.
>>
>> My suggestion to help avoid this is to use a vzdump hook script to mount
>> the NFS volume only when performing a backup then dismount it at
>> completion of backup. Better yet, setup HA NFS.
>>
>>
>>
>>
>> On 01/29/2014 10:33 PM, Cesar Peschiera wrote:
>>> Serious problems with the PVE Cluster
>>> ----------------------------------------
>>>
>>> @any developer that can help in the code:
>>>
>>> I had problems with 2 of 5 PVE Hosts in a PVE cluster when the NFS
>>> Backup Server  was shutdown manually (without  that "KVM Live Backup" is
>>> running in the PVE Hosts),
>>>
>>> The symptom was:
>>> 2 PVE Nodes were disconnected suddenly of PVE Cluster
>>> The PVE GUI shows leds in red for the nodes without connection to PVE
>>> Cluster
>>>
>>> To return to normal operation:
>>> Only was necessary start the NFS Backup Server
>>>
>>> I mean two stuff:
>>> 1- If "KVM Live Backup" is running in the PVE hosts while that NFS
>>> Backup Server is shutdown suddenly, the problem would have been more
>>> serious.
>>> 2- The PVE Cluster not must depend of NFS Backup Server to run
>>> correctly, this situation is "VERY SERIOUS"
>>>
>>> For these reasons i think it will be necessary to correct the code of
>>> PVE Cluster
>>>
>>> Awaiting a answer, i say see you soon
>>>
>>> Best regards
>>> Cesar
>>>
>>>
>>> ----- Part of Original Message ----- From: "Alexandre DERUMIER"
>>> <aderumier at odiso.com>
>>> To: "Cesar Peschiera" <brain at click.com.py>
>>> Cc: <pve-devel at pve.proxmox.com>
>>> Sent: Wednesday, January 29, 2014 2:31 AM
>>> Subject: Re: [pve-devel] KVM Live Backup performance
>>>
>>>
>>>>> And the fifth question:
>>>>> What will happen if this NFS Server suddenly decomposes while "KVM
>>>>> Live
>>>>> Backup" is running?
>>>
>>> mmm, good question....I don't known what happen when backup job is
>>> hanging because of unavailable storage...
>>>
>>> _______________________________________________
>>> pve-devel mailing list
>>> pve-devel at pve.proxmox.com
>>> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>>
>> _______________________________________________
>> pve-devel mailing list
>> pve-devel at pve.proxmox.com
>> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>>
>