[pve-devel] [RFC qemu-server] increase timeout from guest-fsfreeze-freeze

[Thomas Lamprecht]

On 11/23/2016 11:40 AM, Thomas Lamprecht wrote:
> The qmp command 'guest-fsfreeze-freeze' issues in linux a FIFREEZE
> ioctl call on all mounted guest FS.
> This ioctl call locks the filesystem and gets it into an consistent
> state. For this all caches must be synced after blocking new writes
> to the FS, which may need a relative long time, especially under high
> IO load on the backing storage.
>
> In windows a VSS (Volume Shadow Copy Service) request_freeze will
> issued. As of the closed Windows nature the exact mechanisms cannot
> be checked but some microsoft blog posts and other forum post suggest
> that it should return fast but certain workloads can still trigger a
> long delay resulting an similar problems.
>
> Thus try to minimize the error probability and increase the timeout
> significantly.
> We use 60 minutes as timeout as this seems a limit which should not
> get trespassed in a somewhat healthy system.
>
> See:
> https://forum.proxmox.com/threads/22192/
>
> see the 'freeze_super' and 'thaw_super' function in fs/super.c from
> the linux kernel tree for more details on the freeze behavior in
> Linux guests.
>
> Signed-off-by: Thomas Lamprecht <t.lamprecht at proxmox.com>
> ---
>
> The 60 mins are naturally questionable, imo it would be better to
> remove the timeout entirely from this function as it seems that it
> can only cause harm. Thats the reason I choose such a high value.
>
> Over 60 minutes of syncing should normally (!) never be the case, on
> the other hand. With 60 minutes of sync time a guest can sync 450
> megabyte if he only has 125 kilobyte/s (1 mbit/s) bandwidth or 4.5 gigabyte
> if he has 1.25 megabyte/s (10 mbit) bandwidth.
>
>
>   PVE/QMPClient.pm | 9 +++++++--
>   1 file changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/PVE/QMPClient.pm b/PVE/QMPClient.pm
> index a527ffc..78111ac 100755
> --- a/PVE/QMPClient.pm
> +++ b/PVE/QMPClient.pm
> @@ -107,8 +107,13 @@ sub cmd {
>   	    $timeout = 60*60; # 1 hour
>   	} elsif ($cmd->{execute} =~ m/^(eject|change)/) {
>   	    $timeout = 60; # note: cdrom mount command is slow
> -	} elsif ($cmd->{execute} eq 'guest-fsfreeze-freeze' ||
> -		 $cmd->{execute} eq 'guest-fsfreeze-thaw') {
> +	} elsif ($cmd->{execute} eq 'guest-fsfreeze-freeze') {
> +	    # freeze syncs all guest FS, if we kill it it stays in an unfreezable
> +	    # locked state with high probability, so use an generous timeout
> +	    $timeout = 60*60; # 10 mins

The comment should be 1 hour not 10 minutes, obviously. :/
I increased it to an hour to be sure.

> +	} elsif ($cmd->{execute} eq 'guest-fsfreeze-thaw') {
> +	    # thaw has no possible long blocking actions, either it returns
> +	    # instantly or never (dead locked)
>   	    $timeout = 10;
>   	} elsif ($cmd->{execute} eq 'savevm-start' ||
>   		 $cmd->{execute} eq 'savevm-end' ||