[PVE-User] storage migration failed: error with cfs lock 'storage-nfs-scratch': unable to create image: got lock timeout - aborting command

[Fabian Grünbichler]

On September 15, 2021 4:47 pm, Marco Gaiarin wrote:
> Mandi! Fabian Grünbichler
>   In chel di` si favelave...
> 
>> this is an issue with certain shared-storage operations in PVE - they have to 
>> happen under a pmxcfs-lock, which has a hard timeout. if the operation 
>> takes too long, the lock will run into the timeout, and the operation 
>> fail.
> 
> OK. Good to know. but...
> 
> 
> Mandi! Christoph Weber
>   In chel di` si favelave...
> 
>> I think this thread might be relevant
>>   https://forum.proxmox.com/threads/error-with-cfs-lock-unable-to-create-image-got-lock-timeout-aborting-command.65786/
> 
> ...seems i have exactly the same trouble, doing some more tests seems
> that timeout does not happen if i use RAW, but only for QCOW; but in
> the temporary NFS storage i've not space for the RAW disk...

the problem (as described in the patch I linked earlier) is that for 
qcow2, we currently always allocate the metadata for the qcow2 file. if 
the image file is big enough, and the storage slow enough, this can take 
too long. for raw there is no metadata (well there is, but it does not 
scale with the size of the file ;))

the patches allow selecting no pre-allocation for storages where this is 
an issue - it basically trades off a bit of a performance hit when the 
image file is filled with data against more/less work when initially 
creating the image file.

> In this link someone say:
> 
> 	you can manually create the image (with qemu-img create and then rescan to reference it as unused volume in the configuration
> 
> but i need to move the disk, not create it...

a manual offline move would also be possible, it boils down to:
- create new volume (qemu-img create)
- qemu-img convert old volume to new volume
- change references in guest config to point to new volume
- delete old volume or add it back as unused to the guest config (the 
  latter happens automatically if you do a rescan)

a manual online move should only be done if you really understand the 
machinery involved, but it is also an option in theory.

last, you could temporarily switch out the hardcoded
'preallocation=metadata' in /usr/share/perl5/PVE/Storage/Plugin.pm with 
'preallocation=off', then reload pveproxy and pvedaemon. running 'apt 
install --reinstall libpve-storage-perl' reverts to the original code 
(either after you're done, or if something goes wrong).

obviously all of this should be carefully tested with non-production 
images/guests/systems first, as you are leaving supported/tested 
territory!