[PVE-User] Problem with QEMU drive-mirror after cancelling VM disk move

Fabian Grünbichler f.gruenbichler at proxmox.com
Tue Mar 31 13:53:00 CEST 2020


On March 31, 2020 12:21 pm, Mikhail wrote:
> Hello,
> 
> On one of our clusters we're seeing issues with VM backup task - the
> backup task fails with the following:
> 
> ERROR: Node 'drive-scsi0' is busy: block device is in use by block job:
> mirror
> INFO: aborting backup job
> ERROR: Backup of VM 123 failed - Node 'drive-scsi0' is busy: block
> device is in use by block job: mirror
> 
> I tried digging some information and it appears to be that there's
> "drive-mirror" job in QEMU/KVM for this particular VM is blocking backup
> process. It is clear to me that this problem started couple weeks ago
> when we attempted to change VM's disk underlying storage, however the
> "Move disk" operation was cancelled manually by administrator at the
> time and backup tasks started failing right after that. I'm not sure
> whether this is Proxmox issue or QEMU/KVM, but I suppose that stopping
> and starting VM from within Proxmox will remove this block, however in
> our case keeping this virtual machine up and running is critical and we
> should avoid even 1-2 min downtime.

yes, shutting the VM down and starting it again gets rid of any leftover 
block-jobs for sure.

> The question is how to remove this drive-mirror block online and how to
> avoid this in the future.

you should be able to manually clean the messup using the QMP/monitor 
interface:

`man qemu-qmp-ref` gives a detailed tour, you probably want
`query-block-jobs` and `query-block`, and then, depending on the output
`block-job-cancel` or `block-job-complete`.

the HMP interface accessible via 'qm monitor <VMID>' has slightly 
different commands: `info block -v`, `info block-jobs` and 
`block_job_cancel`/`block_job_complete` ('_' instead of '-').

feel free to post the output of the query/info commands before deciding 
how to proceed. the complete task log of the failed 'move disk' 
operation would also be interesting, if it is still available.



More information about the pve-user mailing list