[PVE-User] Online storage migration not working (II)

Eneko Lacunza elacunza at binovo.es
Wed Oct 29 17:05:35 CET 2014 

Hi all,

It's been a while and I have been doing tests in the background.

I found some issues were related to faulty drives, but after removing 
them migration issues continued. I had the same issues with PVE 3.2, 
nfs, rbd and local storage.

I also noticed that out office production cluster doesn't seem to suffer 
this problem (PVE 3.3) .

Today I found this post on the forum:
http://forum.proxmox.com/threads/19733-storage-migration-virtio-failed

Short is that in /usr/share/perl5/PVE/QemuServer.pm
changing:
---

      if ($vmiddst == $vmid) {
                 # switch the disk if source and destination are on the same guest
                 vm_mon_cmd($vmid, "block-job-complete", device => "drive-$drive");
      }
---

with
---

      if ($vmiddst == $vmid) {
                 sleep 10;
                 # switch the disk if source and destination are on the same guest
                 vm_mon_cmd($vmid, "block-job-complete", device => "drive-$drive");
      }
---


So adding a sleep, seems to have fixed all the migration problems so far 
(I continue testing while I write this email).

The problem seems related to performance. The problem-suffering cluster 
has 3 nodes each with 1 desktop CPU, 2 GB RAM, 2 relatively old hard 
disks (~60MB/s write on NFS). Network is 1x1Gbit on each node, all on 
the same HP 1gbit switch.

Although a work-around, maybe putting a sleep 1 could help some users 
while not being noticeable for the rest?

Cheers
Eneko


On 13/10/14 15:04, Eneko Lacunza wrote:
> Just tried moving a VM WS2008R2 with virtio v74 from local storage to 
> NFS (2 different servers) and RBD, all failed.
>
> Does anyone with PVE 3.3 have working online storage migration with 
> Windows guests and virtio?
>
> Thanks
> Eneko
>
> On 13/10/14 11:29, Eneko Lacunza wrote:
>> I have tried with virtio v74, but it also fails. All failures so far 
>> with WS2012R2, Debian8beta2 works ok. I'm trying with WS2008R2 now.
>>
>> On 10/10/14 20:41, Eneko Lacunza wrote:
>>>
>>> I think problem is windows virtio v81. Debian 8beta has worked ok; I 
>>> will do further tests on monday.
>>>
>>> El 10/10/2014 17:49, "Luis G. Coralle" <luiscoralle at fi.uncoma.edu.ar 
>>> <mailto:luiscoralle at fi.uncoma.edu.ar>> escribió:
>>>
>>>     Removing snapshots?
>>>
>>>     2014-10-10 8:36 GMT-03:00 Eneko Lacunza <elacunza at binovo.es
>>>     <mailto:elacunza at binovo.es>>:
>>>
>>>
>>>         Yes doing it with VM off works, but I can't make it work
>>>         with VMs on. It used to work in 3.2 and before, something
>>>         must be broken.
>>>
>>>
>>>         On 10/10/14 13:27, Luis G. Coralle wrote:
>>>>         In some cases the same, migrating a hard drive from a
>>>>         remote storage to a local one.
>>>>         I fix this turning off the vm
>>>>
>>>>         2014-10-10 7:37 GMT-03:00 Eneko Lacunza <elacunza at binovo.es
>>>>         <mailto:elacunza at binovo.es>>:
>>>>
>>>>             Hi all,
>>>>
>>>>             I tried local to RBD storage migration with VM running,
>>>>             and it fails like it does with NFS:
>>>>
>>>>             transferred: 53687091200 bytes remaining: 0 bytes
>>>>             total: 53687091200 bytes progression: 100.00 %
>>>>             Removing all snapshots: 100% complete...done.
>>>>             image has watchers - not removing
>>>>             Removing image: 0% complete...failed.
>>>>             rbd: error: image still has watchers
>>>>             TASK ERROR: storage migration failed: mirroring error:
>>>>             VM 103 qmp command 'block-job-complete' failed - The
>>>>             active block job for device 'drive-virtio0' cannot be
>>>>             completed
>>>>
>>>>             I noticed that during migration HTML5 console's key
>>>>             drop-down menu doesn't work (VM doesn't receive keys).
>>>>
>>>>             # pveversion -v
>>>>             proxmox-ve-2.6.32: 3.3-138 (running kernel: 2.6.32-33-pve)
>>>>             pve-manager: 3.3-2 (running version: 3.3-2/995e687e)
>>>>             pve-kernel-2.6.32-32-pve: 2.6.32-136
>>>>             pve-kernel-2.6.32-33-pve: 2.6.32-138
>>>>             lvm2: 2.02.98-pve4
>>>>             clvm: 2.02.98-pve4
>>>>             corosync-pve: 1.4.7-1
>>>>             openais-pve: 1.1.4-3
>>>>             libqb0: 0.11.1-2
>>>>             redhat-cluster-pve: 3.2.0-2
>>>>             resource-agents-pve: 3.9.2-4
>>>>             fence-agents-pve: 4.0.10-1
>>>>             pve-cluster: 3.0-15
>>>>             qemu-server: 3.1-35
>>>>             pve-firmware: 1.1-3
>>>>             libpve-common-perl: 3.0-19
>>>>             libpve-access-control: 3.0-15
>>>>             libpve-storage-perl: 3.0-23
>>>>             pve-libspice-server1: 0.12.4-3
>>>>             vncterm: 1.1-8
>>>>             vzctl: 4.0-1pve6
>>>>             vzprocps: 2.0.11-2
>>>>             vzquota: 3.1-2
>>>>             pve-qemu-kvm: 2.1-9
>>>>             ksm-control-daemon: 1.1-1
>>>>             glusterfs-client: 3.5.2-1
>>>>
>>>>             Is storage migration working for you in PVE 3.3?
>>>>
>>>>             Cheers
>>>>             Eneko
>>>>
>>>>             -- 
>>>>             Zuzendari Teknikoa / Director Técnico
>>>>             Binovo IT Human Project, S.L.
>>>>             Telf. 943575997 <tel:943575997>
>>>>             943493611 <tel:943493611>
>>>>             Astigarraga bidea 2, planta 6 dcha., ofi. 3-2; 20180
>>>>             Oiartzun (Gipuzkoa)
>>>>             www.binovo.es <http://www.binovo.es>
>>>>
>>>>             _______________________________________________
>>>>             pve-user mailing list
>>>>             pve-user at pve.proxmox.com <mailto:pve-user at pve.proxmox.com>
>>>>             http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>>>>
>>>>
>>>>
>>>>
>>>>         -- 
>>>>         Luis G. Coralle
>>>
>>>
>>>         -- 
>>>         Zuzendari Teknikoa / Director Técnico
>>>         Binovo IT Human Project, S.L.
>>>         Telf.943575997  <tel:943575997>
>>>                943493611  <tel:943493611>
>>>         Astigarraga bidea 2, planta 6 dcha., ofi. 3-2; 20180 Oiartzun (Gipuzkoa)
>>>         www.binovo.es  <http://www.binovo.es>
>>>
>>>
>>>         _______________________________________________
>>>         pve-user mailing list
>>>         pve-user at pve.proxmox.com <mailto:pve-user at pve.proxmox.com>
>>>         http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>>>
>>>
>>>
>>>
>>>     -- 
>>>     Luis G. Coralle
>>>
>>
>>
>> -- 
>> Zuzendari Teknikoa / Director Técnico
>> Binovo IT Human Project, S.L.
>> Telf. 943575997
>>        943493611
>> Astigarraga bidea 2, planta 6 dcha., ofi. 3-2; 20180 Oiartzun (Gipuzkoa)
>> www.binovo.es
>
>
> -- 
> Zuzendari Teknikoa / Director Técnico
> Binovo IT Human Project, S.L.
> Telf. 943575997
>        943493611
> Astigarraga bidea 2, planta 6 dcha., ofi. 3-2; 20180 Oiartzun (Gipuzkoa)
> www.binovo.es


-- 
Zuzendari Teknikoa / Director Técnico
Binovo IT Human Project, S.L.
Telf. 943575997
       943493611
Astigarraga bidea 2, planta 6 dcha., ofi. 3-2; 20180 Oiartzun (Gipuzkoa)
www.binovo.es

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.proxmox.com/pipermail/pve-user/attachments/20141029/b9130866/attachment.htm>

More information about the pve-user mailing list