[PVE-User] Regression on virtio-net ? (Was: Re: Proxmox VE 4.2 released!)

Gilou contact+dev at gilouweb.com
Fri Apr 29 13:53:17 CEST 2016


Le 29/04/2016 10:31, Alexandre DERUMIER a écrit :
> Ok, so maybe it's a kvm module regression in kernel.
> 
> Just to be sure, can you check that vhost_net module is loaded ?
> 
> # lsmod|grep vhost_net
> 
> and also that you have a process
> 
> [vhost-xxxx], where xxxx is the pid of the current qemu-kvm process

Hi,

# lsmod | grep vhost_net
vhost_net              20480  1
vhost                  32768  1 vhost_net
macvtap                20480  1 vhost_net

root at px3:~# ps aux | grep vhost
root      3308 33.8  3.4 1181136 560584 ?      Sl   13:41   1:59
/usr/bin/kvm -id 3004 [...]
net-pci,mac=32:36:65:35:30:62,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300
root      3330  1.2  0.0      0     0 ?        S    13:41   0:04
[vhost-3308]


So: yes, vhost_net is loaded, and I do have a vhost-3308 associated with
my VM. Yet, the CPU is at 100% for a 1 Gbps traffic :(

@+
Gilles


> 
> 
> 
> 
> and lastly, try to use perf, to see which system calls are using cpu
> 
>> #apt-get install linux-tools-4.4
>>
>> and do a 
>>
>> #perf top
> 
> 
> ----- Mail original -----
> De: "Gilou" <contact+dev at gilouweb.com>
> À: "proxmoxve" <pve-user at pve.proxmox.com>
> Envoyé: Vendredi 29 Avril 2016 08:48:20
> Objet: Re: [PVE-User] Regression on virtio-net ? (Was: Re: Proxmox VE 4.2 released!)
> 
> Le 29/04/2016 00:47, Alexandre DERUMIER a écrit : 
>> Can you try with previous kernel 4.2 ? (and keep proxmox 4.2 ) 
>>
>> I would like to known if it's a regression in qemu or kernel. 
> 
> As I said, I did that, the bug is not there (or not that obvious). On a 
> 4.2.6 kernel, using qemu 2.5, I get ~30% of a core for 1 Gbps, which 
> seems reasonnable. 
> 
> I also tried on VirtualBox/KVM, on Ubuntu 16.04 host, and the bug does 
> not appear there. I can try on qemu probably too, but it seems to me the 
> kernel is at fault here. 
> 
>>
>>
>> also, you can try to install 
>>
>> #apt-get install linux-tools-4.4 
>>
>> and do a 
>>
>> #perf top 
>>
>>
>> you should see which system call use a lot of cpu 
> 
> 
> Here's the result, while spitting ~1 Gbps on the LAN: 
> https://framapic.org/eT9JbqGsqSEK/txwSUJRpiIV3.png 
> On the left, perf top, on the right top with relevant fields, showing 
> all threads. 
> 
> so read_tsc using ~23%, ktime_get ~10%, and single thread (even if the 
> VM has multiple cores) using 100% on the host. So maybe it's not 
> virtio-net directly. I have a lot of CPU used on a E1000 as well, but 
> that didn't strike me as obvious, as I expected this. 
> 
> So you know: 
> # uname -a 
> Linux px1 4.4.6-1-pve #1 SMP Thu Apr 21 11:25:40 CEST 2016 x86_64 GNU/Linux 
> 
> # pveversion -v 
> proxmox-ve: 4.2-48 (running kernel: 4.4.6-1-pve) 
> pve-manager: 4.2-2 (running version: 4.2-2/725d76f0) 
> pve-kernel-4.4.6-1-pve: 4.4.6-48 
> pve-kernel-4.2.6-1-pve: 4.2.6-36 
> lvm2: 2.02.116-pve2 
> corosync-pve: 2.3.5-2 
> libqb0: 1.0-1 
> pve-cluster: 4.0-39 
> qemu-server: 4.0-72 
> pve-firmware: 1.1-8 
> libpve-common-perl: 4.0-59 
> libpve-access-control: 4.0-16 
> libpve-storage-perl: 4.0-50 
> pve-libspice-server1: 0.12.5-2 
> vncterm: 1.2-1 
> pve-qemu-kvm: 2.5-14 
> pve-container: 1.0-62 
> pve-firewall: 2.0-25 
> pve-ha-manager: 1.0-28 
> ksm-control-daemon: 1.2-1 
> glusterfs-client: 3.5.2-2+deb8u1 
> lxc-pve: 1.1.5-7 
> lxcfs: 2.0.0-pve2 
> cgmanager: 0.39-pve1 
> criu: 1.6.0-1 
> zfsutils: 0.6.5-pve9~jessie 
> 
> If I can troubleshoot that further, let me know... 
> 
> Cheers, 
> 
> Gilles Pietri 
> 
> 
>>
>>
>>
>>
>>
>> ----- Mail original ----- 
>> De: "Gilou" <contact+dev at gilouweb.com> 
>> À: "proxmoxve" <pve-user at pve.proxmox.com> 
>> Envoyé: Jeudi 28 Avril 2016 19:31:19 
>> Objet: Re: [PVE-User] Regression on virtio-net ? (Was: Re: Proxmox VE 4.2 released!) 
>>
>> Le 28/04/2016 19:18, Gilou a écrit : 
>>> Le 27/04/2016 15:35, Martin Maurer a écrit : 
>>>> Hi all! 
>>>>
>>>> We are proud to announce the final release of our Proxmox VE 4.2 with 
>>>> the new GUI. 
>>>>
>>>> The Sencha Ext JS 6 framework brings a modern 'flat design' look and 
>>>> feel to the Proxmox VE GUI with a reworked icon set providing 
>>>> consistency and an improved user experience. 
>>>>
>>>> Watch our short introduction video - What's new in Proxmox VE 4.2 
>>>> http://www.proxmox.com/en/training/video-tutorials/item/what-s-new-in-proxmox-ve-4-2 
>>>>
>>>>
>>>> This point release is based on latest and greatest Debian Jessie and a 
>>>> 4.4.6 Linux kernel, based on Ubuntu 16.04 LTS kernel. 
>>>
>>> Hi, 
>>>
>>> I seem to have a regression, and a weird one I see neither on Proxmox 
>>> 4.1 4.2.6's kernel, nor on Ubuntu 4.4.0, or 
>>>
>>> Context: Ubuntu Guest VM using Ubuntu 16.04 on a virtio_net device 
>>> (tested also on Debian 8, and CentOS7), connected on a bridge. 
>>>
>>> on 4.1 : 1 Gbps bandwidth, ~1 GHz of CPU 
>>> on 4.2 : 1 Gbps bandwidth, ~3 GHz of CPU 
>>>
>>> Tested using transfer to /dev/null & iperf, so no disk involved. 
>>>
>>> I do not have the issue on ubuntu 4.4.0 using virtualbox/virtio-net. 
>>>
>>> Haven't tested further, but this is an isssue. 
>>
>> Looked into it, it's really 100% of a core, in the following setups: 
>> - single CPU on the VM 
>> - local or remote traffic (local goes up to 24 Gbps, remote is 1 Gbps) 
>> - no matter what distro (haven't tested Windows) 
>>
>> Has anyone noticed something like this or is this something specific? 
>>
>> Regards, 
>>
>>
>>>
>>> Regards, 
>>> Gilles Pietri 
>>>
>>>
>>>>
>>>> We included as always countless small improvements and bugfixes, like 
>>>> the ability to use Let´s Encrypt certificates. 
>>>>
>>>> Release notes 
>>>> http://pve.proxmox.com/wiki/Roadmap#Proxmox_VE_4.2 
>>>>
>>>> ISO Download 
>>>>
>>>> http://www.proxmox.com/en/downloads 
>>>>
>>>> Alternate ISO download: 
>>>> http://download.proxmox.com/iso/ 
>>>>
>>>> Upgrading 
>>>> http://pve.proxmox.com/wiki/Downloads 
>>>>
>>>> Bugtracker 
>>>> https://bugzilla.proxmox.com 
>>>>
>>>> A big THANK-YOU to our active community for all feedback, testing, bug 
>>>> reporting and patch submissions. 
>>>>
>>>
>>> _______________________________________________ 
>>> pve-user mailing list 
>>> pve-user at pve.proxmox.com 
>>> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user 
>>>
>>
>> _______________________________________________ 
>> pve-user mailing list 
>> pve-user at pve.proxmox.com 
>> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user 
>> _______________________________________________ 
>> pve-user mailing list 
>> pve-user at pve.proxmox.com 
>> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user 
>>
> 
> _______________________________________________ 
> pve-user mailing list 
> pve-user at pve.proxmox.com 
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user 
> 
> _______________________________________________
> pve-user mailing list
> pve-user at pve.proxmox.com
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
> 




More information about the pve-user mailing list