[pve-devel] cpu 100% when get vmstaus

[Dietmar Maurer]

Is there a way to reproduce that behavior? You already tested with new pve-qemu-kvm (1.4)?

From: pve-devel-bounces at pve.proxmox.com [mailto:pve-devel-bounces at pve.proxmox.com] On Behalf Of lulin
Sent: Samstag, 23. Februar 2013 11:33
To: pve-devel at pve.proxmox.com
Subject: [pve-devel] cpu 100% when get vmstaus

Hi
I don't know if it is a bug, sometimes when you shudown a VM,  pvedaemon will have 100% cpu usage.
bellow is 'top' output:
------------------------------
Tasks: 761 total,   3 running, 758 sleeping,   0 stopped,   0 zombie
Cpu(s): 14.0%us,  2.8%sy,  0.0%ni, 83.2%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  49378500k total, 45560112k used,  3818388k free,    17408k buffers
Swap: 24764408k total,  4106172k used, 20658236k free,  2291944k cached

  PID USER      PR  NI  VIRT   ;RES  SHR S %CPU %MEM    TIME+  COMMAND
31347 root      20   0  209m  41m 3664 R 99.6  0.1  44:36.22 pvedaemon
-----------------------------
I print some logs to trace it, and find it is deadloop in QMPclient.pm:

240<https://git.proxmox.com/?p=qemu-server.git;a=blob;f=PVE/QMPClient.pm;h=4229e16c3b049572f2a706d9aa50136d694e0e61;hb=HEAD#l240>     for (;;)&n
bsp;{
241<https://git.proxmox.com/?p=qemu-server.git;a=blob;f=PVE/QMPClient.pm;h=4229e16c3b049572f2a706d9aa50136d694e0e61;hb=HEAD#l241>
 242<https://git.proxmox.com/?p=qemu-server.git;a=blob;f=PVE/QMPClient.pm;h=4229e16c3b049572f2a706d9aa50136d694e0e61;hb=HEAD#l242>         $running = &$check_queue($self);
243<https://git.proxmox.com/?p=qemu-server.git;a=blob;f=PVE/QMPClient.pm;h=4229e16c3b%20049572f2a706d9aa50136d694e0e61;hb=HEAD#l243>
 244<https://git.proxmox.com/?p=qemu-server.git;a=blob;f=PVE/QMPClient.pm;h=4229e16c3b049572f2a706d9aa50136d694e0e61;hb=HEAD#l244>         last if !$running;
245<https://git.proxmox.com/?p=qemu-server.git;a=blob;f=PVE/QMPClient.pm;h=4229e16c3b049572f2a706d9aa50136d694e0e61;hb=HEAD#l245>
 246<https://git.proxmox.com/?p=qemu-server.git;a=blob;f=PVE/QMPClient.pm;h=4229e16c3b049572f2a706d9aa50136d694e0e61;hb=HEAD#l246>         $self->{mux}->loop;
247<https://git.proxmox.com/?p=qemu-server.git;a=blob;f=PVE/QMPClient.pm;h=4229e16c3b049572f2a706d9aa50136d694e0e61;hb=HEAD#l247>     }

it seems if a vm is shutdown, the QMP socket can't have a close notify, and use 'netstat' command you can find a connection always stay in CLOSE_WAIT:
[root at OVPMAIN ~]# netstat -antop |grep pvedaemon
tcp        0      0 127.0.0.1:85                0.0.0.0:*                   LISTEN      1018/pvedaemon work off (0.00/0/0)
tcp        1      0 127.0.0.1:85                127.0.0.1:49246             CLOSE_WAIT  31347/pvedaemon wor off (0.00/0/0)

I tried to change the tcp_keepalive_* param in /proc, but not work since keepalive is disabled by default...

Any one have some idea?


Regards

Lenny


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.proxmox.com/pipermail/pve-devel/attachments/20130225/e2c5a30d/attachment.htm>