[pve-devel] Error in PVE with win2008r2 and 256GB RAM

Mon Dec 1 18:27:42 CET 2014

I think this could help, if you have a numa architecture.

for example, if you defined a vm, with 2 sockets and 2 cores, and 256GB ram.

and that your host is 2sockets, with 128GB in each processor bank

 -object memory-backend-ram,size=131072M,policy=bind,host-nodes=0,id=ram-node0 \
 -numa node,nodeid=0,cpus=0-1,memdev=ram-node0 \
 -object memory-backend-ram,size=131072M,policy=bind,host-nodes=1,id=ram-node1 \
 -numa node,nodeid=1,cpus=2-3,memdev=ram-node1

(you need kernel 3.10 for auto numa balancing + qemu 2.1(proxmox 3.3)

----- Mail original ----- 

De: "Alexandre DERUMIER" <aderumier at odiso.com> 
À: "Cesar Peschiera" <brain at click.com.py> 
Cc: pve-devel at pve.proxmox.com 
Envoyé: Lundi 1 Décembre 2014 18:08:46 
Objet: Re: [pve-devel] Error in PVE with win2008r2 and 256GB RAM 

Also, 
seem that they are numa improvement since qemu 2.1 and auto numa balancing, 

I'm currently reading the doc, 

I'll try to send command lines tommorrow to test. 

----- Mail original ----- 

De: "Alexandre DERUMIER" <aderumier at odiso.com> 
À: "Cesar Peschiera" <brain at click.com.py> 
Cc: pve-devel at pve.proxmox.com 
Envoyé: Lundi 1 Décembre 2014 07:55:14 
Objet: Re: [pve-devel] Error in PVE with win2008r2 and 256GB RAM 

also, can you post your vm config file ? 

Proxmox use some hyper-v features to help for some case, including high memory. 

But currently, 2 features are missing. 

in /usr/share/perl5/PVE/QemuServer 

if ($ost eq 'win7' || $ost eq 'win8' || $ost eq 'w2k8' || 
$ost eq 'wvista') { 
push @$globalFlags, 'kvm-pit.lost_tick_policy=discard'; 
push @$cmd, '-no-hpet'; 
#push @$cpuFlags , 'hv_vapic" if !$nokvm; #fixme, my win2008R2 hang at boot with this 
push @$cpuFlags , 'hv_spinlocks=0xffff' if !$nokvm; 
} 

if ($ost eq 'win7' || $ost eq 'win8') { 
push @$cpuFlags , 'hv_relaxed' if !$nokvm; 
} 

maybe can your try to uncomment 
#push @$cpuFlags , 'hv_vapic" if !$nokvm; #fixme, my win2008R2 hang at boot with this 

and restart 
/etc/init.d/pvedaemon restart 

and start your vm again. 

(I think they was a bug in previous kernel, but maybe it's fixed now). 

Another missing feature is hv_time, paravirtualized clock, but AFAIK it's only work with 3.10 kernel. 

so, you can try 

if ($ost eq 'win7' || $ost eq 'win8') { 
push @$cpuFlags , 'hv_relaxed' if !$nokvm; 
push @$cpuFlags , 'hv_time' if !$nokvm; 
} 

----- Mail original ----- 

De: "Alexandre DERUMIER" <aderumier at odiso.com> 
À: "Cesar Peschiera" <brain at click.com.py> 
Cc: pve-devel at pve.proxmox.com 
Envoyé: Lundi 1 Décembre 2014 06:47:35 
Objet: Re: [pve-devel] Error in PVE with win2008r2 and 256GB RAM 

>>3) In parallel to this strange behavior, htop on PVE is showing that the 
>>process that consume a lot of processor is: "/usr/bin/kvm", this behavior is 
>>repetitive in all threads of processor that the VM has access. 

The kvm process is your vm ;) (each guestvm is a kvm process) 

>>4) In parallel to this strange behavior, while that the VM is configured 
>>with 62GB RAM, htop on PVE is showing that the use of the memory is growing 
>>in each second that elapses, and when his memory bar say that have used 
>>"63964/257912MB", the consumption of threads of processors of this VM 
>>returns to normal state. While that the VM has more RAM, the behavior is the 
>>same, but the VM takes longer time to reach to a normal state. 

At boot, windows fill the memory with zero. and that's use cpu 

BTW, do you use balloning/dynamic memory feature of proxmox ? 
can you try to add balloon:0 in your config. (to disable the balloon device) 

>>5) As a second test, after of see all these behaviours, I log to Windows 
>>Server, and htop show me a high consumption of many threads of processors 
>>(+/- 50%), but after that the session was initiated, the consumption of 
>>processors returns to normal state. 

>>6) In htop, i see the same behaviour of consumption of processor while that 
>>a session of windows is closing, i guess that any thing that i do in this VM 
>>will consume processor resources extra needlessly. 

really don't known, this should require some cpu profiling inside windows. 

----- Mail original ----- 

De: "Cesar Peschiera" <brain at click.com.py> 
À: pve-devel at pve.proxmox.com 
Envoyé: Lundi 1 Décembre 2014 06:35:14 
Objet: Re: [pve-devel] Error in PVE with win2008r2 and 256GB RAM 

Hi to PVE team developers 

This is a new summary of a major problem between PVE, KVM and Win2008R2 SP1 
VM. 

Maybe i am doing something wrong, or maybe PVE or KVM need some corrections, 
but i have done many test for get that the Win2008R2 SP1 works correctly 
without successful, so that i would like consult about this to the PVE 
developers 

Summary of the problem: 
------------------------- 
When i turn on the Win2008R2 VM, the cpu go to the 100% in use for each 
thread enabled in the VM, and when the VM is configured for example with 4GB 
RAM, this VM works perfectly. 

This is my Hardware setup: 
--------------------------- 
- DELL poweredge R720 Server, 
- Two processors Intel Xeon CPU E5-2690 v2 @ 3.00GHz., each processor has 
10 cores and 20 threads (Hyperthreading), 
- 256GB RAM. in performace mode 

This is my Software setup: 
-------------------------- 
- The Software is recently installed: 
- PVE 3.3-5 (from his ISO and upgraded), 
- Kernel: pve-kernel-2.6.32-34-pve, 
- As VM: Win2008R2 SP1 using the 2 Intel processors and 248 GB RAM, 
- VM without the use of tablet for pointer enabled in PVE (i use vmmouse of 
VMware). 

Here the symptoms and my actions performed: 
----------------------------------------------- 

1) Doing some tests, i have changed manually the size the page file of 
Windows Server to 10 GB, and configured the VM with 62 GB. RAM (63488 MB. 
RAM), now i can see that the Windows Server start with 100% of CPU, and 
after of two minutes more or less, the processor returns to a normal state. 

2) Seeing the task manager of Win2008R2 VM (while all cores and threads are 
at 100%), i get this: 
Image Name: System 
User Name: System 
CPU: 99% 
memory (Private workspace): 52 KB 
Description: NT Kernel & System 

All other processes are consuming 0% of CPU 

Here comes the more important information of side of PVE, 
and other actions performed: 
------------------------------------------------------------ 
3) In parallel to this strange behavior, htop on PVE is showing that the 
process that consume a lot of processor is: "/usr/bin/kvm", this behavior is 
repetitive in all threads of processor that the VM has access. 

4) In parallel to this strange behavior, while that the VM is configured 
with 62GB RAM, htop on PVE is showing that the use of the memory is growing 
in each second that elapses, and when his memory bar say that have used 
"63964/257912MB", the consumption of threads of processors of this VM 
returns to normal state. While that the VM has more RAM, the behavior is the 
same, but the VM takes longer time to reach to a normal state. 

5) As a second test, after of see all these behaviours, I log to Windows 
Server, and htop show me a high consumption of many threads of processors 
(+/- 50%), but after that the session was initiated, the consumption of 
processors returns to normal state. 

6) In htop, i see the same behaviour of consumption of processor while that 
a session of windows is closing, i guess that any thing that i do in this VM 
will consume processor resources extra needlessly. 

7) Moreover, while that the task manager of Win2008R2 VM says that have +/- 
60GB free, PVE in his tag "Summary" says exactly the conversely. 

8) Trying to solve the problem, i did the suggestions without successful of 
Mr. Spirit, according to this link: 
http://forum.proxmox.com/threads/19309-Windows-KVM-frequent-restarts?p=99436#post99436 

Maybe KVM or PVE have problems for manage lots RAM memory with numa enabled 
in the bios of the Server, but in am not sure. 
Only as a reference, i show this link: 
https://bugzilla.redhat.com/show_bug.cgi?id=872524 

I will be very grateful I will be very grateful to anyone that may help me. 

Best regards 
Cesar 

_______________________________________________ 
pve-devel mailing list 
pve-devel at pve.proxmox.com 
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
_______________________________________________ 
pve-devel mailing list 
pve-devel at pve.proxmox.com 
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
_______________________________________________ 
pve-devel mailing list 
pve-devel at pve.proxmox.com 
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
_______________________________________________ 
pve-devel mailing list 
pve-devel at pve.proxmox.com 
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 