[pve-devel] [PATCH] add hugepages option v5

[Alexandre DERUMIER]

>>Btw. I just noticed that if I increase the number, then start using pages,
>>then set to 0, it'll show the number of used pages, and when the program
>>stops using them the number shrinks down to the previously written value,
>>so perhaps all we need to do is write 0 right after starting qemu so that
>>they get automatically freed once qemu exits?

Seem to works fine here !

I need to used
echo 0 > /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages

because

 echo 0 > /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages
 echo 0 > /sys/devices/system/node/node1/hugepages/hugepages-1048576kB/nr_hugepages

only free node0 only.

(maybe they are some kind of queue)


I have also tried mixing vms with 2048 and 1GB hugpages

start vm 1GB hugepages
echo 0 > /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages
start vm 2MB hugepages
echo 0 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages

killall -9 kvm

-> memory is correctly free


So it's seem to be clean and easy to implement !

----- Mail original -----
De: "Wolfgang Bumiller" <w.bumiller at proxmox.com>
À: "aderumier" <aderumier at odiso.com>, "pve-devel" <pve-devel at pve.proxmox.com>
Envoyé: Mercredi 8 Juin 2016 10:55:00
Objet: Re: [pve-devel] [PATCH] add hugepages option v5

> On June 8, 2016 at 10:50 AM Wolfgang Bumiller <w.bumiller at proxmox.com> wrote: 
> 
> 
> 
> > On June 8, 2016 at 7:13 AM Alexandre DERUMIER <aderumier at odiso.com> wrote: 
> > 
> > 
> > >>One possibility would be adding an inotify handler onto the qemu.slice 
> > >>to pvestatd... 
> > 
> > another way, simply wrote a 
> > 
> > /var/run/qemu-server/vmid.hugepages when vm is starting, with hugepages topology 
> > 
> > then in pvestatd, if vm is down and vmid.hugepages file is present, cleanup memory and remove file 
> > 
> > 
> > Like this we can avoid inotify (in case of pvestatd was stopped/hanging) 
> 
> A suggestion from Dietmar: 
> How about pvestatd just regularly sets nr_hugepages to 0 while holding the lock? 
> The numbers you write to the file get capped by what's available and in-use. 
> Eg if nr_hugepages is 20, you have a VM using 8, and write 0 to it, then 
> it'll contain 8. If we do this while holding the same lock we hold when 
> starting a VM then this should work? Basically: allocate when starting, and 
> let pve-statd free it regularly. 

Btw. I just noticed that if I increase the number, then start using pages, 
then set to 0, it'll show the number of used pages, and when the program 
stops using them the number shrinks down to the previously written value, 
so perhaps all we need to do is write 0 right after starting qemu so that 
they get automatically freed once qemu exits?