[pve-devel] [PATCH 2/2] add memory/dimm hotplug incl. migration support
Alexandre DERUMIER
aderumier at odiso.com
Fri Jul 4 11:26:25 CEST 2014
>>Why do you want to display it at all. In my current implementation you
>>can just higher the memory and everything is done inside qemu. No need
>>to ever now which modules are plugged in - except for migration this is
>>the reason why i store dimmX in config and remove it on a fresh start.
Oh ok, got it!
I'll tests your patches to see how it's work:)
(I just need to rebase pve-qemu-kvm patches first)
----- Mail original -----
De: "Stefan Priebe - Profihost AG" <s.priebe at profihost.ag>
À: "Alexandre DERUMIER" <aderumier at odiso.com>
Cc: pve-devel at pve.proxmox.com
Envoyé: Vendredi 4 Juillet 2014 11:18:38
Objet: Re: [pve-devel] [PATCH 2/2] add memory/dimm hotplug incl. migration support
Am 04.07.2014 11:16, schrieb Alexandre DERUMIER:
>>> how look the vmid.cfg ?
>>>
>>>> What do you mean by this?
>
> Is the vmconfig file like:
>
> dimm0: ...
> dimm1: ....
> dimm2: ....
Yes it is.
> Because I don't known how to display 256 modules in gui ;)
Why do you want to display it at all. In my current implementation you
can just higher the memory and everything is done inside qemu. No need
to ever now which modules are plugged in - except for migration this is
the reason why i store dimmX in config and remove it on a fresh start.
> I wonder if we couldn't have something like:
>
> memory : memory,maxmem=...,dimmsize=...
>
> and compute the dimm devices from this.
>
>
> (But in this case we can't mix dimm size)
>
>
> ----- Mail original -----
>
> De: "Stefan Priebe - Profihost AG" <s.priebe at profihost.ag>
> À: "Alexandre DERUMIER" <aderumier at odiso.com>
> Cc: pve-devel at pve.proxmox.com
> Envoyé: Vendredi 4 Juillet 2014 11:08:36
> Objet: Re: [pve-devel] [PATCH 2/2] add memory/dimm hotplug incl. migration support
>
>
> Am 04.07.2014 11:00, schrieb Alexandre DERUMIER:
>>>> Why? You could also insert a 4GB module or just a 512MB module.
>>
>> I have customer which need to increment memory, step by step by 1GB, each month for example
>> 10 seem to be too short for me. (I don't known if 256 is supported by all os, but accordig to qemu doc it should be ok)
>
> Sure 10 is too small. It was just a test case and as i has various
> problems with acpi tables while testing i set it to 10.
>
>> also ,can we mix module size ?
> Yes
>
>
>> how look the vmid.cfg ?
>
> What do you mean by this?
>
> Stefan
>
>>
>>
>>
>> ----- Mail original -----
>>
>> De: "Stefan Priebe - Profihost AG" <s.priebe at profihost.ag>
>> À: "Alexandre DERUMIER" <aderumier at odiso.com>
>> Cc: pve-devel at pve.proxmox.com
>> Envoyé: Vendredi 4 Juillet 2014 10:53:24
>> Objet: Re: [pve-devel] [PATCH 2/2] add memory/dimm hotplug incl. migration support
>>
>> Am 04.07.2014 10:49, schrieb Alexandre DERUMIER:
>>> Thanks for the patch, I think it's not too much different in qemu 2.1
>>>
>>> I see in you code :
>>> my $MAX_DIMMS = 10;
>>>
>>>
>>>
>>> and
>>>
>>> + push @$cmd, '-m', "size=".$memory.",slots=10,maxmem=".$conf->{maxmemory}."G";
>>>
>>>
>>> is "slots=10" = max_dimms ? (if yes,we should remove this hardcoded value)
>>
>> Yes it is. It was just a test to verify it is working at all.
>>
>>>
>>> I wonder if we could set max_dimm= 256 (max supported by acpi),$dimmsize=1G by default.
>>>
>>> so like this we can easily hotplug 256G by default.
>>
>> Why? You could also insert a 4GB module or just a 512MB module.
>>
>>
>> Stefan
>>
>>> ----- Mail original -----
>>>
>>> De: "Stefan Priebe" <s.priebe at profihost.ag>
>>> À: pve-devel at pve.proxmox.com
>>> Envoyé: Vendredi 4 Juillet 2014 10:23:45
>>> Objet: [pve-devel] [PATCH 2/2] add memory/dimm hotplug incl. migration support
>>>
>>>
>>> Signed-off-by: Stefan Priebe <s.priebe at profihost.ag>
>>> ---
>>> PVE/API2/Qemu.pm | 3 ++
>>> PVE/QemuServer.pm | 97 ++++++++++++++++++++++++++++++++++++++++++++++++++---
>>> 2 files changed, 96 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
>>> index 96c9302..11abcd3 100644
>>> --- a/PVE/API2/Qemu.pm
>>> +++ b/PVE/API2/Qemu.pm
>>> @@ -1008,6 +1008,9 @@ my $update_vm_api = sub {
>>> if($opt eq 'cores' && $conf->{maxcpus}){
>>> PVE::QemuServer::qemu_cpu_hotplug($vmid, $conf, $param->{$opt});
>>> }
>>> + elsif ($opt eq 'memory' && $running){
>>> + PVE::QemuServer::qemu_memory_hotplug($vmid, $conf, $conf_param->{$opt});
>>> + }
>>>
>>> $conf->{$opt} = $param->{$opt};
>>> PVE::QemuServer::update_config_nolock($vmid, $conf, 1);
>>> diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
>>> index b94e976..8913ee8 100644
>>> --- a/PVE/QemuServer.pm
>>> +++ b/PVE/QemuServer.pm
>>> @@ -483,6 +483,7 @@ my $MAX_UNUSED_DISKS = 8;
>>> my $MAX_HOSTPCI_DEVICES = 2;
>>> my $MAX_SERIAL_PORTS = 4;
>>> my $MAX_PARALLEL_PORTS = 3;
>>> +my $MAX_DIMMS = 10;
>>>
>>> my $nic_model_list = ['rtl8139', 'ne2k_pci', 'e1000', 'pcnet', 'virtio',
>>> 'ne2k_isa', 'i82551', 'i82557b', 'i82559er', 'vmxnet3'];
>>> @@ -520,6 +521,20 @@ for (my $i = 0; $i < $MAX_NETS; $i++) {
>>> $confdesc->{"net$i"} = $netdesc;
>>> }
>>>
>>> +my $dimmdesc = {
>>> + optional => 1,
>>> + type => 'string',
>>> + typetext => "size=<BYTES>,start=<startadress>",
>>> + description => <<EODESCR,
>>> +Later / Hotplugged memory DIMM modules
>>> +EODESCR
>>> +};
>>> +PVE::JSONSchema::register_standard_option("pve-qm-dimm", $dimmdesc);
>>> +
>>> +for (my $i = 0; $i < $MAX_DIMMS; $i++) {
>>> + $confdesc->{"dimm$i"} = $dimmdesc;
>>> +}
>>> +
>>> my $drivename_hash;
>>>
>>> my $idedesc = {
>>> @@ -2769,12 +2784,24 @@ sub config_to_command {
>>> push @$devices, '-device', print_drivedevice_full($storecfg, $conf, $vmid, $drive, $bridges);
>>> });
>>>
>>> - $conf->{memory} ||= $defaults->{memory};
>>> + my $memory = $conf->{memory} || $defaults->{memory};
>>> +
>>> + for (my $i = 0; $i < $MAX_DIMMS; $i++) {
>>> + next if !exists $conf->{"dimm$i"};
>>> + $conf->{"dimm$i"} =~ m/size=(\d+)/ or next;
>>> + my $dimmsize = $1;
>>> + $conf->{"dimm$i"} =~ m/start=(\d+)/ or next;
>>> + my $dimmstart = $1;
>>> + $memory -= $dimmsize;
>>> + push @$cmd, "-object" , "memory-ram,id=mem$i,size=".int(${dimmsize}*1024*1024);
>>> + push @$cmd, "-device", "dimm,id=dimm$i,memdev=mem$i,start=$dimmstart";
>>> + }
>>>
>>> - if ($conf->{maxmemory} && $conf->{maxmemory}*1024 > $conf->{memory}) {
>>> - push @$cmd, '-m', "size=".$conf->{memory}.",slots=10,maxmem=".$conf->{maxmemory}."G";
>>> + if ($conf->{maxmemory}) {
>>> + die "Memory is bigger than maxmemory\n" if $conf->{memory} > $conf->{maxmemory}*1024;
>>> + push @$cmd, '-m', "size=".$memory.",slots=10,maxmem=".$conf->{maxmemory}."G";
>>> } else {
>>> - push @$cmd, '-m', $conf->{memory};
>>> + push @$cmd, '-m', $memory;
>>> }
>>>
>>> for (my $i = 0; $i < $MAX_NETS; $i++) {
>>> @@ -3133,6 +3160,56 @@ sub qemu_cpu_hotplug {
>>> }
>>> }
>>>
>>> +sub qemu_memory_hotplug {
>>> + my ($vmid, $conf, $memory) = @_;
>>> +
>>> + return if !check_running($vmid);
>>> +
>>> + die "memory hot add is not possible maxmemory config is missing!\n" if !$conf->{maxmemory};
>>> + die "you cannot add more memory than maxmemory!\n" if $memory > $conf->{'maxmemory'}*1024;
>>> +
>>> + # delta of memory must be a multiple of 1024
>>> + my $toadd = $memory - $conf->{memory};
>>> +
>>> + die "memory hot remove is not supported!\n" if $toadd <= 0;
>>> +
>>> + die "memory hot add must be in gb steps!\n" if ($toadd/1024 != int($toadd/1024));
>>> +
>>> + # check for free dimms
>>> + my $i = 0;
>>> + while (defined $conf->{"dimm$i"} && $i < $MAX_DIMMS) {$i++;}
>>> +
>>> + die "no free dimm slot found!\n" if ($i >= $MAX_DIMMS);
>>> +
>>> + eval {
>>> + # size is in BYTES
>>> + vm_mon_cmd($vmid, "object-add", 'qom-type' => "memory-ram", id => "mem$i", props => { size => int($toadd*1024*1024) } )
>>> + };
>>> + my $err = $@;
>>> +
>>> + $conf->{"dimm$i"} = "size=$toadd";
>>> +
>>> + # slot is consumed even on failure and also if device_add fails
>>> + update_config_nolock($vmid, $conf, 1);
>>> +
>>> + if (!$err) {
>>> + eval {
>>> + vm_mon_cmd($vmid, "device_add", driver => "dimm", id => "dimm$i", memdev => "mem$i");
>>> +
>>> + my $cmd = { execute => 'qom-get', arguments => { path => "/machine/peripheral/dimm$i", property => "start" } };
>>> + my $start = vm_qmp_command($vmid, $cmd);
>>> +
>>> + $conf->{"dimm$i"} .= ",start=$start";
>>> +
>>> + # slot is consumed even on failure and also if device_add fails
>>> + update_config_nolock($vmid, $conf, 1);
>>> + };
>>> + $err = $@;
>>> + }
>>> +
>>> + die "Memory hot add failed!: $err\n" if $err;
>>> +}
>>> +
>>> sub qemu_block_set_io_throttle {
>>> my ($vmid, $deviceid, $bps, $bps_rd, $bps_wr, $iops, $iops_rd, $iops_wr) = @_;
>>>
>>> @@ -3341,6 +3418,18 @@ sub vm_start {
>>> # set environment variable useful inside network script
>>> $ENV{PVE_MIGRATED_FROM} = $migratedfrom if $migratedfrom;
>>>
>>> + # fresh VM start ignore DIMMs / delete them
>>> + if (!$migratedfrom) {
>>> + my $found = 0;
>>> + for (my $i = 0; $i < $MAX_DIMMS; $i++) {
>>> + next if !exists $conf->{"dimm$i"};
>>> + delete($conf->{"dimm$i"});
>>> + $found = 1;
>>> + }
>>> + update_config_nolock($vmid, $conf, 1);
>>> + $conf = load_config($vmid, $migratedfrom);
>>> + }
>>> +
>>> my ($cmd, $vollist, $spice_port) = config_to_command($storecfg, $vmid, $conf, $defaults, $forcemachine);
>>>
>>> my $migrate_port = 0;
>>>
More information about the pve-devel
mailing list