[pve-devel] rfc : qemu fw_cfg as alternative to cloudinit

[Alexandre DERUMIER]

Hi,

I also notice that qemu guest agent now support

guest-file-write
guest-exec


So, I think it's possible to manage ip address,hostname, password,... like we do it for lxc.


so, fw_cfg could be use for vm init  and when vm is running, use guest agent to change config live.



----- Mail original -----
De: "Alexandre Derumier" <aderumier at odiso.com>
À: "Thomas Lamprecht" <t.lamprecht at proxmox.com>
Cc: "pve-devel" <pve-devel at pve.proxmox.com>
Envoyé: Mercredi 4 Octobre 2017 10:31:50
Objet: Re: [pve-devel] rfc : qemu fw_cfg as alternative to cloudinit

>>The writing part got reintroduced with QEMU 2.9, in another form, 
>>your Article from above mentions that it was written during QEMU 2.6 
>>release time so maybe even that could be done in some way (did not 
>>tried it out) 

yes, here an rfc for the kernel support 
https://lists.gnu.org/archive/html/qemu-devel/2017-09/msg04995.html 




>>I saw this pop up on qemu-devel and edk2 (OVMF) devel here and there 
>>but never looked really specific at that time. 
>>It could be promising, IMO. To allow it to work we need more current 
>>OVMF images, but those are already underway :) 

great :) 




>>Side question out of interest, whats your specific use for kubernetes/rancher? 

I'll be to deploy php application (stateless) in production for our company. 
Not yet databases or statefull (I'm still afraid of docker ;). 
So we'll have a mix of database vm + vms hosting dockers in same vlans. 
Security will be done by vlan. (Each website/application have his own vlan with databases vms + docker vms) 


>> Anything that could also be integrated in our ha-manager? 

mmm, if we could have something magic, like auto create a new vm, if we detect that we don't have enough ressource on others vms. 
(maybe inside a pool, like we have a pool of 2vms, and we allow auto create up to 5vms when theses 2vms are overloaded). 
But it's a complex task. 


>>There are plans for basic (at first) load distribution of services with 
>>the intend to make the ha-manager a general resource manager where 
>>service can be ha-managed but also not. That this is not already done 
>>is mostly my fault, duh - other things got in the way (as always). 

Well, some kind of auto balancing of vms from ressources usage (at least cpu) could be wonderfull ! 
We have talked about this since years :) 







----- Mail original ----- 
De: "Thomas Lamprecht" <t.lamprecht at proxmox.com> 
À: "pve-devel" <pve-devel at pve.proxmox.com>, "aderumier" <aderumier at odiso.com> 
Envoyé: Mercredi 4 Octobre 2017 08:47:40 
Objet: Re: [pve-devel] rfc : qemu fw_cfg as alternative to cloudinit 

Hi, 

On 10/04/2017 08:14 AM, Alexandre DERUMIER wrote: 
> Hi, 
> 
> I just discover a new qemu feature : qemu fw_cfg 
> 
> http://www.contrib.andrew.cmu.edu/~somlo/QEMU_fw_cfg/ 
> 

Here the up-to-date specification: 
<https://git.qemu.org/?p=qemu.git;a=blob_plain;f=docs/specs/fw_cfg.txt;h=08c00bdf44a2cca1064356470af2d5438d44e3c9;hb=HEAD> 

> 
> It's allow to add a custom firmware, from a host file 
> 
> like "-fw_cfg name=opt/GuestInfo,file=/etc/pve/qemu-server/100.conf" , for example 
> 
> 
> Then it's possible to read this file in guest in 
> 
> "/sys/firmware/qemu_fw_cfg/by_name/opt/GuestInfo/raw" 
> 
> This need a new module include in guest kernel 4.6 (CONFIG_FW_CFG_SYSFS), 
> but the module is also available out of kernel tree here: 
> http://www.contrib.andrew.cmu.edu/~somlo/QEMU_fw_cfg/qemu_fw_cfg_kmod.tgz 
> 
> (really small, it's simply extend standard firmware , so it's work with old kernel too) 
> 
> for windows, seem that they are working on it (https://bugzilla.redhat.com/show_bug.cgi?id=1375444) 
> 
> 
> Advantage vs cloudinit: 
> 
> - no need to pass cdrom, no extra pci device.(it's simply a firmware) 
> - you can use it in initramfs. (format disk, do mount point,...) 
> 
> 
> disavantage vs cloudinit: 
> 
> - read only (you can change it after boot, but it's really for vm init) 

The writing part got reintroduced with QEMU 2.9, in another form, 
your Article from above mentions that it was written during QEMU 2.6 
release time so maybe even that could be done in some way (did not 
tried it out) 

> - no daemon or standard to parse the config, you need to do it yourself. 
> But coreos for example, have a projet "ignition", 
> https://coreos.com/ignition/docs/latest/ 
> which can use it (it's working in initramfs) 
> 
> 
> 
> I think it should be possible to provide qemu linux distro template like lxc, format the disk in initramfs, do the mount point, deploy source for example. 
> Maybe also provide initramfs to qemu with -initrd, with an integrated custom config parser,to format the disk, do the mountpoint, deploy the template,set ssh keys, .... 
> 
> What do you think about it ? 
> 

I saw this pop up on qemu-devel and edk2 (OVMF) devel here and there 
but never looked really specific at that time. 
It could be promising, IMO. To allow it to work we need more current 
OVMF images, but those are already underway :) 

> 
> ( 
> I'm currently working on a terraform proxmox provider, to deploy vm, as we are looking to deploy Rancher next year (a kubernetes clusters managers), 
> and I need to be able to create vm from outside proxmox, with ip configuration,hostname,...) 
> 

Side question out of interest, whats your specific use for kubernetes/rancher? 
Anything that could also be integrated in our ha-manager? 
There are plans for basic (at first) load distribution of services with 
the intend to make the ha-manager a general resource manager where 
service can be ha-managed but also not. That this is not already done 
is mostly my fault, duh - other things got in the way (as always). 

_______________________________________________ 
pve-devel mailing list 
pve-devel at pve.proxmox.com 
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel