[pve-devel] RFC: qemu-server : add cloudinit support

[Alexandre DERUMIER]

>>It doing nothing is what I'm afraid of. I've tested an ubuntu 
>>installation just now, and after removing the config drive the 
>>cloud-init init-process just sat there, not letting me boot all the way 
>>through to the login screen. IOW the entire boot process was hanging 
>>around doing nothing. This isn't really ideal. 

I'll try to test with ubuntu to see.


>>OTOH it may be desirable to keep the config datasource attached. That 
>>way the WEB UI's configuration is the "permanent" one, and changes made 
>>in the running VM to /etc/ (eg. /etc/network/interfaces) are only 
>>temporary. 
>>
>>Otherwise the options in the UI would have to be deleted/greyed out with 
>>it to reflect the new situation, while if the config is kept around 
>>there'd be some permanently-available settings in the UI. 

I think that Cloud-init should be only use to setup the vm create|provisioning
as I said in a previous mail, I think we should simply pass parameters
to qm clone .... -ipddress -gateway -nameserver ...
This is what ovirt|rhev are doing currently.
http://www.ovirt.org/Features/Cloud-Init_Integration


If we would like to manage ips in gui, we could generate a new configdrive,
but we can't tell to guest to reforce relaunch of cloud-init.
(or maybe with a udev rules trick ?)




>>And with the init process hanging dead in my tests (for around 10 
>>minutes, and afterwards errors appear like using sudo shows "unable to 
>>resolve host <somename>.localdomain", which tells me cloud-init decided 
>>to screw something up...) it means that the only safe way to remove the 
>>datasource is to also have it uninstall the cloud-init package on the OS 
>>- unless the packaged cloud-init is smarter. (I tried an ubuntu-server 
>>14.10 image) 

Maybe the datasource is not configdrive ?
They are other datasources for amazon for exemple, which try to connect to amazon to get the config.

they are details logs in /var/log/cloudinit/




>>According to this cloud-init doesn't actually run on windows directly, 
>>but there's a "port" named cloudbased-init. It supports a config-disk 
>>too, but I doubt it uses the same layout and syntax everywhere. (After 
>>all, network interfaces for instance aren't exactly called 'eth0' and 
>>'eth1' on windows ;-) ) 

Well, the network config in configdrive is at debian format.
But each cloud-init implementation on different guest os, 
is parsing it and write correct config.
So it should work.  (I have tested it with fedora, and it's writen in redhat /sysconfig/ format)




----- Mail original -----
De: "Wolfgang Bumiller" <w.bumiller at proxmox.com>
À: "aderumier" <aderumier at odiso.com>
Cc: "dietmar" <dietmar at proxmox.com>, "pve-devel" <pve-devel at pve.proxmox.com>
Envoyé: Jeudi 11 Juin 2015 13:41:32
Objet: Re: [pve-devel] RFC: qemu-server : add cloudinit support

On Thu, Jun 11, 2015 at 12:51:42PM +0200, Alexandre DERUMIER wrote: 
> >>For one, it seems to be started on several points during the init 
> >>process. There seem to be 5 different systemd units specified for it 
> >>even. 
> yes, I'm seeing this 4 units. 
> 
> I'm think they are related to different cloud-init stage (init,config,final, see /etc/cloud/cloud.cfg). 

Yes, it's just not immediately clear when each of them happen and what 
they're _exactly_ doing. 

> If I remember, init stage is done only at first boot. 
> cloud_init_modules: 
> - migrator 
> - bootcmd 
> - write-files 
> - resizefs 
> - set_hostname 
> - update_hostname 
> - update_etc_hosts 
> - ca-certs 
> - rsyslog 
> - users-groups 
> - ssh 

This part is configured in /etc apparently. So there the list of enabled 
hooks is configured AFAIU. 

> I'm not sure about other stage. 

Yeah, I really wish there was a detailed design document somewhere ;-) 

> >>Then there seems to be a system-wide configuration in /etc/cloud that 
> >>defines not just datasources but also default configurations, like a 
> >>default user, and whether root may login with a password, etc. 
> 
> users,keys,password are not defined in /etc/cloud/cloud.cfg but in configdrive 
> They are some system default thing, like admin account for debian for example (but no password) 

They _can_ be, but I haven't figured out if/how/when they're added if 
you actually do it in there. At least my test user never came to live on 
my arch test vm. 

> but the main thing is the the datasource provider (I'm using configdrive like openstack|ovirt) 
> other things are list of modules we can call from configdrive 

Yes, that much is clear to me by now. 

> >>At this 
> >>point I'm wondering whether this configuration will be re-applied on 
> >>every boot. (The documentation doesn't mention any of this actually, I 
> >>found this part on the ArchWiki[1]...) 
> 
> >>And does that mean cloud-init keeps overwriting the systems's regular 
> >>configuration files? 
> 
> If you don't provide the configdrive (no datasource), it'll do nothing. 
> That's why I remove the configdrive after first init. 

It doing nothing is what I'm afraid of. I've tested an ubuntu 
installation just now, and after removing the config drive the 
cloud-init init-process just sat there, not letting me boot all the way 
through to the login screen. IOW the entire boot process was hanging 
around doing nothing. This isn't really ideal. 

OTOH it may be desirable to keep the config datasource attached. That 
way the WEB UI's configuration is the "permanent" one, and changes made 
in the running VM to /etc/ (eg. /etc/network/interfaces) are only 
temporary. 

Otherwise the options in the UI would have to be deleted/greyed out with 
it to reflect the new situation, while if the config is kept around 
there'd be some permanently-available settings in the UI. 

And with the init process hanging dead in my tests (for around 10 
minutes, and afterwards errors appear like using sudo shows "unable to 
resolve host <somename>.localdomain", which tells me cloud-init decided 
to screw something up...) it means that the only safe way to remove the 
datasource is to also have it uninstall the cloud-init package on the OS 
- unless the packaged cloud-init is smarter. (I tried an ubuntu-server 
14.10 image) 

> >>The documentation only mentions a directory structure in /var. [2] 
> >>Now I'm left to wonder... does the cloud init process somehow 
> >>overwride/interfere with the regular initialization process? Does it 
> >>actually perform physical merging of configuration files from the 
> >>datasource into /var? (The documentation also mentions merging[3]) 
> 
> I don't see nothing special in /var/lib/cloud, no config cache or something like that 

Yes, this section of the documentation still seems somewhat weird to me. 

> >>Finally: OS support. It seems to only ship with support linux (few 
> >>specific distros even) and freebsd [4]. For that purpose it seems a 
> >>little overengineered. 
> 
> linux distro : centos,rhel,fedora,debian,ubuntu,suse,archlinux,gentoo 
> bsd: freebsd 
> windows 
> http://thornelabs.net/2014/06/01/where-to-find-openstack-cloud-images.html 
> 
> don't seem too bad ? 

According to this cloud-init doesn't actually run on windows directly, 
but there's a "port" named cloudbased-init. It supports a config-disk 
too, but I doubt it uses the same layout and syntax everywhere. (After 
all, network interfaces for instance aren't exactly called 'eth0' and 
'eth1' on windows ;-) )