[pve-devel] template considerations

[Alexandre DERUMIER]

Hi Dietmar, here some comments and questions:


>>The current VM template patches heavily depend on the ‘live snapshot’ 
>>feature. This works well for some storage types (rdb, sheeddog, …), but 
>>not for all. Most important it does not work well with qcow2 files. 

Well, my currents patches allow to do clone (clone = linked cloned) of 

qcow2,raw : from current (no snapshot)
rbd,sheepdog,nexenta : from snapshots

But I use snapshot tree panel to manage both.


>>So using ‘live snapshots’ for templates has the following drawbacks: 
>>- does not work well for qcow2 files 
>>- does not work with LVM
Yes indeed, but I don't use snapshots for qcow2,raw clone (I don't have implemetend lvm)


>>- we are not able to backup/restore such templates 

Do we really need to backup template ?

template disk are protected (readonly file or locked volumes), so they are no way to delete them.
In case of a total storage destroy, we lost also the vm disks, so we need to restore them, and currently we backup the full vm (base image + real image are merged).

Or do you plan to find a way to backup separatly the base image and the real image ?


- difficult to copy/migrate such templates 
for rbd,sheepdog,nexenta, as they are shared storage, I'm not sure we need to copy/migrate the templates.
But for local storage, qcow2/raw baseimage copy accross each node could be great, to allow start a clone child on all nodes.


>>Terms and Definitions: 
>>
>>Disk Image : Also called ‘volumes’. We use the following volume naming: 
>>
>>local:100/vm-100-disk-1.raw 
>>
>>The real image file resides at (path): 
>>
>>/var/lib/vz/images/100/vm-100-disk-1.raw 
>>
>>Base image : A ‘read only’ image file which can be ‘cloned’. We can 
>>use the following volume naming: 
>>
>>local:base/an-arbitray-name.raw 

arbitray, really ? Doesn't we need some convention names, or ids somewhere ?


>>The real images resides at (path): 
>>
>>/var/lib/vz/images/base/an-arbitray-name.raw 
>>
>>We can easily clone that by creating a qcow2 image which use this 
>>as base image. 
>>
>>You could also name that a ‘template disk’. 
>>
>>Cloned image: A disk image which refers to a ‘base image’. 
>>
>>VM templates : A VM only using base images. You can never start such VM because 
>>the disk images are read-only. But copy/clone/backup/restore should be very easy 
>>to implement. 

>>Usage scenario 
>>
>>Base images : 
>>
>>When we create/add disks, we can allow to choose from those 
>>base images to create cloned images. 
>>

Do you think we need this ? vm template cloning is not enough ?


>>VM templates : 
>>

So a vm template is only a vm with baseimage in current config right ?

>>There are basically two ways to create them. 
>>
>>1.) manually assemble them by adding ‘base images’ to a VM 

What happen if a vm a baseimage and non base images? 
Or if vm have existing live snasphots ?

>>2.) Transform an existing VM automatically. 
>>
 
>>If the user wants to modify a template, he need to: 
>>
>>1.) clone/copy the VM 
>>2.) start the VM, do any modification, then stop 
>>3.) create a new template from that 

I could be great to rollback a template to a vm, if no clone of the template exist. (You create a template, but forgot something by example)


>>Other storage types 
>>
>>This should work with any storage type, because we do not longer 
>>depend on internal snapshots. 

>>Directory storage: 
>>
>>create base image: 
>>mv images/100/vm-100-disk-1.raw images/base/an-arbitray-name.raw 
>>
>>create clone: 
>>simply create an qcow2 image which refers to the base image 

ok no problem

>>LVM : 
>>
>>create base image: 
>>simply rename the LV to indicate that it is a template and read only. 
>>
>>create clone: 
>>create a snapshot. 

>>Note: I am not really sure if that works on shared storage? 

I have done some fast tests, and it seemed to work, but maybe it can give datas corruptions. 
Need to be tested more.


>>RDB, Sheepdog and others : 
>>
>>create base image: 
>>do whatever is needed to create a ‘cloneable’ image. This most likely 
>>involves to create a ‘snapshot’, and rename/remove the original image. 
>>
>>create clone: 
>>storage driver dependent. 

What are we doing with existing disks snapshots ? (If the original vm have snapshots). 
Do we keep them or deleted them ?

Also we need some convention for snapshot name we create for the template. (To avoid conflict with possible existing disk snapshots)
maybe @base  ? like for the  /base/ for directory storage.




I think it should be easy to implement, I'll have time this week to work on it if you want.
  
----- Mail original ----- 

De: "Dietmar Maurer" <dietmar at proxmox.com> 
À: "Alexandre DERUMIER (aderumier at odiso.com)" <aderumier at odiso.com>, pve-devel at pve.proxmox.com 
Envoyé: Samedi 26 Janvier 2013 11:27:13 
Objet: template considerations 



The current VM template patches heavily depend on the ‘live snapshot’ 
feature. This works well for some storage types (rdb, sheeddog, …), but 
not for all. Most important it does not work well with qcow2 files. 

But the major drawback is that our backup/restore does not work 
well with ‘live snapshots’. 

So using ‘live snapshots’ for templates has the following drawbacks: 

- does not work well for qcow2 files 
- does not work with LVM 
- we are not able to backup/restore such templates 
- difficult to copy/migrate such templates 

The following is just an idea to improve things. I would like that 
Alexandre poof-read and comment on that. I will explain all 
things using ‘local’ file storage first. 

Terms and Definitions: 

Disk Image : Also called ‘volumes’. We use the following volume naming: 

local:100/vm-100-disk-1.raw 

The real image file resides at (path): 

/var/lib/vz/images/100/vm-100-disk-1.raw 

Base image : A ‘read only’ image file which can be ‘cloned’. We can 
use the following volume naming: 

local:base/an-arbitray-name.raw 

The real images resides at (path): 

/var/lib/vz/images/base/an-arbitray-name.raw 

We can easily clone that by creating a qcow2 image which use this 
as base image. 

You could also name that a ‘template disk’. 

Cloned image: A disk image which refers to a ‘base image’. 

VM templates : A VM only using base images. You can never start such VM because 
the disk images are read-only. But copy/clone/backup/restore should be very easy 
to implement. 

Usage scenario 

Base images : 

When we create/add disks, we can allow to choose from those 
base images to create cloned images. 

VM templates : 

There are basically two ways to create them. 

1.) manually assemble them by adding ‘base images’ to a VM 
2.) Transform an existing VM automatically. 

If the user wants to modify a template, he need to: 

1.) clone/copy the VM 
2.) start the VM, do any modification, then stop 
3.) create a new template from that 

Other storage types 

This should work with any storage type, because we do not longer 
depend on internal snapshots. 

Directory storage: 

create base image: 
mv images/100/vm-100-disk-1.raw images/base/an-arbitray-name.raw 

create clone: 
simply create an qcow2 image which refers to the base image 

LVM : 

create base image: 
simply rename the LV to indicate that it is a template and read only. 

create clone: 
create a snapshot. 

Note: I am not really sure if that works on shared storage? 

RDB, Sheepdog and others : 

create base image: 
do whatever is needed to create a ‘cloneable’ image. This most likely 
involves to create a ‘snapshot’, and rename/remove the original image. 

create clone: 
storage driver dependent.