[pve-devel] KVM Templates on CephFS are only shown on one node - not all nodes in the cluster

[Victor Hooi]

Hi Dominik,

Aha - I apologise - let me explain the use-case.

We have two Proxmox clusters in our lab - both are HA clusters, using
hyper-converged Ceph to store the VM disk images.

In each cluster, we have a template of various VM images (e.g. Windows 7,
Windows 8.1, Windows 10 etc), in various different configurations/software
combinations.

We want the same templates to appear *under* each node because:

   - Redundancy/HA - if one of the nodes is down - Ceph can still function,
   and we can still spin-up VMs. However, if a template only appears under
   node1, and node1 dies - then we cannot use any templates anymore and the
   cluster is useless (even if the files are technically still accessible via
   Ceph)
   - Distributing VMs across the cluster - If all the templates appear
   under node1 - then when a user goes to clone it, it will default to node1.
   Proxmox currently doesn't have a feature to automatically distribute VMs
   across the cluster (see this thread for a discussion on the feature
   <https://forum.proxmox.com/threads/3-node-proxmox-ceph-cluster-how-to-automatically-distribute-vms-among-nodes.52443/>).
   If the templates appear under each node (even if they point to the same
   underlying file), it makes it easier to get users to understand you can
   spin them up there


I tried to do it just now - this is what I did.

I had a VM on node1. I converted it into a template:

https://i.imgur.com/xlLXUy5.png - this refers to
vm_storage:base-500-disk-0/base-100-disk-0,size=30G

I then did a linked clone of this onto node2 - which I also converted into
a template

https://i.imgur.com/PUel6RB.png - this refers to
vm_storage:base-100-disk-0/base-104-disk-0,size=30G

Do you know the correct steps to go from a VM, to two templates on two
different nodes, pointing to the same base image?

Thanks,
Victor

On Mon, Mar 9, 2020 at 7:36 PM Dominik Csapak <d.csapak at proxmox.com> wrote:

> On 3/7/20 9:46 PM, Victor Hooi wrote:
> > Hi Dominik,
> >
>
> hi,
>
> > Is there any way to have the templates available on each node in the
> > cluster, without making separate copies for each one in Ceph? (That is -
> > have them all point to the same underlying files).
>
> i still do not understand why you'd want this?
> as you already tried, you can clone it across nodes already, why would
> you need the templates on all nodes?
>
> >
> > I tried making a thin clone from one node to another - and then when I
> > converted to a template, it simply turned it into a full clone, and
> > created a separate copy of the VM disk images - such that I had multiple
> > copies of the same files in RBD.
>
> hmm? what exactly did you do and why do you think that they are
> full clones?
>
> i just tested it here and converting a 'linked clone' to template
> simply renames the vm disks to 'base-...'
>
> 'rbd ls -l'
>
> still shows me the correct parent image...
>
> kind regards
> Dominik
>
>