[PVE-User] Some thought about zfs based configurations

[Pongrácz István]

----------------eredeti üzenet-----------------
Feladó: "Dietmar Maurer" dietmar at proxmox.com 
Címzett: "pve-user pve.proxmox.com" 
, "Pongrácz István" 
Dátum: Fri, 31 Jul 2015 17:43:28 +0200 (CEST)
-------------------------------------------------

>> 
>> In ZFS, it is possible to create independent filesystems for all VMs and CTs,
>> instead of using a top-level filesystem and put every VMs/CTs under that. To
>> achieve this, it would be necessary to rewrite the storage subsystem to 
>> handle
>> the zfs case.
> 
> No - Proxmox VE is already able to create and use ZFS subvolumes.


In 3.4, I tested, maybe I missed something.
I can create logical zfs volume on the GUI and assign virtual disk (zfs volume) of a VM.
That's all I found.
This case it is possible to have independent volumes for VMs and handle them as I wrote, but this is not true for CT storages. I cannot put CT, templates into that volume.

I created 2 x CT and 2 x VM, here are the config and the zfs list output:
100 - VM
101 - CT
102 - VM (storage on zfs subvolume)
103 - CT (cannot create zfs filesystem and put a container to it)


root at pve34 :/# zfs list
NAME USED AVAIL REFER MOUNTPOINT
rpool 2.24G 13.1G 96K /rpool
rpool/ROOT 1.31G 13.1G 96K /rpool/ROOT
rpool/ROOT/pve-1 1.31G 13.1G 1.31G /
rpool/data 96K 13.1G 96K /rpool/data
rpool/swap 953M 14.1G 64K -
tank2 1.03G 2.81G 96K /tank2
tank2/vm-102-disk-1 1.03G 3.84G 64K -


I have no individual zfs filesystems automatically on CT/VM creation time, like:

var/lib/vz/private/101
var/lib/vz/private/103
var/lib/vz/images/100

Of course, I can do it manually and I usually did it. I just say, separation of all VMs/CTs does not happen on creation time, but it is possible to do it manually.
Due to that, (except VMs on zfs subvolumes), everything located in one zfs filesystem, which could be improved.

Maybe now it is more clear than my previous email.




>> So, using ZFS with CT like this is a very good combination: individual files
>> easily accessible from hardware nodes, even from snapshots without any 
>> magic
>> trick: quick and elegant. And a plus, we have all zfs benefits without 
>> hacking
>> the CT.
> 
> Dont really got this one (same as part one)?

Yes, it is similar, as I focus on separating all CTs and VMs into their own filesystems and have the host operating system also independent of guest systems.

Openvz containers in 3.4 use the underlying storage directly, in our case, they are just simple directories in the filesystem. This is good, flexible, can be managed by zfs (manually can be improved to have separated filesystems for each CT or even subdirectory in a CT, like postgresql data directory). (Not really good for gluster, of course, due to heavy metadata activity).


>> I would like to recommend to consider using btrfs instead of ext4 on lxc
>> containers. I think, this choice is better in any case, even the underlaying
>> filesystem is not zfs.
> 
> We can make that configurable - should be easy to implement.

Great. btrfs on loop on zfs is better than ext4 on loop on zfs.

Thanks for your attention.

Bye,
István