[PVE-User] DRBD & Proxmox scenario

[Gilou]

Le 09/03/2015 19:37, Dmitry Petuhov a écrit :
> 09.03.2015 20:33, Gilou wrote:
>> Hi,
>> 
>> I've been reading all the documentation about Proxmox & DRBD, and I
>> have questions for those who might have deployed such a setup.
>> 
>> The idea is to run 3 nodes on similar hardware, and to begin, host
>> the disks on 2 of the nodes using drbd.
> Strange setup. What third node will do? DRBD is (was?) good for
> 2-node setups. If you have 3 nodes, you'd better look at ZFS (if you
> need OpenVZ) or CEPH (if you don't need OpenVZ or ready to setup
> cephfs on nodes manually).

Third node is a node use for other VMs and a lab on RAID0, that won't
share this storage (think, developper things), and is mostly intended to
guarantee quorum for HA.

>> The wiki reference here is the most extensive info: 
>> https://pve.proxmox.com/wiki/DRBD
>> 
>> Which seems to point at the idea of having 2 LVs, one for each node
>> to host the VMs that will "mostly be on the node", and let drbd do
>> its magic as single primary setup.
>> 
>> It seems that the current way of doing things. I'm not too much a
>> fan of this idea, as it makes the nodes more "powerful" than they
>> should be in a cluster. If this is still "how things are done", so
>> be it.
>> 
>> So how to go for dual primary setup? It seems GlusterFS & OCFS2
>> aren't really neatly supported, but without a clustered filesystem,
>> what to do?
> Dual-primary DRBD was pretty unstable few years ago. Even without
> filesystem (as LVM storage like in wiki). Adding filesystem layer
> will add its bugs to setup.

Hmmm, as in, with Proxmox you mean? DRBD by itself works nicely in dual
primary mode, as long as you take care of the locking on top of it. It
actually works really nicely outside of Proxmox once exported as NFS ;)

> 
>> Ceph doesn't seem any more ready for show time, and it doesn't seem
>> like a good idea to run it off proxmox nodes atm?
>> 
>> I only dealt with storage outside of Proxmox until now (using
>> drbd, btw), but I have a case where I'd like to avoid having to put
>> on a separate setup for it, and am left wondering.
> Just look at other storages. DRBD isn't what you want for >2 nodes.

That's for later, for now, the 2 nodes needs to talk to each others. If
we need to work on 3 nodes, we'll just externalize the storage on a SAN.

Thanks for your feedback,

Gilles