[PVE-User] PVE HBA best practice

[Ronny Aasen]

On 20.04.2018 17:12, Gilberto Nunes wrote:
> Hi there PVE Users!
>
> I have here a PowerEdge Dell R740 and a Storage MD1400.
> This is a DAS Dell solution.
> Further, we will have another server, that we will buy later, to compose a
> cluster/HA solution.
> So, what is the proper way to use the HBA within Proxmox, in order to make
> it reliable and prepared to work later as Cluster/HA solution?
> May I have ZFS pools?
> May I have see the disk as local discs, and use it directly or what?
>
> I need some guide in this matter, some doc will be appreciated.
>
> Thanks a lot.
> ---
> Gilberto Nunes Ferreira


if you only have directly attached storage it is hard to make a HA 
solution. since once the server with the storage dies the other can not 
see it any more.

normally in proxmox you have 3 servers (you can use a third lightweight 
server as witness if you only need 2 servers. eg a raspberry pi or 
similar) All the 3 servers can access the same storage. and run the same 
vm's from the shared storage. the shared storage can be a ceph cluster. 
a SAN, or NAS solution. but it should also be HA or you have just moved 
the Single point of failure from the vm hypervisor to the storage 
solution.    Now if a server dies, the leftover quorum of 2 servers 
decides the one server is gone and starts it's VM's on one of the 2 
remaining servers.

normally the single server loosing connection to the quorum should stop 
all vm's. but you should implement a STONITH  solution to prevent vm 
disk image corruption.

with directly attached storage in servers you have a problem. you can 
either have storage in all servers and use zfs replication between them. 
it will not be as up to date as a shared system when it fail tho. Or you 
can build a HA storage system as a backend for your proxmox HA 
virtualization system.

a server in front of the MD1400 can be a NFS/ISCSI server allowing 
multiple proxmox servers to use it as shared storage for proxmox HA. the 
storage solution is single point of failure.

if you can connect the MD1400 to 2 servers at the same time you may have 
some kind of SAS  failover cluster.  something like HA-LVM 
https://www.linuxjournal.com/content/high-availability-storage-ha-lvm. 
MD1400 will be the new single point of failure. but it is less complex 
then a server so less likely to die. but HA-LVM is added complexity. and 
sometimes a simple NFS server is good enough (tm)

if you have 4 similarly sized servers you can build a ceph cluster. 
https://pve.proxmox.com/wiki/Ceph_Server  you can even install proxmox 
and ceph together on 4 nodes in a hyperconverged solution, but this do 
require intimate knowledge of ceph and proxmox and enough juice in your 
server to deal with both jobs at the same time (especially in a failure 
situation when load will spike) 
https://pve.proxmox.com/wiki/Hyper-converged_Infrastructure

good luck with proxmox !
Ronny Aasen