[PVE-User] Ceph or Gluster

[Eneko Lacunza]

Hi Mohamed,

El 22/04/16 a las 15:00, Mohamed Sadok Ben Jazia escribió:
> Thank you Eneko,
> I read in proxmox forum that distributed storage needs 10GBit or faster on
> the local network and a dedicated network.
> Could you detail your used infrastructure to see if it matches those
> conditions?
We only have small/tiny clusters (3 clusters). I'll detail the one in 
our office that is the "biggest".

4 Proxmox 4.x nodes:
     - 1 node for backup/helper: 2 cores, 16GB RAM, NFS export for 
backups/ISO
     - 3 main nodes: 4 cores (one with HT), 32 GB RAM, 2x1gbit ethernet
         - Execute VMs (46 total, 17 running right now)
         - Ceph storage: each node has
             - 1 MDS daemon
             - 3x1TB OSD disk
             - 1xSSD disk for OSD journals

We are using 1gbit interface for VM access, Proxmox cluster. The other 
1gbit port is for ceph public/private networks. We're using size=2 for 
Ceph. (Two replicas for each data).

With this setup we only saturate the ceph network in special backup 
times (weekend), network is below 20% of average use normally.

Of course, this depends on your loads, also if you're planning nodes 
with more OSD, then it is easier to saturate network.

Also, you have to take into account failures; what happens when 
something brokes (node, OSD disk) and how long it will take until 
replica count is restored. In this, your network can also have a big impact.

What size of storage/virtualization load are you planning?

Cheers
Eneko

-- 
Zuzendari Teknikoa / Director Técnico
Binovo IT Human Project, S.L.
Telf. 943493611
       943324914
Astigarraga bidea 2, planta 6 dcha., ofi. 3-2; 20180 Oiartzun (Gipuzkoa)
www.binovo.es