[PVE-User] Ceph or Gluster

[Brian ::]

Hi Mohamed

10Gbps or faster at a minimum or you will have pain. Even using 4
nodes with 4 spinner disks in each node and you will be maxing out
1Gbps network. For any backfills or adding new OSDs you don't want to
be waiting on 1Gbps ethernet speeds.

Dedicated 10Gbps network for ceph communication at a minimum and you
will have nice results.



On Fri, Apr 22, 2016 at 2:00 PM, Mohamed Sadok Ben Jazia
<benjazia.mohamedsadok at gmail.com> wrote:
> Thank you Eneko,
> I read in proxmox forum that distributed storage needs 10GBit or faster on
> the local network and a dedicated network.
> Could you detail your used infrastructure to see if it matches those
> conditions?
>
>
>
> On 22 April 2016 at 12:06, Eneko Lacunza <elacunza at binovo.es> wrote:
>
>> Hi Mohamed,
>>
>> El 22/04/16 a las 12:42, Mohamed Sadok Ben Jazia escribió:
>>
>>> Hello list,
>>> In order to set a high scalable proxmox infrastructure with a number of
>>> clusters, i plan to use distributed storing system, for this i have some
>>> questions.
>>> 1- I have a choice between Ceph and Gluster, which is better for proxmox.
>>>
>> I have no experience with Gluster, Ceph has been great for our use.
>>
>>> 2- Is it better to install one of those systems on the nodes or on
>>> separated servers.
>>>
>> Better on separated systems, but works quite well on the same systems if
>> the load is ok. Proxmox Ceph Server integration is very nice and saves lots
>> of work.
>>
>>> 3- Can this architecture realise a stable product, with VM and LXC
>>> migration (not live migration), store backups and snapshots, store iso
>>> files and lxc container templates.
>>>
>> In order to use Ceph for backups and ISO/templates, you'll have to use
>> CephFS. It is considered experimental in current Ceph version in Proxmox
>> (Hammer) but today a new Ceph LTS version has been released (Jewel), that
>> marks CephFS stable and production-ready. I think this will be integrated
>> in Proxmox shortly, developers are talking about this in the mailing list
>> today.
>>
>> I use NFS for backups and ISO/templates. For our storage needs this is
>> enough.
>>
>> Cheers
>> Eneko
>>
>> --
>> Zuzendari Teknikoa / Director Técnico
>> Binovo IT Human Project, S.L.
>> Telf. 943493611
>>       943324914
>> Astigarraga bidea 2, planta 6 dcha., ofi. 3-2; 20180 Oiartzun (Gipuzkoa)
>> www.binovo.es
>>
>> _______________________________________________
>> pve-user mailing list
>> pve-user at pve.proxmox.com
>> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>>
> _______________________________________________
> pve-user mailing list
> pve-user at pve.proxmox.com
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user