[PVE-User] Best configuration among these

[Eneko Lacunza]

Hi Philippe,

You email is quite dense, so I'll try to help with what I have understood.

I don't know the Dell/HP disk enclosures in regard to their ability for 
HA, but I think you're spending a lot to get a Proxmox HA cluster, but 
if the storage fails (FreeNAS node), then that effort won't help.

Have you considered some HA storage like RBD/Ceph?

Also, my experience with Dell (PERC H200, H310, H710) is that 
non-certified disks work, but OMSA won't be happy. It works like a charm.

Regarding the proxy server, maybe more RAM for that VM could help with 
the high IO...

Cheers
Eneko

On 05/02/15 09:59, Philippe Schwarz wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi,
> i'm going to receive a pack of three servers.
> Dell R730
> 1 X Xeon  E5 2600
> 16 GB
> 1 X Gb NIC
> 1 UPS 1500 W
>
> Going to pay for add-ons:
> Second Xeon
> + 48GB (Ensuring it's ECC RAM)
> 1 Dual 10Gbe SFP+ Intel NIC
> +IDRAC Enterprise 8GB (for fencing)
> + Hot plug PSU
> + 2 Raid1 SD Card (OS)
> + 6 CPU Proxmox Licence (base or standard)
> 5 year NBD warranty
>
> Those 3 servers will be involved in a three-way HA cluster.
>
>
> I'm wondering about the shared storage solution.
> Whether home-made (10Gbe + iSCSI/NFS) or built-in solution (Vendors
> DAS bay) :
>
> "home-made"
> FreeBSD 10.1 + ZFS + iSCSI (or NFS)
> Dell/HP  5U + 32 2,5" bays
> 2 x Intel Xeon E5-2630
> 64 GB RAM ECC
> 2 X  RAID 1GB non-volatile cache
> Firmware should'nt block non-certified disks... (Hp reseller already
> told me this would be nearly impossible)
> IDRAC Enterprise 8GB
> 2 X Hotplug PSU > 800W
> 1 X Dual 10GbE SFP+ Intel NIC
> 1 X Quad 1Gb Intel NIC
> 2 X SD card 8GB / SSD 64 GB (OS)
> + 1 (not redundant) or 2 (redundant configuration) 8 ports 10Gbe SFP+
> + 1 UPS
> 5 year NBD warranty
>
> "Built-in"
> SAS attachment to servers
> Dell/HP  5U + 32 2,5" bays
> 2 X  RAID 1GB non-volatile cache
> Firmware should'nt block non-certified disks.(Hp reseller already told
> me this would be nearly impossible)
> - - Ciphering of the data pool neither using SED drives nor having to
> pay licence fees. (Hp reseller already told me this (not using SED
> drives) would be impossible)
> - - Ability to use SSD caches without licence fees
> - - Ability to replicate to another storage without licence fees
> 1 UPS
> 5 year NBD warranty
>
>
> Despite the fact i love ZFS (snapshot, easy control, high IOPS,..) ,
> i'm not against having a look at a black box (In fact i'm not sure,
> i'd love to..)
>
> Dell's offer is a MD3200 with 4X900GB disks,3 year NBD
> warranty..Cheap, but under our needs (I really don't trust Raid5 and
> can't affort RAID10, with 1.8 TB usable space).
>
> Needs :
> - - start at 5 TB of usable space .
> - - 500 clients
> - - HA for
> + 1 openLDAP/Samba 3 (soon 4) file/directory servers
> + few Windows servers (antivirus, WSUS, autocad,solidworks licence,..)
> + 1 proxy squid/Squidguard (100Mb optical fiber Wan link) (this one is
> gonna eat a huge amount of IOPS)
> + 1 Web server (very poor IOPS)
> +...all the services we're going to set up in the future (Packetfence,
> LTSP,...)
>
> BTW, i'm public school, so rich enough to buy thoses equipments,
> but... no money for many years after that....
> And admin time is far cheaper than any couple of bucks ;-)
>
>
>
>
> Disk Setup:
> For 1TB SSD, my ZFS setup would be :
> On each controller (redundancy) : 6X1TB RAIDZ1 (Can afford it due to
> the tremendous level of IOPS)
> So : a mirror of 5TB raidz1
> I'm able to loose 1 controller and 1 disk on the last mirror.
> Easy to extend, huge IOPS, pretty expensive
> 12 SSD ~ 7200 €
>
>
> For 1.2TB HDD, my ZFS setup would be :
> A stripe of mirror, each 2ways mirror is split upon the 2 controllers
> ( 4 X 1.2TB mirror)
> I'm able to loose 1 controller or 1 disk on each mirror.
> Following the constructor's advices (using it's own brand disks) easy
> to extend, a bit cheaper...
> 8 HDD ~ 4500 €
>
> My questions, at last :
> - - Issue with any of these configurations ? Pros&cons
> - - Ability to evolve in a near/far future ?
> - - Issue with fencing ?
> - - How to backup ? (ZFS send/receive or rsync for the first, but wha
> about the second ??)
> - - Should i use high consumer-grade SSD (Samsung 850 Pro) for huge IOPS
> (and high level of raid of course) or entreprise-grade HDD (10Krpm,SAS
> in a RAID10 ZFS pool) for roughly the same ratio size/price ? None of
> the constructor will assume the replacement of their branded HD by
> consumer grade SSD, but i found links that proved it should work....It
> "should".. Should i run such a risk ??? If it doesn't work, i'm
> getting into big trouble.... The reseller focused on that point..
> - - Is ZFS setup wrong ?
> - - Forgot something ?
>
>
> Thanks by advance
> Best regards
>
>
> BCC to admin team
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.12 (GNU/Linux)
>
> iEYEARECAAYFAlTTMO4ACgkQlhqCFkbqHRapsACfehUn6iHFZOQydtUW2pAxWBaf
> hx8AnR1v5B9TGejA4TYHJdSodseBOGzw
> =6Xu/
> -----END PGP SIGNATURE-----
> _______________________________________________
> pve-user mailing list
> pve-user at pve.proxmox.com
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user


-- 
Zuzendari Teknikoa / Director Técnico
Binovo IT Human Project, S.L.
Telf. 943575997
       943493611
Astigarraga bidea 2, planta 6 dcha., ofi. 3-2; 20180 Oiartzun (Gipuzkoa)
www.binovo.es