[PVE-User] Three node Hyperconverged PVE+Ceph and failure domains...
gaio at sv.lnf.it
Wed Mar 10 11:47:31 CET 2021
One of the most interesting configuration of PVE is the three node,
switchless (full mesh) configuration, depicted in some PVE docs, most
But lurking 'ceph-user' mailing list, some weeks ago, lead to an
interesting discussion about 'failure domains', and many user depicted
the three node cluster as 'insecure'.
The reasoning are about:
a) 'min_size = 2' is a must if you need to keep your data safe; you can
set 'min_size = 1', but clearly there's no scrub/checksumming, so no
real guarantee against data corruption.
b) but in a three node setup, with 'min_size = 2', if a node goes down,
the cluster switch in 'readonly' at the very first subsequent failure,
eg the cluster does not handle more then a failure.
c) you can change the failure domain, eg:
mon osd down out subtree limit = osd
but in this way you have to guarantee (at worst case) room for the
double of the space on a single node (eg, three node cluster with 2TB of
space each, to guarantee the 'min_size = 2' you cannot use more then 1TB
space on overral cluster; so, a 6TB total disk space for a 1TB usable
I'm wrong? If not, the 3-node hyperconverged cluster is suitable only
dott. Marco Gaiarin GNUPG Key ID: 240A3D66
Associazione ``La Nostra Famiglia'' http://www.lanostrafamiglia.it/
Polo FVG - Via della Bontà, 7 - 33078 - San Vito al Tagliamento (PN)
marco.gaiarin(at)lanostrafamiglia.it t +39-0434-842711 f +39-0434-842797
Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA!
(cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA)
More information about the pve-user