[PVE-User] Ceph doesn't show all available space !
ronny+pve-user at aasen.cx
Sat Aug 25 21:42:13 CEST 2018
On 25.08.2018 00:48, Gilberto Nunes wrote:
> Ok guys
> I have success deploy a 3 node ceph server.
> In this server I have:
> OSD.0 = 4 TB
> OSD.1 = 4 TB
> OSD.2 = 4 TB
> OSD.3 = 4 TB
> OSD.4 = 3 TB
> OSD.5 = 3 TB
> OSD.6 = 3 TB
> OSD.7 = 3 TB
> OSD.8 = 3 TB
> OSD.9 = 3 TB
> OSD.10 = 2 TB
> In the Ceph cluster, the total among of data storage is:
> 11.17 GiB of 32.75 TiB
> I create a pool, with default value.
> But when I attach the ceph cluster storage to my proxmox, I just see 10 TB!
> Why I can see the 32 TB or at least 21.58 ( 32.75 - 11.17) ????
> So odd!
ceph does show you the RAW total space available, since you can
configure different pools. ie 3x replication
Since you have 3 nodes, your data will be replicated on all 3 nodes with
the default size=3 pools.
so you should in theory only be able to use 3+3+2= 8TB TB of data since
once those are full you can not have 3 copies of objects any more. IOW
you get the space of the smalles node, and the rest of the space is
"wasted" once you add a 4th node, or more disks on the small one. ceph
can use the remaining storage on the larger nodes as well.
for small clusters, you should try to make the storage homogeneous,
since it makes balancing easier..
try to make the machines more even, if you want more usable space.
More information about the pve-user