[PVE-User] Proxmox Ceph high memory usage

[Gilberto Nunes]

pve-ceph01:~# ceph status
  cluster:
    id:     e67534b4-0a66-48db-ad6f-aa0868e962d8
    health: HEALTH_WARN
            nobackfill,norebalance,norecover,nodeep-scrub flag(s) set
            394106/2589186 objects misplaced (15.221%)
            Degraded data redundancy: 124011/2589186 objects degraded
(4.790%), 158 pgs degraded, 76 pgs undersized

  services:
    mon: 5 daemons, quorum
pve-ceph01,pve-ceph02,pve-ceph03,pve-ceph04,pve-ceph05
    mgr: pve-ceph05(active), standbys: pve-ceph01, pve-ceph03, pve-ceph04,
pve-ceph02
    osd: 21 osds: 21 up, 21 in; 230 remapped pgs
         flags nobackfill,norebalance,norecover,nodeep-scrub

  data:
    pools:   1 pools, 512 pgs
    objects: 863.06k objects, 3.17TiB
    usage:   9.73TiB used, 53.0TiB / 62.8TiB avail
    pgs:     124011/2589186 objects degraded (4.790%)
             394106/2589186 objects misplaced (15.221%)
             180 active+clean
             76  active+remapped+backfill_wait
             70  active+recovery_wait
             63  active+undersized+degraded+remapped+backfill_wait
             49  active+recovery_wait+degraded+remapped
             32  active+recovery_wait+degraded
             28  active+recovery_wait+remapped
             12  active+recovery_wait+undersized+degraded+remapped
             1   active+recovering+degraded+remapped
             1   active+undersized+degraded+remapped+backfilling

  io:
    client:   694KiB/s rd, 172KiB/s wr, 118op/s rd, 38op/s wr
    recovery: 257KiB/s, 0objects/s
---
Gilberto Nunes Ferreira

(47) 3025-5907
(47) 99676-7530 - Whatsapp / Telegram

Skype: gilberto.nunes36





Em qua, 16 de jan de 2019 às 10:23, Gilberto Nunes <
gilberto.nunes32 at gmail.com> escreveu:

> Hi...
> I am using BlueStore.
> pve-manager/5.3-7/e8ed1e22 (running kernel: 4.15.18-9-pve)
>  ceph                                 12.2.10-pve1
>
> Thanks
> ---
> Gilberto Nunes Ferreira
>
> (47) 3025-5907
> (47) 99676-7530 - Whatsapp / Telegram
>
> Skype: gilberto.nunes36
>
>
>
>
>
> Em qua, 16 de jan de 2019 às 10:17, Eneko Lacunza <elacunza at binovo.es>
> escreveu:
>
>> Hi Gilberto,
>>
>> Are you using Blustore? What version of Ceph?
>>
>> El 16/1/19 a las 13:11, Gilberto Nunes escribió:
>> > Hi there
>> >
>> > Anybody else experiment hight memory usage in Proxmox CEPH Storage
>> Server?
>> > I have a 6 node PVE CEPH and after upgrade, I have noticed this high
>> memory
>> > usage...
>> > All server has 16GB of ram. I know this is not recomended, but that
>> what I
>> > have at the moment.
>> > In fact, just 3 servers ran with about 90% of memory usage.
>> > All server is IBM x3200 m2 with SATA disks...
>> > Here's ceph osd tree
>> > ceph osd tree
>> > ID  CLASS WEIGHT   TYPE NAME           STATUS REWEIGHT PRI-AFF
>> >   -1       38.50000 root default
>> >   -3        4.00000     host pve-ceph01
>> >    0   hdd  1.00000         osd.0           up  1.00000       0
>> >    1   hdd  1.00000         osd.1           up  1.00000       0
>> >    2   hdd  1.00000         osd.2           up  1.00000       0
>> >    3   hdd  1.00000         osd.3           up  1.00000       0
>> >   -5        8.00000     host pve-ceph02
>> >    4   hdd  2.00000         osd.4           up  1.00000 1.00000
>> >    5   hdd  2.00000         osd.5           up  1.00000 1.00000
>> >    6   hdd  2.00000         osd.6           up  1.00000 1.00000
>> >    7   hdd  2.00000         osd.7           up  1.00000 1.00000
>> >   -7        9.00000     host pve-ceph03
>> >    8   hdd  3.00000         osd.8           up  1.00000 1.00000
>> >    9   hdd  3.00000         osd.9           up  1.00000 1.00000
>> >   10   hdd  3.00000         osd.10          up  1.00000 1.00000
>> >   -9       12.00000     host pve-ceph04
>> >   11   hdd  3.00000         osd.11          up  1.00000 1.00000
>> >   12   hdd  3.00000         osd.12          up  1.00000 1.00000
>> >   13   hdd  3.00000         osd.13          up  1.00000 1.00000
>> >   14   hdd  3.00000         osd.14          up  1.00000 1.00000
>> > -11        1.00000     host pve-ceph05
>> >   15   hdd  0.50000         osd.15          up  1.00000       0
>> >   16   hdd  0.50000         osd.16          up  1.00000       0
>> > -13        4.50000     host pve-ceph06
>> >   17   hdd  1.00000         osd.17          up  1.00000       0
>> >   18   hdd  1.00000         osd.18          up  1.00000       0
>> >   20   hdd  1.00000         osd.20          up  1.00000       0
>> >   21   hdd  1.50000         osd.21          up  1.00000 1.00000
>> >
>> > ---
>> > Gilberto Nunes Ferreira
>> >
>> > (47) 3025-5907
>> > (47) 99676-7530 - Whatsapp / Telegram
>> >
>> > Skype: gilberto.nunes36
>> > _______________________________________________
>> > pve-user mailing list
>> > pve-user at pve.proxmox.com
>> > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>>
>>
>> --
>> Zuzendari Teknikoa / Director Técnico
>> Binovo IT Human Project, S.L.
>> Telf. 943569206
>> Astigarraga bidea 2, 2º izq. oficina 11; 20180 Oiartzun (Gipuzkoa)
>> www.binovo.es
>>
>> _______________________________________________
>> pve-user mailing list
>> pve-user at pve.proxmox.com
>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>>
>