[PVE-User] [Extern] - Re: "nearfull" status in PVE Dashboard not consistent

[David der Nederlanden | ITTY]

Hi Frank,

The images didn't work :)

Pool and osd nearfull are closely related, when OSD's get full your pool also gets nearfull as Ceph needs to be able to follow the crush rules,
which it can't if one of the OSD's gets full, hence it warns when it gets nearfull.

I see that you're mixing OSD sizes, deleting and recreating the OSD's one by one caused this, as the OSD's got new weights you should be OK when you reweight them.
You can do this by hand or using reweight-by-utilization, what you prefer.

Not quite sure about the pool sizes, but an RBD pool with a 2/3 min/max rule should never be above 80%, as this gives you a nearfull pool when it starts backfilling when you lose one node, or even a full pool worst case, rendering the pool read only.

Kind regards,
David



-----Oorspronkelijk bericht-----
Van: pve-user <pve-user-bounces at lists.proxmox.com> Namens Frank Thommen
Verzonden: Tuesday, September 10, 2024 14:02
Aan: pve-user at lists.proxmox.com
Onderwerp: Re: [PVE-User] [Extern] - Re: "nearfull" status in PVE Dashboard not consistent

Yes, but Ceph also reports five(!) nearfull pools, but the pool overview doesn't reflect that. I'd love to show this with CLI, but I could not find an appropriate command so far. Hopefully, inline images work:

 From the Ceph overview dashboard:

And from the pool overview

Two pools are quite full (80% and 82%), but far from nearfull, and the other tree are basically empty.

Frank


On 09.09.24 12:36, Eneko Lacunza via pve-user wrote:
> Hi Frank,
>
> El 8/9/24 a las 14:17, Frank Thommen escribió:
>>  5 hdd  3.81450 1.00000  3.8 TiB  3.3 TiB  3.1 TiB   37 MiB  8.5 GiB
>> 568 GiB 85.45  1.16  194      up
>> 16    hdd  1.90039   1.00000  1.9 TiB  1.6 TiB  1.6 TiB   19 MiB 5.4 
>> GiB  261 GiB  86.59  1.18   93      up
>
> Those OSD are nearfull, if you have the default 0.85 value for that. 
> You can try to lower a bit their weight.
>
> Cheers
>
> Eneko Lacunza
> Zuzendari teknikoa | Director técnico
> Binovo IT Human Project
_______________________________________________
pve-user mailing list
pve-user at lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user