[PVE-User] ceph rebalance/ raw vs pool usage

[Mark Adams]

Hi All,

I would appreciate a little pointer or clarification on this.

My "ceph" vm pool is showing 84.80% used. But the %RAW usage is only 71.88%
used. is this normal? there is nothing else on this ceph cluster apart from
this one pool.

Also, I notice some of my OSD's are out of balance. I've done some
reweighting using "ceph osd reweight-by-utilization" which has helped a
bit, but I think it needs to be tweaked some more (still some OSDs are over
82% utilised, while some are around 65-70%).

Is it safe enough to keep tweaking this? (I believe I should run ceph osd
reweight-by-utilization 101 0.05 15) Is there any gotchas I need to be
aware of when doing this apart from the obvious load of reshuffling the
data around? The cluster has 30 OSDs and it looks like it will reweight 13.

btw, it shows still only 58 moves if I do this with max_change 0.05
(~1.8%). Should I increase the max_change? (for example, to 0.1 or 0.15? It
appears the max I can get to even with 0.25 is 108 moves which is ~3.5%.

Or is there a better or more sensible way to get things balanced out?

Thanks in advance for your assistance, and advice gracefully received!

Regards,
Mark