[PVE-User] Ceph Bluestore - lvmcache versus WAL/DB on SSD

[Mark Schouten]

On Tue, Jun 30, 2020 at 12:07:40AM +1000, Lindsay Mathieson wrote:
> As per the title :) I have 23 OSD spinners on 5 hosts, Data+WAL+DB all on
> the disk. All VM's are windows running with Writeback Cache. Performance is
> adequate but see occasional high IO loads that make the VM's sluggish.

Could be that (deep) scrubs are periodically killing your performance.
There are some tweaks available to make them less invading:

osd_scrub_chunk_min=20 # 5
osd_scrub_sleep=4 # 0

And then some:
https://docs.ceph.com/docs/master/rados/configuration/osd-config-ref/

Best option is really to place spinning disks with SSD..

-- 
Mark Schouten     | Tuxis B.V.
KvK: 74698818     | http://www.tuxis.nl/
T: +31 318 200208 | info at tuxis.nl