[PVE-User] High ceph OSD latency

Lindsay Mathieson lindsay.mathieson at gmail.com
Thu Jan 15 13:17:07 CET 2015

On Thu, 15 Jan 2015 11:25:44 AM Fabrizio Cuseo wrote:
> What is strange is that on OSD tree I have high latency: tipically Apply
> latency is between 5 and 25, but commit lattency is between 150 and 300
> (and sometimes 5/600), with 5/10 op/s and some B/s rd/wr (i have only 3
> vms, and only 1 is working now, so the cluster is really unloaded).
> I am using a pool with 3 copies, and I have increased pg_num to 256 (the
> default value of 64 is too low); but OSD latency is the same with a
> different pg_num value.
> I have other clusters (similar configuration, using dell 2950, dual ethernet
> for ceph and proxmox, 4 x OSD with 1Tbyte drive, perc 5i controller), with
> several vlms, and the commit and apply latency is 1/2ms.
> Another cluster (test cluster) with 3 x dell PE860, with only 1 OSD per
> node, have better latency (10/20 ms).
> What can i check ? 

POOMA U, but if you have one drive or controller that is marginal or failing, 
it can slow down the whole cluster.

Might be worth while benching individual osd's

More information about the pve-user mailing list