[pve-devel] ceph replication speed

Eric Blevins eric at netwalk.com
Mon Dec 30 18:20:10 CET 2013

> I only run 6 OSDs on 3 nodes - so maybe not enough OSDs?

When you write to just three OSDs, each one replicates to another OSD, a 
total of 6 OSD writes.
Each OSD gets one write, the client only sees the bandwidth of the three 
it wrote to.

When you write to only three OSDs, each one replicates to two more OSDs, 
a total of 9 OSD writes.
Half of the OSDs make two writes, I would expect reduced performance 
with 50% of the OSDs doing twice the work.

I suspect if you added three more OSDs then size=3 would perform at 
100MB/sec and size=2 would be faster than 100MB/sec.

More information about the pve-devel mailing list