[pve-devel] ceph replication speed

[Eric Blevins]

> I only run 6 OSDs on 3 nodes - so maybe not enough OSDs?
>
>

size=2:
When you write to just three OSDs, each one replicates to another OSD, a 
total of 6 OSD writes.
Each OSD gets one write, the client only sees the bandwidth of the three 
it wrote to.

size=3:
When you write to only three OSDs, each one replicates to two more OSDs, 
a total of 9 OSD writes.
Half of the OSDs make two writes, I would expect reduced performance 
with 50% of the OSDs doing twice the work.

I suspect if you added three more OSDs then size=3 would perform at 
100MB/sec and size=2 would be faster than 100MB/sec.