[PVE-User] Ceph Journal Performance

[Lindsay Mathieson]

On Sun, 2 Nov 2014 09:58:51 AM Dmitry Petuhov wrote:
> 02.11.2014 5:18, Lindsay Mathieson пишет:
> > Have been doing a lot of testing with a three node/2 osd setup
> > - 3TB WD red drives (about 170MB/s write)
> > - 2 * 1GB Ethernet Bonded dedicated to the network filesystem
> 
> 2 OSD each node or only 2 OSD?

One OSD per node.

> You're breaking CEPH's philosophy. It's
> designed to be used with at least tens of OSDs. You can use any old/cheap
> drives. Just more-better.

Yah, bit of a learning curve here, having to adjust my preconceptions and 
expectations.

If I increased the number of OSD's to 4 per node would that improve the write 
performance?

- Could I ensue there was a copy of all data on each node?

- Would it be usable with one node down? (redundancy)


> 
> For 2 drives maybe better use DRBD.

Yes, I'm wondering if I trying to force square pegs into round wholes. I like 
ceph, especially its flexibility and potential for growth, but I'm wondering 
if its overkill and a mismatch for my requirements.

There were several things I was wanting out of  this exercise:

- Shared Storage. Migration  is essential.

- Redundancy. I need this to operate with one node down.

- Better performance. Our NAS is only just adequate. 

- Quick recovery. We have unreliable power and several times this year, even 
with a big UPS the nodes and nas have been forced to shutdown. The NAS takes a 
lot longer to come up than proxmox, meaning none of the virtual servers 
autostart.

GlusterFS has tested out well so far and is easier to setup than DRBD, but I'd 
really like to give ceph a go as it seems the future for proxmox.

> 
> > Would it be ok to use spare space on the proxmox boot/swap/system SSD? or
> > bad  idea?
> 
> If your system SSD is fast enough, then why not? PVE itself not uses much
> bandwidth on it.

Do the OSD journals all use the same raw device partition, or would I have to 
create a partition on the SSD for each OSD?

thanks,

-- 
Lindsay
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.proxmox.com/pipermail/pve-user/attachments/20141102/e146c128/attachment.sig>