[PVE-User] Ceph Cache Tiering
elacunza at binovo.es
Mon Oct 10 14:22:00 CEST 2016
El 10/10/16 a las 13:46, Lindsay Mathieson escribió:
> On 10/10/2016 8:19 PM, Brian :: wrote:
>> I think with clusters with VM type workload and at the scale that
>> proxmox users tend to build < 20 OSD servers that cache tier is adding
>> layer of complexity that isn't going to payback. If you want decent
>> IOPS / throughput at this scale with Ceph no spinning rust allowed
> I think you're right, for all the talk of small scale deployments and
> commodity hardware, ceph on the small business scale is a poor
> price/performance ratio. To get decent performance out of it you have
> to spend big on terrabyte SSD's, battery backed disk controllers etc.
We have multiple small scale ceph clusters in SMB, all the clients and
ourselves very happy so far :)
> Its a shame, the flexibility of it is really good and rock solid in
> the 9.x range if you don't push it :) Its ability to quickly heal only
> dirty data is outsanding. I just tested a simple 3 node setup backed
> by our ZFS pools and was only getting 50MB/s seq writes. If I added
> SSD journals that probably would have got to the 100MB/s level, which
> would have been good enough, but there is one deal breaker for us and
> thats snapshots - they are incredibly slow to restore, 45 minutes for
> one that was only a few minutes old. It gets worse the more writes you
> add. qcow2 snapshots on gluster only take a couple of minutes, and we
> use snapshots a lot for testing development and support.
But this is nonsense, ZFS backed Ceph?! You're supposed to give full
disks to ceph, so that performance increases as you add more disks! :)
Can't comment on snapshots, we don't use them.
>> Additionally i think it is error prone.
>> I ran into the problem, that a ssd stuck because it was full causing
>> the complete storage to stuck.
> It does seem to be very much a work in progress :)
I think cache pools are in better shape in 10.2.x, but we haven't used them.
Zuzendari Teknikoa / Director Técnico
Binovo IT Human Project, S.L.
Astigarraga bidea 2, planta 6 dcha., ofi. 3-2; 20180 Oiartzun (Gipuzkoa)
More information about the pve-user