[PVE-User] Ceph Cache Tiering
elacunza at binovo.es
Mon Oct 10 18:05:31 CEST 2016
The choice of filesystem is not "largely irrelevant"; filesystems are
quite complex and the choice is relevant. With ZFS, you're in unknown
territory AFAIK as it is not regularly tested in ceph development; I
think only ext4 and XFS are regularly tested. And there are known
limits/problems with ext4 for example, and seems that also apply to
zfsonlinux (I think ext4 has lower limit yet):
Also no word about ZFS in recommendations:
It can be done? Yes - Lindsay is doing it successfully.
Is it advisable? - I don't think so. :-)
Anyway it seems they're getting rid of the filesystem in the near future
with bluestore ;)
El 10/10/16 a las 16:29, Adam Thompson escribió:
> The default PVE setup puts an XFS filesystem onto each "full disk" assigned to CEPH. CEPH does **not** write directly to raw devices, so the choice of filesystem is largely irrelevant.
> Granted, ZFS is a "heavier" filesystem than XFS, but it's no better or worse than running CEPH on XFS on Hardware RAID, which I've done elsewhere.
> CEPH gives you the ability to not need software or hardware RAID.
> ZFS gives you the ability to not need hardware RAID.
> Layering them - assuming you have enough memory and CPU cycles - can be very beneficial.
> Neither CEPH nor XFS does deduplication or compression, which ZFS does. Depending on what kind of CPU you have, turning on compression can dramatically *speed up* I/O. Depending on how much RAM you have, turning on deduplication can dramatically decrease disk space used.
> Although, TBH, at that point I'd just do what I have running in production right now: a reasonably-powerful SPARC64 NFS fileserver, and run QCOW2 files over NFS. Performs better than CEPH did on 1Gbps infrastructure.
>> -----Original Message-----
>> From: pve-user [mailto:pve-user-bounces at pve.proxmox.com] On
>> Behalf Of Lindsay Mathieson
>> Sent: October 10, 2016 09:21
>> To: pve-user at pve.proxmox.com
>> Subject: Re: [PVE-User] Ceph Cache Tiering
>> On 10/10/2016 10:22 PM, Eneko Lacunza wrote:
>>> But this is nonsense, ZFS backed Ceph?! You're supposed to give full
>>> disks to ceph, so that performance increases as you add more disks
>> I've tried it both ways, the performance is much the same. ZFS also
>> increases in performance the more disks you throw it, which is passed
>> onto ceph.
>> +Auto Bit rot detection and repair
>> +A lot of flexibility
>> Lindsay Mathieson
>> pve-user mailing list
>> pve-user at pve.proxmox.com
> pve-user mailing list
> pve-user at pve.proxmox.com
Zuzendari Teknikoa / Director Técnico
Binovo IT Human Project, S.L.
Astigarraga bidea 2, planta 6 dcha., ofi. 3-2; 20180 Oiartzun (Gipuzkoa)
More information about the pve-user