[PVE-User] ZFZ questions - performance and caching

[Mikhail]

Hi Gregor,

How do you plan to setup backup on this server? Are you going to use
Proxmox Backup Server for this, or just plain vzdump files?

If you're considering PBS, which you should, and you have at least a
pair of NVME devices available, then I would set ZFS to use those NVME
as special device. We have similar setup deployed recently - 12x3,5 HGST
14TB SAS disks + 2 x Intel 1.6TB NVME (using rear bays, Supermicro 2U
platform). This server is purely used as PBS. ZFS is built on top of 11
x 14TB SAS drives in RAIDZ-3 config (I know not optimal for speed, but
we want to see all space available under single pool) and NVME disks are
used as ZFS special device (mirrored). Remaining 12th SAS disk is set to
be hot-spare under ZFS pool. This gives us ~100TB of space and we are
satisfied with RAIDZ-3 speeds we get - we're able to read and write at 1
Gigabyte/s, which is enough for our case as we're limited by 10gbit
networking. Using NVME as SLOG/L2ARC device does not seem to be optimal
in case when used with PBS. Also, if you're thinking about Supermicro
then I would add SATADOM devices for the OS - this will allow you to use
entire NVME disks for ZFS without partitioning it for OS/boot.

regards,
Mikhail.

On 3/29/21 4:28 PM, Gregor Burck wrote:
> Hi,
> 
> for calculating an backup server wich should be in worst case
> additionaly an virtualisation server we thougth about a setup with HDDs
> and for ZFZ chaching NVMes.
> 
> We plan all with enterprise products,...
> 
> I guess following Setup:
> 
> A 12 3,5 '' bay Server, 6 PVEe sockets, maybee with an 2,5'' Rear
> entension kit for the OS
> 
> Backup envirement:
> 
> 4 (or6) HDDs, 6 (or 8) TB without chaching
> 
> For virtualisation enviroment:
> 
> 4 (or6) HDDs, 6 (or 8) TB with NVMe chaching
> 
> 
> I think, for the backup enviroment there shouldn't be a chaching
> neccessary, cause here is the network the limitation.
> 
> The VE part should be a little performer, but how big should the NVMe
> cache? As I read, it is for the ZFZ log.
> And how much is the performance increasemnt?
> 
> Bye
> 
> Gregor
> 
> 
> _______________________________________________
> pve-user mailing list
> pve-user at lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>