[PVE-User] Supported configuration question
pongracz.istvan at gmail.com
Sat Sep 21 22:16:09 CEST 2013
Feladó: "Enrico M" enrico at majaglug.net
Címzett: pve-user at pve.proxmox.com
Dátum: Sat, 21 Sep 2013 17:24:57 +0200
>> So, technically you can adjust proxmox as you need. If one not a tech.
>> genie, no problem, just follow the mainstream: a supported hardware raid
>> card + some disks and that's all.
> I'm interested in zfs on linux + ssd for zil and l2arc.
> Could you tell some details about your configuration?
> Thanks a lot
- one SSD for boot/root filesystem, dedicated to proxmox itself.
- two SSD, partitioned for L2ARC and ZIL (mirrored, some GBs enough)
- 4 WD green drives (anyway, green drives not really recommended) in raid10 equivalent
- Here is the pool layout
scan: scrub repaired 0 in 5h59m with 0 errors on Sat Sep 21 10:11:02 2013
NAME STATE READ WRITE CKSUM
zfsstorage1 ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
scsi-SATA_WDC_WD15EARS-00_WD-WMAZA3142219 ONLINE 0 0 0
scsi-SATA_WDC_WD15EARS-00_WD-WCAYY0094106 ONLINE 0 0 0
mirror-1 ONLINE 0 0 0
scsi-SATA_WDC_WD15EARS-00_WD-WCAZA3999177 ONLINE 0 0 0
scsi-SATA_WDC_WD15EARS-00_WD-WMAZA0845754 ONLINE 0 0 0
mirror-2 ONLINE 0 0 0
scsi-SATA_OCZ-AGILITY3_OCZ-35CM9ZXZ3O5XOC90-part2 ONLINE 0 0 0
scsi-SATA_OCZ-AGILITY3_OCZ-Z8J00QD2S3RG4SXQ-part2 ONLINE 0 0 0
scsi-SATA_OCZ-AGILITY3_OCZ-35CM9ZXZ3O5XOC90-part1 ONLINE 0 0 0
scsi-SATA_OCZ-AGILITY3_OCZ-Z8J00QD2S3RG4SXQ-part1 ONLINE 0 0 0
- I created the vz layout under zfsstorage1 (private, images, dump, template), including all virtual machines in their own dataset, like this example:
- In proxmox gui I set a new directory, pointed to zfsstorage1/vz layout.
- use standard sata controllers without any cache or turn off cache, avoid hardware raid cards
- zfs 6.2 is the latest and greatest, easy to compile
- when you upgrade your kernel, make sure you compile the zfs/spl for it and put into initrd
- limit your ARC (depends on your memory requirements, 1/4 of your physical ram is safe), example: limit to 4GB:
options zfs zfs_arc_max=4294967296
- good balance between performance and compress ratio with lz4 compression
- if you have issues with proxmox populates your vz layout and cannot mount zfs datasets, do the following:
* stop pve-cluster, pve-manager, apache2, get them out from starting services
* wipe out the vz layout (be careful and check you do not delete live data)
* write zfs mount -a and after that start these services stopped before in rc.local
- consider to put swap space to a zvol instead of ssd/hard drive
- use zfs-auto-snapshot and send/recv instead of pve provided backup
- on reliable network, you can replicate your zfs datasets incrementally via ssh using zfs send/recv of snapshots (I use openvpn for off-site backup, very reliable)
- you can play with blocksize, metadata/all cache for arc and l2arc for individual datasets, for example you can select optimal blocksize and cache (primarycache=metadata) for mysql and postgresql to get maximum performance
- you can clone existing dataset to save space, (plan ahead),
- you can use cluster filesystems on top of zfs, like glusterfs, afs, whatever
More information about the pve-user