[PVE-User] Proxmox VE 3.4 and ZFS: Using compression=lz4 instead ofcompression=on by default

[Daniel Mettler]

That's a valid question, but a different one ;)

My post was about _which_ compression algorithm to use and the current recommendation is to use lz4 for most use cases.

Proxmox VE 3.4 currently enables compression (compression=on) by default, but uses a different algorithm (probably lzjb as this still seems to be the default for ZoL; using 'zfs get compression' this can't be determined, unfortunately - one would probably have to check the sources of ZoL).

The question _why_ to use (transparent) compression for root & user partitions at all:

With ZFS, when using an efficient and effective compression algorithm like lz4, one can both save space and even increase the overall disk I/O performance in most cases, i.e. particularly in modern servers with fast and usually "idle"/underused multi-core CPUs (which can process data much faster than any raid array of disks or SSDs ever could). If compression is used, less data has to be written to and read from the raid/disks/SSDs (also reducing the load on the according bus, but that's usually not the bottleneck anyway), making this relatively slow task considerably quicker, whereas the tiny bit of additional time the CPU spends decompressing/compressing that data is almost negligible.

Cheers

Dani

-- 
Daniel Mettler                                 http://www.numlock.ch

----- Original Message -----
From: "Lindsay Mathieson" <lindsay.mathieson at gmail.com>
To: pve-user at pve.proxmox.com
Sent: Thursday, March 19, 2015 4:23:02 AM
Subject: Re: [PVE-User] Proxmox VE 3.4 and ZFS: Using compression=lz4 instead ofcompression=on by default




Why would you need disk compression on the root & user partitions? 

Lindsay Mathieson 

From: Daniel Mettler 
Sent: ‎19/‎03/‎2015 9:32 AM 
To: pve-user at pve.proxmox.com 
Subject: [PVE-User] Proxmox VE 3.4 and ZFS: Using compression=lz4 instead ofcompression=on by default 

Servus! 

My questions: 
************* 

1) Will there soon be a Proxmox VE ISO that uses lz4 for transparent ZFS compression by default, i.e. at installation time already? 

2) Is there any documentation on how the Proxmox VE 3.4 ISO image is being built/created (particularly, how it's made bootable)? 

3) Why isn't it possible to activate and use other consoles (e.g. by hitting CTRL-ALT-F4) during the Proxmox VE installation? Or is it possible? 

Background: 
*********** 

I tried various workarounds to make Proxmox VE 3.4 use compression=lz4 for ZFS at installation time already (reason: lz4 is considered the best choice for ZFS compression algorithm in most cases. AFAIK, ZoL still uses lzjb by default, but there are plans to move to lz4 soon). 

I finally decided, the best way for us to go is creating a bootable, patched ISO based on the original Proxmox VE 3.4 ISO. 

Here’s how I created the patched ISO: 

1. Loop-mount the original ISO (can only be mounted read-only): 

# mount -o loop /path/to/proxmox-ve_3.4-3f2d890e-1.iso /mnt/cdrom 

2. Copy /usr/bin/proxinstall from the mounted ISO image to a read-writeable directory (create directories as necessary): 

# cp -a /mnt/cdrom/usr/bin/proxinstall /mnt/cdrom_patched/usr/bin/proxinstall 

3. Edit the /usr/bin/proxinstall script as follows: 

# diff -u ../cdrom/usr/bin/proxinstall usr/bin/proxinstall 
--- ../cdrom/usr/bin/proxinstall 2015-02-12 17:52:50.000000000 +0100 
+++ usr/bin/proxinstall 2015-03-17 21:50:07.662031284 +0100 
@@ -592,7 +592,7 @@ 
# disable atime during insatll 
syscmd ("zfs set atime=off $zfspoolname") == 0 || 
die "unable to set zfs properties\n"; 
- syscmd ("zfs set compression=on $zfspoolname") == 0 || 
+ syscmd ("zfs set compression=lz4 $zfspoolname") == 0 || 
die "unable to set zfs properties\n"; 
} 

4. Use the complicated but very powerful tool xorriso (install it, if necessary) to create a bootable ISO image based on the original ISO, but with /usr/bin/proxinstall “overwritten” with the edited proxinstall script: 

# xorriso -boot_image grub patch -indev proxmox-ve_3.4-3f2d890e-1.iso -overwrite on -outdev proxmox-ve_3.4-3f2d890e-1-with-lz4-patch.iso -blank as_needed -pathspecs on -add /usr/bin/proxinstall=/mnt/cdrom_patched/usr/bin/proxinstall -- -commit 

5. Use this patched ISO to install Proxmox VE 3.4 as usual 

After rebooting the installed Proxmox VE 3.4 host/server, use the following command to verify that lz4 has been used by default: 

# zfs get compression 
NAME PROPERTY VALUE SOURCE 
rpool compression lz4 local 
rpool/ROOT compression lz4 inherited from rpool 
rpool/ROOT/pve-1 compression lz4 inherited from rpool 
rpool/swap compression lz4 inherited from rpool 

I hope this is of help to other users as well. And of course, that an official ISO implementing this small, but important improvement will be released soon :) 

Cheers 

Dani 

-- 
Daniel Mettler http://www.numlock.ch 

_______________________________________________ 
pve-user mailing list 
pve-user at pve.proxmox.com 
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user 

_______________________________________________
pve-user mailing list
pve-user at pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user