[PVE-User] Shared storage on NAS speed - LVM(over iSCSI) vs NFS

[Mikhail]

Hello,

Having another issue here that I can't figure out.

I have a 2-node cluster here, running Proxmox 4.4.
This cluster is using shared storage that connects to a Linux storage
server using iSCSI via direct (NIC-to-NIC) 10-Gbit LAN (storage server
has 2x10-Gbit ports network cards; Proxmox nodes also have 10Gbit LAN NICs).
My storage model for the VMs is LVM-over-iSCSI (storage.cfg from one of
the nodes):

iscsi: Storage
	portal 192.168.4.1
	target iqn.2016-03.eu.company:storage.all.luns
	content none

lvm: WWW-hosts
	vgname vg-www-hosts
	base Storage:0.0.1.scsi-1494554000000000060fd8f3f2072a735bf2387ea56fe72a0
	content images
	shared 1

nfs: vmnfs
	export /mnt/vmnfs
	path /mnt/pve/vmnfs
	server 192.168.4.1
	content images
	maxfiles 1
	options vers=4

Problem is that LVM access to the virtual machine's LVs is terribly
slow, while NFS transfers are fast enough (storage server has 4x4TB SAS
drives in RAID10 configured with MDADM).

Here's 2 simple test I performed to get the transfer speeds:

1) For the LVM-over-iSCSI transfers:

(on the proxmox node)

# dd if=/dev/vg-www-hosts/vm-82105-disk-1 of=vm-82105-disk-1 bs=1M
count=3000
3000+0 records in
3000+0 records out
3145728000 bytes (3.1 GB) copied, 70.0969 s, 44.9 MB/s
#

2) For the NFS transfer from the same storage server (NFS is configured
as above):

root at pm1:~# dd if=/mnt/pve/ISOimages/template/iso/myOS.iso of=myOS.iso
bs=1M count=3000
3000+0 records in
3000+0 records out
3145728000 bytes (3.1 GB) copied, 1.17121 s, 2.7 GB/s
root at pm1:~#

So I cannot figure out why LVM-over-iSCSI is so slow. I'm using
iscsitarget package on Debian Jessie on the storage server as a iSCSI
target. Could this be due to iscsitarget package? As I wrote earlier,
iscsitarget appears to be abandoned in Debian Stretch, which leads me to
question if this is the right iSCSI target software I'm using here..

Thanks!