[PVE-User] ZFS on iSCSI + Pacemaker/corosync/DRBD

[Mark Adams]

Hi,

Thanks for the response.

I was planning on using active/backup bonding on 10Gbe for my network fault
tolerance so no multipath support shouldn't be an issue.

I've come across some strange behaviour with the iet provider though, in
that after 9 LUNS it starts changing the existing luns rather than adding
additional.

Hard disk config in proxmox for VM:

Hard Disk (virtio0) ZFSOVERISCSI:vm-112-disk-1,size=10G
Hard Disk (virtio1) ZFSOVERISCSI:vm-112-disk-2,size=10G
Hard Disk (virtio2) ZFSOVERISCSI:vm-112-disk-3,size=10G
Hard Disk (virtio3) ZFSOVERISCSI:vm-112-disk-4,size=10G
Hard Disk (virtio4) ZFSOVERISCSI:vm-112-disk-5,size=10G
Hard Disk (virtio5) ZFSOVERISCSI:vm-112-disk-6,size=10G
Hard Disk (virtio6) ZFSOVERISCSI:vm-112-disk-7,size=10G
Hard Disk (virtio7) ZFSOVERISCSI:vm-112-disk-8,size=10G
Hard Disk (virtio8) ZFSOVERISCSI:vm-112-disk-9,size=10G
Hard Disk (virtio9) ZFSOVERISCSI:vm-112-disk-10,size=10G

ietd.conf file on zfs/iscsi storage host:

Lun 0 Path=/dev/VMSTORE/vm-112-disk-1,Type=blockio
Lun 1 Path=/dev/VMSTORE/vm-112-disk-2,Type=blockio
Lun 2 Path=/dev/VMSTORE/vm-112-disk-3,Type=blockio
Lun 3 Path=/dev/VMSTORE/vm-112-disk-4,Type=blockio
Lun 4 Path=/dev/VMSTORE/vm-112-disk-6,Type=blockio
Lun 5 Path=/dev/VMSTORE/vm-112-disk-7,Type=blockio
Lun 6 Path=/dev/VMSTORE/vm-112-disk-8,Type=blockio
Lun 7 Path=/dev/VMSTORE/vm-112-disk-9,Type=blockio
Lun 8 Path=/dev/VMSTORE/vm-112-disk-10,Type=blockio

as you can see, "disk-5" is missing since I added "disk-10"

Is anyone using zfs over iscsi with iet? have you seen this behaviour?

Thanks,
Mark

On 23 November 2016 at 20:40, Michael Rasmussen <mir at miras.org> wrote:

> On Wed, 23 Nov 2016 09:40:55 +0000
> Mark Adams <mark at openvs.co.uk> wrote:
>
> >
> > Has anyone else tried to get this or a similar setup working? Any views
> > greatly received.
> >
> What you are trying to achieve is not a good idea with
> corosync/pacemaker since iSCSI is a block device. To create a cluster
> over a LUN will require a cluster aware filesystem like NFS, CIFS etc.
> The proper way of doing this with iSCSI would be using multipath to a
> SAN since iSCSI LUNs cannot be shared. Unfortunately the current
> implementation of ZFS over iSCSI does not support multipath (a
> limitation in libiscsi). Also may I remind you that Iet development has
> stopped in favor of LIO targets (http://linux-iscsi.org/wiki/LIO). I am
> currently working on making an implementation of LIO for proxmox which
> will use a different architecture than the current ZFS over iSCSI
> implementation. The new implementation will support multipath. As this
> is developed in my spare time progress is not a high as it could be.
>
> Alternatively you could look at this:
> http://www.napp-it.org/doc/downloads/z-raid.pdf
>
> --
> Hilsen/Regards
> Michael Rasmussen
>
> Get my public GnuPG keys:
> michael <at> rasmussen <dot> cc
> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
> mir <at> datanom <dot> net
> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
> mir <at> miras <dot> org
> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
> --------------------------------------------------------------
> /usr/games/fortune -es says:
> The computer should be doing the hard work.  That's what it's paid to
> do, after all.
>                 -- Larry Wall in <199709012312.QAA08121 at wall.org>
>
> _______________________________________________
> pve-user mailing list
> pve-user at pve.proxmox.com
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>