[PVE-User] Windows & Gluster 3.8
Lindsay Mathieson
lindsay.mathieson at gmail.com
Wed Dec 14 13:25:56 CET 2016
When host VM's on gluster you should be setting the following:
performance.readdir-ahead: on
cluster.data-self-heal: on
cluster.quorum-type: auto
cluster.server-quorum-type: server
performance.strict-write-ordering: off
performance.stat-prefetch: on
performance.quick-read: off
performance.read-ahead: off
performance.io-cache: off
cluster.eager-lock: enable
network.remote-dio: enable
cluster.granular-entry-heal: yes
cluster.locking-scheme: granular
And possibly consider downgrading to 3.8.4?
On 14/12/2016 10:05 PM, contact at makz.me wrote:
> I've tried with ide / sata & scsi, for an existing VM it crash at boot screen
> (windows 7 logo), for new install it crash when i click next at the disk
> selector
>
>
>
> root at hvs1:/var/log/glusterfs# **gluster volume info**
>
> Volume Name: vm
> Type: Replicate
> Volume ID: 76ff16d4-7bd3-4070-b39d-8a173c6292c3
> Status: Started
> Snapshot Count: 0
> Number of Bricks: 1 x 3 = 3
> Transport-type: tcp
> Bricks:
> Brick1: hvs1-gluster:/gluster/vm/brick
> Brick2: hvs2-gluster:/gluster/vm/brick
> Brick3: hvsquorum-gluster:/gluster/vm/brick
> root at hvs1:/var/log/glusterfs# **gluster volume status**
> Status of volume: vm
> Gluster process TCP Port RDMA Port Online Pid
> \-----------------------------------------------------------------------------
> -
> Brick hvs1-gluster:/gluster/vm/brick 49153 0 Y 3565
> Brick hvs2-gluster:/gluster/vm/brick 49153 0 Y 3465
> Brick hvsquorum-gluster:/gluster/vm/brick 49153 0 Y 2696
> NFS Server on localhost N/A N/A N N/A
> Self-heal Daemon on localhost N/A N/A Y 3586
> NFS Server on hvsquorum-gluster N/A N/A N N/A
> Self-heal Daemon on hvsquorum-gluster N/A N/A Y 4190
> NFS Server on hvs2-gluster N/A N/A N N/A
> Self-heal Daemon on hvs2-gluster N/A N/A Y 6212
>
> Task Status of Volume vm
> \-----------------------------------------------------------------------------
> -
> There are no active volume tasks
>
> root at hvs1:/var/log/glusterfs# **gluster volume heal vm info**
> Brick hvs1-gluster:/gluster/vm/brick
> Status: Connected
> Number of entries: 0
>
> Brick hvs2-gluster:/gluster/vm/brick
> Status: Connected
> Number of entries: 0
>
> Brick hvsquorum-gluster:/gluster/vm/brick
> Status: Connected
> Number of entries: 0
>
>
>
> **Here's the kvm log**
>
>
>
> [2016-12-14 11:08:29.210641] I [MSGID: 104045] [glfs-master.c:91:notify]
> 0-gfapi: New graph 68767331-2e73-7276-2e62-62772e62652d (0) coming up
> [2016-12-14 11:08:29.210689] I [MSGID: 114020] [client.c:2356:notify] 0-vm-
> client-0: parent translators are ready, attempting connect on transport
> [2016-12-14 11:08:29.211020] I [MSGID: 114020] [client.c:2356:notify] 0-vm-
> client-1: parent translators are ready, attempting connect on transport
> [2016-12-14 11:08:29.211272] I [rpc-clnt.c:1947:rpc_clnt_reconfig] 0-vm-
> client-0: changing port to 49153 (from 0)
> [2016-12-14 11:08:29.211285] I [MSGID: 114020] [client.c:2356:notify] 0-vm-
> client-2: parent translators are ready, attempting connect on transport
> [2016-12-14 11:08:29.211838] I [MSGID: 114057] [client-
> handshake.c:1446:select_server_supported_programs] 0-vm-client-0: Using
> Program GlusterFS 3.3, Num (1298437), Version (330)
> [2016-12-14 11:08:29.211910] I [rpc-clnt.c:1947:rpc_clnt_reconfig] 0-vm-
> client-1: changing port to 49153 (from 0)
> [2016-12-14 11:08:29.211980] I [rpc-clnt.c:1947:rpc_clnt_reconfig] 0-vm-
> client-2: changing port to 49153 (from 0)
> [2016-12-14 11:08:29.212227] I [MSGID: 114046] [client-
> handshake.c:1222:client_setvolume_cbk] 0-vm-client-0: Connected to vm-
> client-0, attached to remote volume '/gluster/vm/brick'.
> [2016-12-14 11:08:29.212239] I [MSGID: 114047] [client-
> handshake.c:1233:client_setvolume_cbk] 0-vm-client-0: Server and Client lk-
> version numbers are not same, reopening the fds
> [2016-12-14 11:08:29.212296] I [MSGID: 108005] [afr-common.c:4298:afr_notify]
> 0-vm-replicate-0: Subvolume 'vm-client-0' came back up; going online.
> [2016-12-14 11:08:29.212316] I [MSGID: 114035] [client-
> handshake.c:201:client_set_lk_version_cbk] 0-vm-client-0: Server lk version =
> 1
> [2016-12-14 11:08:29.212426] I [MSGID: 114057] [client-
> handshake.c:1446:select_server_supported_programs] 0-vm-client-1: Using
> Program GlusterFS 3.3, Num (1298437), Version (330)
> [2016-12-14 11:08:29.212590] I [MSGID: 114057] [client-
> handshake.c:1446:select_server_supported_programs] 0-vm-client-2: Using
> Program GlusterFS 3.3, Num (1298437), Version (330)
> [2016-12-14 11:08:29.212874] I [MSGID: 114046] [client-
> handshake.c:1222:client_setvolume_cbk] 0-vm-client-2: Connected to vm-
> client-2, attached to remote volume '/gluster/vm/brick'.
> [2016-12-14 11:08:29.212886] I [MSGID: 114047] [client-
> handshake.c:1233:client_setvolume_cbk] 0-vm-client-2: Server and Client lk-
> version numbers are not same, reopening the fds
> [2016-12-14 11:08:29.212983] I [MSGID: 114046] [client-
> handshake.c:1222:client_setvolume_cbk] 0-vm-client-1: Connected to vm-
> client-1, attached to remote volume '/gluster/vm/brick'.
> [2016-12-14 11:08:29.212992] I [MSGID: 114047] [client-
> handshake.c:1233:client_setvolume_cbk] 0-vm-client-1: Server and Client lk-
> version numbers are not same, reopening the fds
> [2016-12-14 11:08:29.213042] I [MSGID: 114035] [client-
> handshake.c:201:client_set_lk_version_cbk] 0-vm-client-2: Server lk version =
> 1
> [2016-12-14 11:08:29.227393] I [MSGID: 114035] [client-
> handshake.c:201:client_set_lk_version_cbk] 0-vm-client-1: Server lk version =
> 1
> [2016-12-14 11:08:29.228239] I [MSGID: 108031] [afr-
> common.c:2068:afr_local_discovery_cbk] 0-vm-replicate-0: selecting local
> read_child vm-client-0
> [2016-12-14 11:08:29.228832] I [MSGID: 104041] [glfs-
> resolve.c:885:__glfs_active_subvol] 0-vm: switched to graph
> 68767331-2e73-7276-2e62-62772e62652d (0)
> [2016-12-14 11:08:29.232505] W [MSGID: 114031] [client-rpc-
> fops.c:2210:client3_3_seek_cbk] 0-vm-client-0: remote operation failed [No
> such device or address]
> kvm: block/gluster.c:1182: find_allocation: Assertion `offs >= start' failed.
> **<\- This is when it crash**
> Aborted
>
>
>
>
> On Dec 14 2016, at 12:55 pm, Lindsay Mathieson <lindsay.mathieson at gmail.com>
> wrote:
>
>> On 14/12/2016 8:51 PM, contact at makz.me wrote:
>> Today i've upgraded to 4.4, snapshot now works well but since i've updated,
>> all my windows VM are broken, when windows try to write on disk the vm crash
>> without error, even when i try to reinstall a windows, when i partition or
>> format the vm instantly crash.
>>
>>
>> Hmmm - I upgraded to PVE 4.4 today (rolling upgrade, 3 nodes) and am
> using Gluster 3.8.4, all my windows VM's are running fine + snapshots
> ok. Sounds like you a gluster volume in trouble.
>
>>
> What virtual disk interface are you using with the windows VM'? ide?
> virtio? scsi+virtio controller?
>
>> can you post:
>> gluster volume info
>>
> gluster volume status
>
>> gluster volume heal <vol name> heal info
>>
> \--
> Lindsay Mathieson
>
>> _______________________________________________
> pve-user mailing list
> pve-user at pve.proxmox.com
> <http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user>
>
> _______________________________________________
> pve-user mailing list
> pve-user at pve.proxmox.com
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
--
Lindsay Mathieson
More information about the pve-user
mailing list