Hi,<br><br>We are building a 2 nodes cluster (proxmoxdev1 and proxmoxdev2) with<br>- LVMed iSCSI as shared storage<br>- Dell BMC IPMI card as fencing devices<br>- An iSCSI quorum disk<br><br>Each server has 2 NIC, one for Storage Network (iSCSI), one for user access and cluster communication (will be separated with a third NIC in the furture)<br>
<br>Software versions used :<br>pve-manager: 2.1-1 (pve-manager/2.1/f9b0f63a)<br>running kernel: 2.6.32-12-pve<br>proxmox-ve-2.6.32: 2.1-68<br>pve-kernel-2.6.32-10-pve: 2.6.32-63<br>pve-kernel-2.6.32-12-pve: 2.6.32-68<br>
lvm2: 2.02.95-1pve2<br>clvm: 2.02.95-1pve2<br>corosync-pve: 1.4.3-1<br>openais-pve: 1.1.4-2<br>libqb: 0.10.1-2<br>redhat-cluster-pve: 3.1.8-3<br>resource-agents-pve: 3.9.2-3<br>fence-agents-pve: 3.1.7-2<br>pve-cluster: 1.0-26<br>
qemu-server: 2.0-39<br>pve-firmware: 1.0-16<br>libpve-common-perl: 1.0-27<br>libpve-access-control: 1.0-21<br>libpve-storage-perl: 2.0-18<br>vncterm: 1.0-2<br>vzctl: 3.0.30-2pve5<br>vzprocps: 2.0.11-2<br>vzquota: 3.0.12-3<br>
pve-qemu-kvm: 1.0-9<br>ksm-control-daemon: 1.1-1<br><br>All nodes quorates, live migration works...Now let's run this scenario :<br>- Unplug the user access NIC on proxmoxdev2<br>- Heuristic checks fails, proxmoxdev2 is fenced, ressources restarts on proxmoxdev1<br>
- proxmoxdev2 restarts and does NOT quorate. This is normal, NIC is still unpluged.<br>- Replug the NIC, and check logs (the details lines have been removed):<br><br>Jul 12 14:02:57 proxmoxdev2 kernel: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready<br>
Jul 12 14:03:08 proxmoxdev2 corosync[1589]: [CLM ] CLM CONFIGURATION CHANGE<br>Jul 12 14:03:08 proxmoxdev2 corosync[1589]: [TOTEM ] A processor joined or left the membership and a new membership was formed.<br>Jul 12 14:03:28 proxmoxdev2 pmxcfs[1473]: [status] notice: node has quorum<br>
Jul 12 14:03:28 proxmoxdev2 corosync[1589]: [MAIN ] Completed service synchronization, ready to provide service.<br>Jul 12 14:03:28 proxmoxdev2 pmxcfs[1473]: [dcdb] notice: all data is up to date<br>Jul 12 14:03:29 proxmoxdev2 rgmanager[1997]: Quorum formed<br>
Jul 12 14:03:29 proxmoxdev2 kernel: dlm: no local IP address has been set<br>Jul 12 14:03:29 proxmoxdev2 kernel: dlm: cannot start dlm lowcomms -107<br>Jul 12 14:03:31 proxmoxdev2 corosync[1589]: [QUORUM] Members[2]: 1 2<br>
<br>"kernel: dlm" error lines seems to refer to a known bug already fixed by redhat (rhbz#688154 and rhbz#679274)<br>Apparently, it is a bad timer check in qdiskd wich breaks votes for quorum...<br>
<br>
Here's a diff from redhat : <a href="https://www.redhat.com/archives/cluster-devel/2011-March/msg00074.html">https://www.redhat.com/archives/cluster-devel/2011-March/msg00074.html</a><br>Other link : <a href="http://comments.gmane.org/gmane.linux.redhat.cluster/19598">http://comments.gmane.org/gmane.linux.redhat.cluster/19598</a><br>
<br>No services (pvevm) are shown and rgmanager is not running on proxmoxdev2. Running clustat returns :<br><br>Member Status: Quorate<br>Member Name ID Status<br>--------------------------------------------<br>proxmoxdev1 1 Online<br>
proxmoxdev2 2 Online, Local<br>/dev/block/8:17 0 Online, Quorum Disk<br><br><br>Running clustat on proxmoxdev1 returns:<br><br>Member Status: Quorate<br>
Member Name ID Status<br>
--------------------------------------------<br>proxmoxdev1 1 Online, Local, rgmanager<br>proxmoxdev2 2 Online<br>/dev/block/8:17 0 Online, Quorum Disk<br><br>Service Name Owner (Last) State <br>
----------------------------------------------------------<br>pvevm:100 proxmoxdev1 started<br><br>The only way to retreive au fully functional 2-nodes cluster is to restart manualy proxmoxdev2 AFTER having replug the NIC<br>
<br><span id="result_box" class="short_text" lang="en"><span class="hps">is it really the same bug as the redhat one and is </span><span class="hps"></span>there a workaround<span class="hps"> in</span> <span class="hps">Proxmox</span></span> ?<br>
Thanks<br><br>