<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
</head>
<body dir="auto">
<div>Just an FYI to anyone who might be thinking about trying this...these commands stopped every single running vm we had...I assume because of the unloading of the KVM kernel modules.</div>
<div><br>
</div>
<div>Use With Caution.</div>
<div><br>
</div>
<div>Shain</div>
<div><br>
</div>
<div>ps...we are still having problems with the cluster.</div>
<div><br>
Sent from my iPhone</div>
<div><br>
On Mar 13, 2015, at 5:57 AM, Juan Serra Costa <<a href="mailto:juaserco@gmail.com">juaserco@gmail.com</a>> wrote:<br>
<br>
</div>
<blockquote type="cite">
<div>
<div dir="ltr">
<div>
<div>This is not a solution, but it may help. After upgrading to 3.4 I had problems with the cluster. I solved it by restarting some daemons.<br>
<br>
/etc/init.d/pve-manager stop<br>
/etc/init.d/pvestatd stop<br>
/etc/init.d/pvedaemon stop<br>
/etc/init.d/cman stop<br>
/etc/init.d/pve-cluster stop<br>
<br>
</div>
At this point I had a problem with cman because some processes didn't stop, so I killed them.<br>
<br>
ps aux|egrep "(dlm_controld|fenced|corosync)"<br>
<br>
</div>
Just to be sure, I foreced unmounting some folders.<br>
<br>
umount -l /sys/kernel/config<br>
umount -l /etc/pve<br>
<br>
<div>After that I started the daemons.<br>
<br>
/etc/init.d/pve-cluster start<br>
/etc/init.d/cman start<br>
/etc/init.d/pvedaemon start<br>
/etc/init.d/pvestatd start<br>
/etc/init.d/pve-manager start<br>
<br>
</div>
<div>Doing this with all the machines one by one I recovered the cluster without rebooting. I know that this is not a solution, but it may help.<br>
</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">2015-03-12 19:47 GMT+01:00 Shain Miley <span dir="ltr"><<a href="mailto:smiley@npr.org" target="_blank">smiley@npr.org</a>></span>:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Thank you for the info.<br>
<br>
No we are not using ipv6 at the moment.<br>
<br>
>From what I can tell using ssmping and omping...server to server works when using '<a href="http://all-systems.mcast.net" target="_blank">all-systems.mcast.net</a>' as the multicast host...however when I use '239.192.129.102' the address assigned by proxmox
(I guess)...there is 100% multicast dropped packets.<br>
<br>
We see the 100% dropped packets when using two 2.6.32-34 noes to test....but not when using two 2.6.32-37 nodes to test.<br>
<br>
The problem is even with 2.6.32-37 when we did have a quorum (11 of 19 nodes joined and restarting pve-cluster)../etc/pve/ was still read-only and our web gui showed red...so just going to 2.6.32-37 does not seem like the answer either.<br>
<br>
Thanks again,<br>
<br>
Shain
<div class="HOEnZb">
<div class="h5"><br>
<br>
<br>
On 03/12/2015 01:57 PM, Holger Hampel | RA Consulting wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Are the server resolved to IPv6? 2.6.32-37-pve don't like it!<br>
<br>
-----Ursprüngliche Nachricht-----<br>
Von: pve-user [mailto:<a href="mailto:pve-user-bounces@pve.proxmox.com" target="_blank">pve-user-bounces@pve.<u></u>proxmox.com</a>] Im Auftrag von Shain Miley<br>
Gesendet: Donnerstag, 12. März 2015 16:17<br>
An: Dietmar Maurer; <a href="mailto:pve-user@pve.proxmox.com" target="_blank">pve-user@pve.proxmox.com</a>; Eneko Lacunza<br>
Betreff: Re: [PVE-User] TASK ERROR: cluster not ready - no quorum?<br>
<br>
These two look similar to what we are seeing as well:<br>
<br>
<a href="http://forum.proxmox.com/threads/21060-Adding-new-server-to-older-existing-cluster-fails" target="_blank">http://forum.proxmox.com/<u></u>threads/21060-Adding-new-<u></u>server-to-older-existing-<u></u>cluster-fails</a><br>
<br>
<a href="http://forum.proxmox.com/threads/20523-Quorum-problems-with-PVE-2-3-and-3-3" target="_blank">http://forum.proxmox.com/<u></u>threads/20523-Quorum-problems-<u></u>with-PVE-2-3-and-3-3</a><br>
<br>
<br>
At this point it looks like the nodes on 2.6.32-37-pve can use multicast<br>
(testing via ssmping) and the ones on 2.6.32-34-pve fail.<br>
<br>
I cannot understand why there would be a problem here, are others<br>
running mixed kernel (2.6.32) kernel versions with 3.4?<br>
<br>
Or should we try to get to a state where everything is using the same<br>
kernel version?<br>
<br>
Thanks,<br>
<br>
Shain<br>
<br>
<br>
<br>
On 03/12/2015 10:51 AM, Shain Miley wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi.<br>
No...we don't...and I know that the post references infiniband...but I<br>
thought that maybe there was a more widespread multicast issue in<br>
general with that kernel.<br>
<br>
We have been using Proxmox for close to 5 or 6 years and I have never<br>
lost a quorum, or had these kinds of issues before, so I am just<br>
trying figure out what went wrong and how to fix it.<br>
<br>
Thanks,<br>
Shain<br>
<br>
<br>
On 03/12/2015 10:46 AM, Dietmar Maurer wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I found this post here:<br>
<br>
<a href="http://pve.proxmox.com/pipermail/pve-devel/2015-February/014356.html" target="_blank">http://pve.proxmox.com/<u></u>pipermail/pve-devel/2015-<u></u>February/014356.html</a><br>
<br>
which suggests there might be an issue with the 2.6.32-37 kernel .<br>
</blockquote>
Do you use infiniband?<br>
<br>
</blockquote>
______________________________<u></u>_________________<br>
pve-user mailing list<br>
<a href="mailto:pve-user@pve.proxmox.com" target="_blank">pve-user@pve.proxmox.com</a><br>
<a href="http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user" target="_blank">http://pve.proxmox.com/cgi-<u></u>bin/mailman/listinfo/pve-user</a><br>
</blockquote>
______________________________<u></u>_________________<br>
pve-user mailing list<br>
<a href="mailto:pve-user@pve.proxmox.com" target="_blank">pve-user@pve.proxmox.com</a><br>
<a href="http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user" target="_blank">http://pve.proxmox.com/cgi-<u></u>bin/mailman/listinfo/pve-user</a><br>
______________________________<u></u>_________________<br>
pve-user mailing list<br>
<a href="mailto:pve-user@pve.proxmox.com" target="_blank">pve-user@pve.proxmox.com</a><br>
<a href="http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user" target="_blank">http://pve.proxmox.com/cgi-<u></u>bin/mailman/listinfo/pve-user</a><br>
</blockquote>
<br>
______________________________<u></u>_________________<br>
pve-user mailing list<br>
<a href="mailto:pve-user@pve.proxmox.com" target="_blank">pve-user@pve.proxmox.com</a><br>
<a href="http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user" target="_blank">http://pve.proxmox.com/cgi-<u></u>bin/mailman/listinfo/pve-user</a><br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<blockquote type="cite">
<div><span>_______________________________________________</span><br>
<span>pve-user mailing list</span><br>
<span><a href="mailto:pve-user@pve.proxmox.com">pve-user@pve.proxmox.com</a></span><br>
<span><a href="http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user">http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user</a></span><br>
</div>
</blockquote>
</body>
</html>