[PVE-User] Quorum Activity blocked

[Eneko Lacunza]

Hi Piviul,

This is usually due to network connectivity issues. Are you able to ping 
hosts on 192.168.255.* interfaces?

El 7/11/22 a las 8:44, Piviul escribió:
> Good morning sirs, in a 3 nodes proxmox 6.4 all the 3 nodes seems to 
> works, all vm guest continue to works but If I try to start a vm guest 
> the starting fails with the message: "cluster not ready - no quorum? 
> (500)". This is the cluster manager status:
>
> # pvecm status
> Cluster information
> -------------------
> Name:             CSA-cluster1
> Config Version:   3
> Transport:        knet
> Secure auth:      on
>
> Quorum information
> ------------------
> Date:             Mon Nov  7 08:37:20 2022
> Quorum provider:  corosync_votequorum
> Nodes:            1
> Node ID:          0x00000002
> Ring ID:          2.91e
> Quorate:          No
>
> Votequorum information
> ----------------------
> Expected votes:   3
> Highest expected: 3
> Total votes:      1
> Quorum:           2 Activity blocked
> Flags:
>
> Membership information
> ----------------------
>     Nodeid      Votes Name
> 0x00000002          1 192.168.255.2 (local)
>
>
> These are the first logs in syslog showing that some problem occurs:
>
> Nov  4 23:38:01 pve02 systemd[1]: Started Proxmox VE replication runner.
> Nov  4 23:38:26 pve02 corosync[1703]:   [KNET  ] link: host: 3 link: 0 
> is down
> Nov  4 23:38:26 pve02 corosync[1703]:   [KNET  ] host: host: 3 
> (passive) best link: 0 (pri: 1)
> Nov  4 23:38:26 pve02 corosync[1703]:   [KNET  ] host: host: 3 has no 
> active links
> Nov  4 23:38:28 pve02 corosync[1703]:   [TOTEM ] Token has not been 
> received in 2737 ms
> Nov  4 23:38:30 pve02 corosync[1703]:   [KNET  ] rx: host: 3 link: 0 
> is up
> Nov  4 23:38:30 pve02 corosync[1703]:   [KNET  ] host: host: 3 
> (passive) best link: 0 (pri: 1)
> Nov  4 23:38:32 pve02 corosync[1703]:   [QUORUM] Sync members[2]: 1 2
> Nov  4 23:38:32 pve02 corosync[1703]:   [QUORUM] Sync left[1]: 3
> Nov  4 23:38:32 pve02 corosync[1703]:   [TOTEM ] A new membership 
> (1.873) was formed. Members left: 3
> Nov  4 23:38:32 pve02 corosync[1703]:   [TOTEM ] Failed to receive the 
> leave message. failed: 3
> Nov  4 23:38:32 pve02 pmxcfs[1578]: [dcdb] notice: members: 1/1626, 
> 2/1578
> Nov  4 23:38:32 pve02 pmxcfs[1578]: [dcdb] notice: starting data 
> syncronisation
> Nov  4 23:38:32 pve02 pmxcfs[1578]: [status] notice: members: 1/1626, 
> 2/1578
> Nov  4 23:38:32 pve02 pmxcfs[1578]: [status] notice: starting data 
> syncronisation
> Nov  4 23:38:32 pve02 corosync[1703]:   [QUORUM] Members[2]: 1 2
> Nov  4 23:38:32 pve02 corosync[1703]:   [MAIN  ] Completed service 
> synchronization, ready to provide service.
> Nov  4 23:38:32 pve02 pmxcfs[1578]: [dcdb] notice: received sync 
> request (epoch 1/1626/00000009)
> Nov  4 23:38:32 pve02 pmxcfs[1578]: [status] notice: received sync 
> request (epoch 1/1626/00000009)
> Nov  4 23:38:32 pve02 pmxcfs[1578]: [dcdb] notice: received all states
> Nov  4 23:38:32 pve02 pmxcfs[1578]: [dcdb] notice: leader is 1/1626
> Nov  4 23:38:32 pve02 pmxcfs[1578]: [dcdb] notice: synced members: 
> 1/1626, 2/1578
> Nov  4 23:38:32 pve02 pmxcfs[1578]: [dcdb] notice: all data is up to date
> Nov  4 23:38:32 pve02 pmxcfs[1578]: [dcdb] notice: dfsm_deliver_queue: 
> queue length 2
> Nov  4 23:38:32 pve02 pmxcfs[1578]: [status] notice: received all states
> Nov  4 23:38:32 pve02 pmxcfs[1578]: [status] notice: all data is up to 
> date
> Nov  4 23:38:32 pve02 pmxcfs[1578]: [status] notice: 
> dfsm_deliver_queue: queue length 46
> Nov  4 23:38:34 pve02 corosync[1703]:   [KNET  ] link: host: 3 link: 0 
> is down
> Nov  4 23:38:34 pve02 corosync[1703]:   [KNET  ] host: host: 3 
> (passive) best link: 0 (pri: 1)
> Nov  4 23:38:34 pve02 corosync[1703]:   [KNET  ] host: host: 3 has no 
> active links
> Nov  4 23:38:41 pve02 corosync[1703]:   [KNET  ] link: host: 1 link: 0 
> is down
> Nov  4 23:38:41 pve02 corosync[1703]:   [KNET  ] host: host: 1 
> (passive) best link: 0 (pri: 1)
> Nov  4 23:38:41 pve02 corosync[1703]:   [KNET  ] host: host: 1 has no 
> active links
> Nov  4 23:38:42 pve02 corosync[1703]:   [TOTEM ] Token has not been 
> received in 2737 ms
> Nov  4 23:38:43 pve02 corosync[1703]:   [TOTEM ] A processor failed, 
> forming new configuration: token timed out (3650ms), waiting 4380ms 
> for consensus.
> Nov  4 23:38:48 pve02 corosync[1703]:   [QUORUM] Sync members[1]: 2
> Nov  4 23:38:48 pve02 corosync[1703]:   [QUORUM] Sync left[1]: 1
> Nov  4 23:38:48 pve02 corosync[1703]:   [TOTEM ] A new membership 
> (2.877) was formed. Members left: 1
> Nov  4 23:38:48 pve02 corosync[1703]:   [TOTEM ] Failed to receive the 
> leave message. failed: 1
> Nov  4 23:38:48 pve02 pmxcfs[1578]: [dcdb] notice: members: 2/1578
> Nov  4 23:38:48 pve02 pmxcfs[1578]: [status] notice: members: 2/1578
> Nov  4 23:38:48 pve02 corosync[1703]:   [QUORUM] This node is within 
> the non-primary component and will NOT provide any services.
> Nov  4 23:38:48 pve02 corosync[1703]:   [QUORUM] Members[1]: 2
> Nov  4 23:38:48 pve02 corosync[1703]:   [MAIN  ] Completed service 
> synchronization, ready to provide service.
> Nov  4 23:38:48 pve02 pmxcfs[1578]: [status] notice: node lost quorum
> Nov  4 23:38:48 pve02 pmxcfs[1578]: [dcdb] crit: received write while 
> not quorate - trigger resync
> Nov  4 23:38:48 pve02 pmxcfs[1578]: [dcdb] crit: leaving CPG group
> Nov  4 23:38:48 pve02 pve-ha-lrm[1943]: unable to write lrm status 
> file - unable to open file '/etc/pve/nodes/pve02/lrm_status.tmp.1943' 
> - Permission denied
> Nov  4 23:38:49 pve02 pmxcfs[1578]: [dcdb] notice: start cluster 
> connection
> Nov  4 23:38:49 pve02 pmxcfs[1578]: [dcdb] crit: cpg_join failed: 14
> Nov  4 23:38:49 pve02 pmxcfs[1578]: [dcdb] crit: can't initialize service
> Nov  4 23:38:55 pve02 pmxcfs[1578]: [dcdb] notice: members: 2/1578
> Nov  4 23:38:55 pve02 pmxcfs[1578]: [dcdb] notice: all data is up to date
> Nov  4 23:39:00 pve02 systemd[1]: Starting Proxmox VE replication 
> runner...
> Nov  4 23:39:01 pve02 pvesr[2146320]: trying to acquire cfs lock 
> 'file-replication_cfg' ...
> [...]
>
> What's happened to my cluster? Someone has some suggestions to 
> troubleshoot the problem?
>
> Piviul
>
>
> _______________________________________________
> pve-user mailing list
> pve-user at lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user

Eneko Lacunza
Zuzendari teknikoa | Director técnico
Binovo IT Human Project

Tel. +34 943 569 206 |https://www.binovo.es
Astigarragako Bidea, 2 - 2º izda. Oficina 10-11, 20180 Oiartzun

https://www.youtube.com/user/CANALBINOVO
https://www.linkedin.com/company/37269706/