[pve-devel] got stuck while setup new dev custer

[Stefan Priebe]

Hi,

i wanted to setup a new proxmox dev cluster of 3 nodes. I already had a 
single pve machine i want to extend.

So i used that one as a base.

# pvecm create pve-dev

Restarting pve cluster filesystem: pve-cluster[dcdb] notice: wrote new 
cluster config '/etc/cluster/cluster.conf'
.
Starting cluster:
    Checking if cluster has been disabled at boot... [  OK  ]
    Checking Network Manager... [  OK  ]
    Global setup... [  OK  ]
    Loading kernel modules... [  OK  ]
    Mounting configfs... [  OK  ]
    Starting cman... [  OK  ]
    Waiting for quorum... [  OK  ]
    Starting fenced... [  OK  ]
    Starting dlm_controld... [  OK  ]
    Tuning DLM kernel config... [  OK  ]
    Unfencing self... [  OK  ]

# pvecm status; pvecm nodes
Version: 6.2.0
Config Version: 1
Cluster Name:  pve-dev
Cluster Id: 51583
Cluster Member: Yes
Cluster Generation: 236
Membership state: Cluster-Member
Nodes: 1
Expected votes: 1
Total votes: 1
Node votes: 1
Quorum: 1
Active subsystems: 5
Flags:
Ports Bound: 0
Node name: node1
Node ID: 1
Multicast addresses: 239.192.201.73
Node addresses: 10.255.0.10
Node  Sts   Inc   Joined               Name
    1   M    236   2015-03-23 19:48:20  node1

I then tried to add the 2nd node which just hangs:

# pvecm add 10.255.0.10
copy corosync auth key
stopping pve-cluster service
Stopping pve cluster filesystem: pve-cluster.
backup old database
Starting pve cluster filesystem : pve-cluster.
Starting cluster:
    Checking if cluster has been disabled at boot... [  OK  ]
    Checking Network Manager... [  OK  ]
    Global setup... [  OK  ]
    Loading kernel modules... [  OK  ]
    Mounting configfs... [  OK  ]
    Starting cman... [  OK  ]
    Waiting for quorum... [  OK  ]
    Starting fenced... [  OK  ]
    Starting dlm_controld... [  OK  ]
    Tuning DLM kernel config... [  OK  ]
    Unfencing self... [  OK  ]
waiting for quorum...

That one hangs at quorum.

And the first one shows in log:
Mar 23 19:56:41 node1 pmxcfs[7740]: [status] notice: cpg_send_message 
retried 100 times
Mar 23 19:56:41 node1 pmxcfs[7740]: [status] crit: cpg_send_message 
failed: 6
Mar 23 19:56:42 node1 pmxcfs[7740]: [status] notice: cpg_send_message 
retry 10
Mar 23 19:56:43 node1 pmxcfs[7740]: [status] notice: cpg_send_message 
retry 20
...

I already checked omping which is fine.

Whats wrong ;-(

Greets,
Stefan