<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">Hi everyone,<div><br></div><div>I have an issue with a intel modular server with 5 nodes. Whenever the nodes run a backup, some of them fail and often the VM’s become locked. These VM’s are all QEMU based virtual servers and not openvz as they are Windows servers.</div><div><br></div><div>Often everyday, I need to restart the cluster after something like this appears in the backup logs by going to one of the nodes that is red and restarting the PVECluster, which I believe is the correct way to do this, so please correct my if I’m wrong.</div><div><br></div><div>--------------------------------------------------------------------------------------------------------------------------------------------------------------------------</div><div><span style="font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);">INFO: starting new backup job: vzdump 103 101 --quiet 1 --mailto snip@snip --mode snapshot --compress lzo --storage nfsData --node node2</span><br style="box-sizing: border-box; font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);"><span style="font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);">INFO: Starting Backup of VM 101 (qemu)</span><br style="box-sizing: border-box; font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);"><span style="font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);">INFO: status = running</span><br style="box-sizing: border-box; font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);"><span style="font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);">INFO: unable to open file '/etc/pve/nodes/node2/qemu-server/101.conf.tmp.666400' - Software caused connection abort</span><br style="box-sizing: border-box; font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);"><span style="font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);">INFO: update VM 101: -lock backup</span><br style="box-sizing: border-box; font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);"><span style="font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);">ERROR: Backup of VM 101 failed - command 'qm set 101 --lock backup' failed: exit code 107</span><br style="box-sizing: border-box; font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);"><span style="font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);">INFO: Starting Backup of VM 103 (qemu)</span><br style="box-sizing: border-box; font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);"><span style="font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);">ERROR: Backup of VM 103 failed - unable to find configuration file for VM 103 - no such machine</span><br style="box-sizing: border-box; font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);"><span style="font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);">INFO: Backup job finished with errors</span><br style="box-sizing: border-box; font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);"><span style="font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);">TASK ERROR: job errors</span></div><div>--------------------------------------------------------------------------------------------------------------------------------------------------------------------------</div><div><div>--------------------------------------------------------------------------------------------------------------------------------------------------------------------------</div></div><div><span style="background-color: rgb(255, 255, 255); font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre;">INFO: starting new backup job: vzdump 106 --quiet 1 --mailto snip@snip --mode snapshot --compress lzo --storage nfsData --node node4</span></div><div><span style="font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);">INFO: Starting Backup of VM 106 (qemu)</span><br style="box-sizing: border-box; font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);"><span style="font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);">INFO: status = running</span><br style="box-sizing: border-box; font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);"><span style="font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);">INFO: unable to open file '/etc/pve/nodes/node4/qemu-server/106.conf.tmp.641046' - File exists</span><br style="box-sizing: border-box; font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);"><span style="font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);">INFO: update VM 106: -lock backup</span><br style="box-sizing: border-box; font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);"><span style="font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);">ERROR: Backup of VM 106 failed - command 'qm set 106 --lock backup' failed: exit code 17</span><br style="box-sizing: border-box; font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);"><span style="font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);">INFO: Backup job finished with errors</span><br style="box-sizing: border-box; font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);"><span style="font-family: tahoma, arial, verdana, sans-serif; font-size: 11px; line-height: 16px; white-space: pre; background-color: rgb(255, 255, 255);">TASK ERROR: job errors</span></div><div><div>--------------------------------------------------------------------------------------------------------------------------------------------------------------------------</div></div><div><br></div><div><br></div><div><span style="background-color: rgb(255, 255, 255);">The backups all go to an NFS share mounted on on a private network over gig ethernet. </span></div><div><span style="background-color: rgb(255, 255, 255);">Any ideas as to why this is happening? The backups are all staggered from the hosts to back up to the share<br></span><div apple-content-edited="true">
<div><div><br></div><div>Thanks for any insight.</div><div><br></div><div>_____________________________</div><div>David Thompson </div></div><div><br></div><br class="Apple-interchange-newline">
</div>
<br></div></body></html>