<div dir="ltr">When you migrate vm from one node to another, and it fail(any reason), Proxmox delete local copy of vm on local node.<div><br></div><div>Logic of migrate must be like this.</div><div>1 - Copy CT to another node</div>
<div>2 - sync CT</div><div>3 - suspend, dump</div><div>4 - copy on another node</div><div>5 - restore, unsuspend.</div><div><br></div><div>If one of this actions fail, it MUST be restored on current node.</div><div>Now we can see something like this:</div>
<div><br></div><div><span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:32:55 container is running - using online migration</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:32:56 starting rsync phase 1</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">-----------</span></div><div><span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:18 start live migration - suspending container</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:18 dump container state</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:18 copy dump file to target node</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:20 starting rsync (2nd pass)</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:20 dump 2nd level quota</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:20 copy 2nd level quota to target node</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:22 initialize container on remote node 'node2'</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:22 initializing remote quota</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:22 turn on remote quota</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:22 load 2nd level quota</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:22 starting container on remote node 'node2'</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:22 restore container state</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:23 Restoring container ...</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:23 Starting container ...</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:23 Container is mounted</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:23 undump...</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:23 Setting CPU units: 1000</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:23 Setting CPUs: 1</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:23 Setting devices</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:23 Error: undump failed: Cannot allocate memory</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:23 Restoring failed:</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:23 Error: do_rst_vma: sc_m(un)lock failed</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:23 Error: do_rst_mm: failed to restore vma: -12</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:23 Error: do_rst_mm 62421312</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:23 Error: rst_mm: -12</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:23 Error: make_baby: -12</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:23 Error: rst_clone_children</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:23 Container is unmounted</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:23 ERROR: online migrate failure - Failed to restore container: Container start failed</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:23 <b>removing container files on local node</b></span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:25 start final cleanup</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">Nov 01 13:34:26 ERROR: migration finished with problems (duration 00:01:31)</span><br style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">
<span style="color:rgb(0,0,0);font-family:tahoma,arial,verdana,sans-serif;font-size:11px;line-height:16px;white-space:pre">TASK ERROR: migration problems</span><br></div><div><br></div><div>It's a bug or a feature?</div>
</div>