[pve-devel] [RFC qemu-server++ 0/22] remote migration

[Fabian Grünbichler]

On April 15, 2021 4:04 pm, alexandre derumier wrote:
> Hi,
> 
> thanks for working on this !
> 
> I'll be able to test it soon as I'll need to migrate 200-300 vms between 
> 2 datacenter soon.

looking forward to feedback :) you'll need to put the 
proxmox-websocket-tunnel binary into $PATH of pveproxy/qm, after 
building it with 'cargo build'.

if your inter-DC link is fast enough, you'll likely be hit by the 
pveproxy bottleneck. it would still be interesting to get some 
real-world numbers, I haven't tested with baremetal and fast storage 
yet.

please be aware that this is very much experimental code still!

> I think it could be great to add optionnal "tag" option to targetbridge, 
> as it could be different on target cluster.

hmm, we could have another (optional) map for VLAN tags? since tags and 
bridges are not one entity (you can have on interface on bridge A with 
tag X, and another interface on bridge A with tag Y, and those need to 
be mapped to bridge B with tag P and bridge B with tag Q, for example).

> Also, we should transfert vm firewall config.

yes, that's definitely true. another source of potential 
mismatches/things to check before migrating (security groups/aliases!)

> 
> On 13/04/2021 14:16, Fabian Grünbichler wrote:
>> this series adds remote migration for VMs. there's still plenty of
>> TODOs/FIXMEs/stuff that requires discussion, hence the RFC. live
>> migration with NBD and storage-migrated disks should work already.
>>
>> the performance bottle neck (~190MB/s on loopback) for the websocket
>> connection seems to be in pveproxy at the moment - the rust code should
>> manage about 700MB/s.
>>
>> overview over affected repos and changes, see individual patches for
>> more details.
>>
>> proxmox:
>>
>> some compatible changes to make websocket code usable for client-side
>> connections, required by proxmox-websocket-tunnel
>>
>> proxmox-websocket-tunnel:
>>
>> new tunnel helper tool for forwarding commands and data over websocket
>> connections, required by qemu-server on source side
>> TODO: better error handling
>> TODO: fingerprint checking/valid certs/..
>> TODO: WS key generation
>> TODO: decide on mask?
>> TODO: investigate performance bottlenecks once PVE api server gets
>> faster
>>
>> pve-access-control:
>>
>> new ticket type, required by qemu-server on target side
>>
>> pve-cluster:
>>
>> new remote.cfg and related helpers, required by qemu-server on source
>> side
>> TODO: ACLs, CLI, API for managing config
>> TODO: handling of discovered nodes with valid certificates
>> TODO: add additional information like default bwlimits, storage/bridge
>> mappings
>>
>> pve-common:
>>
>> bridgepair format akin to storage pair, pve-bridge-id option, required
>> by qemu-server
>> TODO: adapt pve-container
>>
>> pve-guest-common:
>>
>> handle remote migration (no SSH) in AbstractMigrate,
>> required by qemu-server
>>
>> pve-manager:
>>
>> new 'addr' endpoint for retrieving remote node IPs, required on target
>> node
>>
>> pve-storage:
>>
>> extend 'pvesm import' to allow import from UNIX socket, required on
>> target node by qemu-server
>>
>> qemu-server:
>>
>> some refactoring, new mtunnel endpoints, new remote_migration endpoints
>> TODO: check remote ACLs
>> TODO: handle pending changes and snapshots
>> TODO: CLI for remote migration
>> potential TODO: expose remote info via additional endpoints (resources? vmids?
>> permissions? ...)
>>
>> as usual, some of the patches are best viewed with '-w', especially in
>> qemu-server..
>>
>>
>> _______________________________________________
>> pve-devel mailing list
>> pve-devel at lists.proxmox.com
>> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>>
> 
>