[pve-devel] [RFC qemu-server++ 0/22] remote migration
aderumier at odiso.com
Fri Apr 16 09:36:15 CEST 2021
>>looking forward to feedback :) you'll need to put the
>>proxmox-websocket-tunnel binary into $PATH of pveproxy/qm, after
>>building it with 'cargo build'.
oh, I didn't known that the rust version was already ready. great :)
I'll try to do my first rust build :) (I really need to learn it, seem to be a great language)
>>if your inter-DC link is fast enough, you'll likely be hit by the
>>pveproxy bottleneck. it would still be interesting to get some
>>real-world numbers, I haven't tested with baremetal and fast storage
yes, I have enough bandwidth (25gbs), so no problem here. I'll try with && without any storage to compare.
(as I'm using ceph with nvme, I think it'll be the bottleneck with qemu mirror)
>>please be aware that this is very much experimental code still!
yes, sure , no problem ;)
On 15/04/2021 16:32, Fabian Grünbichler wrote:
> On April 15, 2021 4:04 pm, alexandre derumier wrote:
>> thanks for working on this !
>> I'll be able to test it soon as I'll need to migrate 200-300 vms between
>> 2 datacenter soon.
> looking forward to feedback :) you'll need to put the
> proxmox-websocket-tunnel binary into $PATH of pveproxy/qm, after
> building it with 'cargo build'.
> if your inter-DC link is fast enough, you'll likely be hit by the
> pveproxy bottleneck. it would still be interesting to get some
> real-world numbers, I haven't tested with baremetal and fast storage
> please be aware that this is very much experimental code still!
>> I think it could be great to add optionnal "tag" option to targetbridge,
>> as it could be different on target cluster.
> hmm, we could have another (optional) map for VLAN tags? since tags and
> bridges are not one entity (you can have on interface on bridge A with
> tag X, and another interface on bridge A with tag Y, and those need to
> be mapped to bridge B with tag P and bridge B with tag Q, for example).
>> Also, we should transfert vm firewall config.
> yes, that's definitely true. another source of potential
> mismatches/things to check before migrating (security groups/aliases!)
>> On 13/04/2021 14:16, Fabian Grünbichler wrote:
>>> this series adds remote migration for VMs. there's still plenty of
>>> TODOs/FIXMEs/stuff that requires discussion, hence the RFC. live
>>> migration with NBD and storage-migrated disks should work already.
>>> the performance bottle neck (~190MB/s on loopback) for the websocket
>>> connection seems to be in pveproxy at the moment - the rust code should
>>> manage about 700MB/s.
>>> overview over affected repos and changes, see individual patches for
>>> more details.
>>> some compatible changes to make websocket code usable for client-side
>>> connections, required by proxmox-websocket-tunnel
>>> new tunnel helper tool for forwarding commands and data over websocket
>>> connections, required by qemu-server on source side
>>> TODO: better error handling
>>> TODO: fingerprint checking/valid certs/..
>>> TODO: WS key generation
>>> TODO: decide on mask?
>>> TODO: investigate performance bottlenecks once PVE api server gets
>>> new ticket type, required by qemu-server on target side
>>> new remote.cfg and related helpers, required by qemu-server on source
>>> TODO: ACLs, CLI, API for managing config
>>> TODO: handling of discovered nodes with valid certificates
>>> TODO: add additional information like default bwlimits, storage/bridge
>>> bridgepair format akin to storage pair, pve-bridge-id option, required
>>> by qemu-server
>>> TODO: adapt pve-container
>>> handle remote migration (no SSH) in AbstractMigrate,
>>> required by qemu-server
>>> new 'addr' endpoint for retrieving remote node IPs, required on target
>>> extend 'pvesm import' to allow import from UNIX socket, required on
>>> target node by qemu-server
>>> some refactoring, new mtunnel endpoints, new remote_migration endpoints
>>> TODO: check remote ACLs
>>> TODO: handle pending changes and snapshots
>>> TODO: CLI for remote migration
>>> potential TODO: expose remote info via additional endpoints (resources? vmids?
>>> permissions? ...)
>>> as usual, some of the patches are best viewed with '-w', especially in
>>> pve-devel mailing list
>>> pve-devel at lists.proxmox.com
More information about the pve-devel