[pve-devel] [PATCH 4/6] add targetstorage to vm_start
Alexandre DERUMIER
aderumier at odiso.com
Tue Oct 25 14:08:09 CEST 2016
>>I also noticed that if it fails to connect to the address passed below
>>(ie I first just removed the above line without inserting it below) it
>>wrongly assumes the disk got copying was finished instantly - something
>>to look into, might be an issue with catching an error in
>>qemu_drive_mirror_monitor()'s job handling, or in qemu_drive_mirror
>>already?
I have tried with a blank localip=""
and indeed, in migrate code, This skip this part
elsif ($line =~ m/^storage migration listens on nbd:(localhost|[\d\.]+|\[[\d\.:a-fA-F]+\]):(\d+):exportname=(\S+) volume:(\S+)$/) {
my $volid = $4;
my $nbd_uri = "nbd:$1:$2:exportname=$3";
my $targetdrive = $3;
$targetdrive =~ s/drive-//g;
$self->{target_drive}->{$targetdrive}->{volid} = $volid;
$self->{target_drive}->{$targetdrive}->{nbd_uri} = $nbd_uri;
But I need to add check to forbid vm migration if local disks are present in this case.
If the localip is a wrong ip, the start mirror is hanging (but the qmp socket too).
I'll try to add a tcp check on ip:port to be sure before starting.
----- Mail original -----
De: "Wolfgang Bumiller" <w.bumiller at proxmox.com>
À: "aderumier" <aderumier at odiso.com>
Cc: "pve-devel" <pve-devel at pve.proxmox.com>
Envoyé: Mardi 25 Octobre 2016 13:22:20
Objet: Re: [pve-devel] [PATCH 4/6] add targetstorage to vm_start
On Tue, Oct 25, 2016 at 09:53:02AM +0200, Alexandre Derumier wrote:
> This will create a new drive for each local drive found,
> and start the vm with this new drives.
>
> if targetstorage == 1, we use same sid than original vm disk
>
> a nbd server is started in qemu and expose local volumes to network port
>
> Signed-off-by: Alexandre Derumier <aderumier at odiso.com>
> ---
> PVE/API2/Qemu.pm | 14 ++++++++++-
> PVE/QemuServer.pm | 69 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-
> 2 files changed, 81 insertions(+), 2 deletions(-)
>
> diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
> index 013203a..da533df 100644
> --- a/PVE/API2/Qemu.pm
> +++ b/PVE/API2/Qemu.pm
> @@ -1627,6 +1627,11 @@ __PACKAGE__->register_method({
> stateuri => get_standard_option('pve-qm-stateuri'),
> migratedfrom => get_standard_option('pve-node',{ optional => 1 }),
> machine => get_standard_option('pve-qm-machine'),
> + targetstorage => {
> + description => "Target migration storage . (1 = same storeid than original)",
> + type => 'string',
> + optional => 1
> + }
> },
> },
> returns => {
> @@ -1657,6 +1662,13 @@ __PACKAGE__->register_method({
> raise_param_exc({ migratedfrom => "Only root may use this option." })
> if $migratedfrom && $authuser ne 'root at pam';
>
> + my $targetstorage = extract_param($param, 'targetstorage');
> + raise_param_exc({ targetstorage => "Only root may use this option." })
> + if $targetstorage && $authuser ne 'root at pam';
> +
> + raise_param_exc({ targetstorage => "targetstorage can only by used with migratedfrom." })
> + if $targetstorage && !$migratedfrom;
> +
> # read spice ticket from STDIN
> my $spice_ticket;
> if ($stateuri && ($stateuri eq 'tcp') && $migratedfrom && ($rpcenv->{type} eq 'cli')) {
> @@ -1697,7 +1709,7 @@ __PACKAGE__->register_method({
> syslog('info', "start VM $vmid: $upid\n");
>
> PVE::QemuServer::vm_start($storecfg, $vmid, $stateuri, $skiplock, $migratedfrom, undef,
> - $machine, $spice_ticket);
> + $machine, $spice_ticket, $targetstorage);
>
> return;
> };
> diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
> index d0310b6..2489c42 100644
> --- a/PVE/QemuServer.pm
> +++ b/PVE/QemuServer.pm
> @@ -4429,7 +4429,7 @@ sub vmconfig_update_disk {
>
> sub vm_start {
> my ($storecfg, $vmid, $statefile, $skiplock, $migratedfrom, $paused,
> - $forcemachine, $spice_ticket) = @_;
> + $forcemachine, $spice_ticket, $targetstorage) = @_;
>
> PVE::QemuConfig->lock_config($vmid, sub {
> my $conf = PVE::QemuConfig->load_config($vmid, $migratedfrom);
> @@ -4450,6 +4450,54 @@ sub vm_start {
> # set environment variable useful inside network script
> $ENV{PVE_MIGRATED_FROM} = $migratedfrom if $migratedfrom;
>
> + my $local_volumes = {};
> +
> + if( $targetstorage ){
> +
> + foreach_drive($conf, sub {
> + my ($ds, $drive) = @_;
> +
> + return if drive_is_cdrom($drive);
> +
> + my $volid = $drive->{file};
> +
> + return if !$volid;
> +
> + my ($storeid, $volname) = PVE::Storage::parse_volume_id($volid);
> +
> + my $scfg = PVE::Storage::storage_config($storecfg, $storeid);
> + return if $scfg->{shared};
> + $local_volumes->{$ds} = [$volid, $storeid, $volname];
> + });
> +
> + my $format = undef;
> +
> + foreach my $opt (sort keys %$local_volumes) {
> +
> + my ($volid, $storeid, $volname) = @{$local_volumes->{$opt}};
> + my $drive = parse_drive($opt, $conf->{$opt});
> +
> + #if remote storage is specified, use default format
> + if ($targetstorage && $targetstorage != 1) {
> + $storeid = $targetstorage;
> + my ($defFormat, $validFormats) = PVE::Storage::storage_default_format($storecfg, $storeid);
> + $format = $defFormat;
> + } else {
> + #else we use same format than original
> + $format = $drive->{format};
> + }
> +
> + my $newvolid = PVE::Storage::vdisk_alloc($storecfg, $storeid, $vmid, $format, undef, ($drive->{size}/1024));
> + my $newdrive = $drive;
> + $newdrive->{format} = $format;
> + $newdrive->{file} = $newvolid;
> + my $drivestr = PVE::QemuServer::print_drive($vmid, $newdrive);
> + $local_volumes->{$opt} = $drivestr;
> + #pass drive to conf for command line
> + $conf->{$opt} = $drivestr;
> + }
> + }
> +
> my ($cmd, $vollist, $spice_port) = config_to_command($storecfg, $vmid, $conf, $defaults, $forcemachine);
>
> my $migrate_port = 0;
> @@ -4571,6 +4619,25 @@ sub vm_start {
> warn $@ if $@;
> }
>
> + #start nbd server for storage migration
> + if ($targetstorage) {
> +
> + my $nodename = PVE::INotify::nodename();
> + my $localip = PVE::Cluster::remote_node_ip($nodename, 1);
> + $localip = "[$localip]" if Net::IP::ip_is_ipv6($localip);
the nbd-server-start command below has separate parameters for IP and
port and thus does not accept brackets around the IP.
This must be moved down to where $migrate_storage_uri is created.
I also noticed that if it fails to connect to the address passed below
(ie I first just removed the above line without inserting it below) it
wrongly assumes the disk got copying was finished instantly - something
to look into, might be an issue with catching an error in
qemu_drive_mirror_monitor()'s job handling, or in qemu_drive_mirror
already?
> + my $pfamily = PVE::Tools::get_host_address_family($nodename);
> + $migrate_port = PVE::Tools::next_migrate_port($pfamily);
> +
> + vm_mon_cmd_nocheck($vmid, "nbd-server-start", addr => { type => 'inet', data => { host => "${localip}", port => "${migrate_port}" } } );
> +
> + foreach my $opt (sort keys %$local_volumes) {
> + my $volid = $local_volumes->{$opt};
> + vm_mon_cmd_nocheck($vmid, "nbd-server-add", device => "drive-$opt", writable => JSON::true );
Here we need it to separate the :port part.
+ $localip = "[$localip]" if Net::IP::ip_is_ipv6($localip);
> + my $migrate_storage_uri = "nbd:${localip}:${migrate_port}:exportname=drive-$opt";
> + print "storage migration listens on $migrate_storage_uri volume:$volid\n";
> + }
> + }
> +
> if ($migratedfrom) {
>
> eval {
> --
> 2.1.4
>
> _______________________________________________
> pve-devel mailing list
> pve-devel at pve.proxmox.com
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
More information about the pve-devel
mailing list