[pve-devel] [RFC container] alloc disk: fix #3970 avoid ambiguous rbd image path

[Aaron Lauterer]

If two RBD storages use the same pool, but connect to different
clusters, we cannot say to which cluster the mapped RBD image belongs
to. To avoid potential data loss, we need to verify that no other
storage is configured that could have a volume mapped under the same
path before we format anything.

The ambiguous mapping is in
/dev/rbd/<pool>/<ns>/<image> where the namespace <ns> is optional.

Once we can tell the clusters apart in the mapping, we can remove these
checks again.

See bug #3969 for more information on the root cause.

Signed-off-by: Aaron Lauterer <a.lauterer at proxmox.com>
---
RFC because I would like someone else to take a look at the logic and I
wasn't sure how to format the grouping of the conditions in a way that
is easy to read

 src/PVE/LXC.pm | 45 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 45 insertions(+)

diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
index fe63087..b048ce0 100644
--- a/src/PVE/LXC.pm
+++ b/src/PVE/LXC.pm
@@ -1970,6 +1970,51 @@ sub alloc_disk {
     my $scfg = PVE::Storage::storage_config($storecfg, $storage);
     # fixme: use better naming ct-$vmid-disk-X.raw?
 
+    # check if another rbd storage with the same pool name but different
+    # cluster exists. If so, allocating a new volume can potentially be
+    # dangerous because the RBD mapping, exposes it in an ambiguous way under
+    # /dev/rbd/<pool>/<ns>/<image>. Without any information to which cluster it
+    # belongs, we cannot clearly determine which image we access and
+    # potentially format an already used image.  See
+    # https://bugzilla.proxmox.com/show_bug.cgi?id=3969 and
+    # https://bugzilla.proxmox.com/show_bug.cgi?id=3970
+    # TODO: remove these checks once #3969 is fixed and we can clearly tell to
+    # which cluster an image belongs to
+    if ($scfg->{type} eq 'rbd') {
+	my $pool = $storecfg->{ids}->{$storage}->{pool};
+	foreach my $stor  (keys %{$storecfg->{ids}}) {
+	    next if $stor eq $storage;
+
+	    my $ccfg = PVE::Storage::storage_config($storecfg, $stor);
+	    next if $ccfg->{type} ne 'rbd';
+
+	    if ($scfg->{pool} eq $ccfg->{pool}) {
+		if (
+		    (
+			defined($scfg->{monhost})
+			&& defined($ccfg->{monhost})
+			&& $scfg->{monhost} eq $ccfg->{monhost}
+		    )
+		    || (
+			!defined($scfg->{monhost})
+			&& !defined($ccfg->{monhost})
+		    )
+		) {
+		    # both external ones same or both hyperconverged
+		    next;
+		}
+		# different cluster here
+		# we are ok if namespaces are not the same or only one storage has one
+		if (defined($scfg->{namespace}) && defined($ccfg->{namespace})) {
+		    next if $scfg->{namespace} ne $ccfg->{namespace};
+		} elsif (defined($scfg->{namespace}) || defined($ccfg->{namespace})) {
+		    next;
+		}
+		die "Cannot determine which Ceph cluster the volume mapping belongs to. Abort!\n";
+	    }
+	}
+    }
+
     eval {
 	my $do_format = 0;
 	if ($scfg->{content}->{rootdir} && $scfg->{path}) {
-- 
2.30.2