[pve-devel] [PATCH container] Move volume activation to vm_start

[Fabian Grünbichler]

On June 17, 2020 2:18 pm, Stoiko Ivanov wrote:
> currently all volumes for a container are activated in the pre-start hook,
> which runs in a separate mount namespace (lxc.monitor.unshare is set to 1
> in our container config).
> This leads to problems with ZFS, if a pool is imported by this call the
> filesystems are mounted only inside the containers mount namespace.
> 
> by running the volume activation inside vm_start, right before starting the
> container via systemctl there should be no regression when using the API/our
> CLI tools.
> 
> Starting a container manually using lxc-start is usually done for obtaining
> debug-logs (after starting failed with our tooling) - so the potential for
> regression in that case should also be small.
> 
> The $loopdevlist variable is not used anywhere in our codebase since 2015
> (da6298481ea4dfe7d894f42fa105cda015ebe5ce).
> 
> Tested by creating a zpool, two containers with 'onboot: 1' stored on it,
> `echo -n '' > /etc/zfs/zpool.cache`, `update-initramfs -k all -u`, reboot
> 
> Signed-off-by: Stoiko Ivanov <s.ivanov at proxmox.com>
> ---
> This should address multiple reports in the forum (e.g. [0,1,2]), about
> containers on ZFS not starting after a reboot (mitigated by setting the correct
> cache-file, thus making the pool get imported through zfs-import-cache.service
> + zfs-mount.service)
> 
> Huge Thanks to Fabian and Wolfgang for providing input on this!
> 
> [0] https://forum.proxmox.com/threads/zfs-mount-on-start-problem-segfault-at-0-error-4-in-libc-2-28-so-subvolumes-not-mounted.68519/
> [1] https://forum.proxmox.com/threads/kann-keine-container-mehr-starten.69975/
> [2] https://forum.proxmox.com/threads/why-i-need-to-mount-zfs-volume-manualy-for-container-after-update-initramfs-usage.71008/
> 
> 
>  src/PVE/LXC.pm            | 5 +++++
>  src/lxc-pve-prestart-hook | 5 -----
>  2 files changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
> index f3aca7a..db5b8ca 100644
> --- a/src/PVE/LXC.pm
> +++ b/src/PVE/LXC.pm
> @@ -2186,6 +2186,11 @@ sub vm_start {
>  	close($fh);
>      }
>  
> +    my $storage_cfg = PVE::Storage::config();
> +    my $vollist = PVE::LXC::Config->get_vm_volumes($conf);
> +
> +    PVE::Storage::activate_volumes($storage_cfg, $vollist);
> +
>      my $cmd = ['systemctl', 'start', "pve-container\@$vmid"];
>  
>      PVE::GuestHelpers::exec_hookscript($conf, $vmid, 'pre-start', 1);
> diff --git a/src/lxc-pve-prestart-hook b/src/lxc-pve-prestart-hook
> index ed25aa4..8823cad 100755
> --- a/src/lxc-pve-prestart-hook
> +++ b/src/lxc-pve-prestart-hook
> @@ -38,11 +38,6 @@ PVE::LXC::Tools::lxc_hook('pre-start', 'lxc', sub {
>  
>      my $storage_cfg = PVE::Storage::config();
>  
> -    my $vollist = PVE::LXC::Config->get_vm_volumes($conf);
> -    my $loopdevlist = PVE::LXC::Config->get_vm_volumes($conf, 'rootfs');
> -
> -    PVE::Storage::activate_volumes($storage_cfg, $vollist);
> -

we are now missing an activate_volumes call for the following (unlikely, 
but possible) scenario:

pre-requisite:
running CT with pending mpX addition
pending volume needs activation (e.g., LVM)

trigger:
reboot (via API or within container)

expected outcome:
reboot, pending changes successfully applied, mpX propery mounted

actual outcome:
pending changes applied, CT fails to reboot

one way to trigger this is to:
add volume A to container
remove volume A again (now unused)
add volume B to container
start container
detach volume B (pending)
re-attach volume A to slot where volume B was/is
reboot container

maybe pve-container-stop-wrapper would be a good place for that 
activation?

there is also some other, unrelated issue with hotplugging newly created 
volumes over pending-detached volumes which I've filed as #2820.

>      my $rootdir = $vars->{ROOTFS_PATH};
>  
>      # Delete any leftover reboot-trigger file
> -- 
> 2.20.1
> 
> 
> _______________________________________________
> pve-devel mailing list
> pve-devel at pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
>