[pve-devel] [PATCH storage 1/2] fix #5779: rbd: allow to pass custom krbd map options

[Aaron Lauterer]

Does what it claims to do, setting the parameter `rxbounce` when mapping 
the RBD disk.

Therefore:

Tested-By: Aaron Lauterer <a.lauterer at proxmox.com>

But one question inline...

On  2024-10-25  13:13, Friedrich Weber wrote:
> When KRBD is enabled for an RBD storage, the storage plugin calls out
> to `rbd map` to map an RBD image as a block device on the host.
> Sometimes it might be necessary to pass custom options to `rbd map`.
> For instance, in some setups with Windows VMs, KRBD logs `bad
> crc/signature` and VMs performance is degraded unless the `rxbounce`
> option is enabled, as reported in the forum [1].
> 
> To allow users to specify custom options for KRBD, introduce a
> corresponding `krbd-map-options` property to the RBD plugin. The
> property is designed to only accept a supported set of map options.
> For now, this is only the `rxbounce` map option, but the supported set
> can be extended in the future.
> 
> The reasoning for constraining the supported set of map options
> instead of allowing to pass a free-form option string is as follows:
> If `rxbounce` turns out to be a sensible default, accepting a
> free-form option string now will make it hard to switch over the
> default to `rxbounce` while still allowing users to disable `rxbounce`
> if needed. This would require scanning the free-form string for a
> `norxbounce` or similar, which is cumbersome.
> 
> If users need to set a map option that `krbd-map-options` does not
> support (yet), they can alternatively set the RBD config option
> `rbd_default_map_options` [2].
> 
> [1] https://forum.proxmox.com/threads/155741/
> [2] https://github.com/ceph/ceph/blob/b2a4bd840/src/common/options/rbd.yaml.in#L507
> 
> Signed-off-by: Friedrich Weber <f.weber at proxmox.com>
> ---
>   src/PVE/Storage/Plugin.pm    | 10 ++++++++++
>   src/PVE/Storage/RBDPlugin.pm | 14 +++++++++++++-
>   2 files changed, 23 insertions(+), 1 deletion(-)
> 
> diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
> index 8cc693c..02be257 100644
> --- a/src/PVE/Storage/Plugin.pm
> +++ b/src/PVE/Storage/Plugin.pm
> @@ -394,6 +394,16 @@ sub verify_dir_override {
>       die "invalid override '$value'\n";
>   }
>   
> +PVE::JSONSchema::register_format('pve-storage-krbd-map-option', \&verify_krbd_map_option);
> +sub verify_krbd_map_option {
> +    my ($option, $noerr) = @_;
> +
> +    return $option if $option eq 'rxbounce';
> +
> +    return undef if $noerr;
> +    die "invalid krbd map option '$option'\n";
> +}
> +

Why do you place this RBD specific option in the general `Plugin.pm` and 
not into the `RBDPlugin.pm`?

A quick test of mine where I places this in the `RBDPlugin.pm` before 
the properties sub seemed to have worked fine.

>   sub private {
>       return $defaultData;
>   }
> diff --git a/src/PVE/Storage/RBDPlugin.pm b/src/PVE/Storage/RBDPlugin.pm
> index f45ad3f..8110cff 100644
> --- a/src/PVE/Storage/RBDPlugin.pm
> +++ b/src/PVE/Storage/RBDPlugin.pm
> @@ -394,6 +394,12 @@ sub properties {
>   	    description => "Client keyring contents (for external clusters).",
>   	    type => 'string',
>   	},
> +	'krbd-map-options' => {
> +	    description => "Additional map options (only for krbd). Supported:"
> +		." rxbounce.",
> +	    type => 'string',
> +	    format => 'pve-storage-krbd-map-option-list',
> +	},
>       };
>   }
>   
> @@ -410,6 +416,7 @@ sub options {
>   	krbd => { optional => 1 },
>   	keyring => { optional => 1 },
>   	bwlimit => { optional => 1 },
> +	'krbd-map-options' => { optional => 1 },
>       };
>   }
>   
> @@ -726,7 +733,12 @@ sub map_volume {
>       # features can only be enabled/disabled for image, not for snapshot!
>       $krbd_feature_update->($scfg, $storeid, $img_name);
>   
> -    my $cmd = $rbd_cmd->($scfg, $storeid, 'map', $name);
> +    my @additional_options = ();
> +    if ($scfg->{'krbd-map-options'}) {
> +	@additional_options = ('--options', $scfg->{'krbd-map-options'});
> +    }
> +
> +    my $cmd = $rbd_cmd->($scfg, $storeid, 'map', $name, @additional_options);
>       run_rbd_command($cmd, errmsg => "can't map rbd volume $name");
>   
>       return $kerneldev;