[pve-devel] [PATCH manager 6/8] ceph: add MDS create/delete/list API

[Thomas Lamprecht]

On 11/21/18 11:42 AM, Thomas Lamprecht wrote:
> Allow to create, list and destroy and Ceph Metadata Server (MDS) over
> the API and the CLI `pveceph` tool.
> 
> Besides setting up the local systemd service template and the MDS
> data directory we also add a reference to the MDS in the ceph.conf
> We note the backing host (node) from the respective MDS and set up a
> 'mds standby for name' = 'pve' so that the PVE created ones are a
> single group. If we decide to add integration for rank/path specific
> MDS (possible useful for CephFS with quite a bit of load) then this
> may help as a starting point.
> 
> On create, check early if a reference already exists in ceph.conf and
> abort in that case. If we only see existing data directories later
> on we do not remove them, they could well be from an older manual
> create - where it's possible dangerous to just remove it. Let the
> user handle it themself in that case.
> 
> Signed-off-by: Thomas Lamprecht <t.lamprecht at proxmox.com>
> Co-authored-by: Alwin Antreich <a.antreich at proxmox.com>
> ---
> 
> Not fully sure about the 'mds standby for name' so if you have objections
> regarding (negative) implications of this we can remove it just fine.
> 
> diff --git a/PVE/API2/Ceph/MDS.pm b/PVE/API2/Ceph/MDS.pm
> new file mode 100644
> index 00000000..990fd0fc
> --- /dev/null
> +++ b/PVE/API2/Ceph/MDS.pm
> @@ -0,0 +1,196 @@
> ....
> +__PACKAGE__->register_method ({
> +    name => 'index',
> +    path => '',
> +    method => 'GET',
> +    description => "Directory index.",
> +    permissions => {
> +	check => ['perm', '/', [ 'Sys.Audit', 'Datastore.Audit' ], any => 1],
> +    },
> +    parameters => {
> +	additionalProperties => 0,
> +	properties => {
> +	    node => get_standard_option('pve-node'),
> +	},
> +    },
> +    returns => {
> +	type => 'array',
> +	items => {
> +	    type => "object",
> +	    properties => {},
> +	},
> +	links => [ { rel => 'child', href => "{id}" } ],
> +    },
> +    code => sub {
> +	my ($param) = @_;
> +
> +	my $mds_list = PVE::CephTools::list_local_mds_ids();
> +
> +	my $res = [];
> +	foreach my $mds (@$mds_list) {
> +	    push @$res, { 'id' => $mds };
> +	}
> +
> +	return $res;


I'll probably do it more like:

        my $res = [];
                                                 
        my $cfg = PVE::CephTools::parse_ceph_config();         

        my $mds_hash = {};             
                                                                  
        foreach my $section (keys %$cfg) {
            my $d = $cfg->{$section};
                                                    
            if ($section =~ m/^mds\.(\S+)$/) {
                my $mds_id = $1;                                 
                if (defined($d->{host})) {
                    $mds_hash->{$mds_id} = {                                  
                        id => $mds_id,                                         
                        state => 'unknown',                   
                        host => $d->{host},                                  
                    };                                     
                }                       
            }                                           
        }
                             
        if (scalar(keys %$mds_hash) > 0) {
            my $rados = PVE::RADOS->new();

            my $mds_state = $rados->mon_command({ prefix => 'mgr dump' });              
     
            my $active_mds = $mds_state->{active_name};

            if ($active_mds && defined($mds_hash->{$active_mds})) {
                $mds_hash->{$active_mds}->{state} = 'active';
            }      
                       
            foreach my $standby (@{$mds_state->{standbys}}) {
                my $name = $standby->{name};
                if ($mds_hash->{$name}) {
                    $mds_hash->{$name}->{state} = 'standby';
                }                                
            }
        }          
                                  
        return PVE::RESTHandler::hash_to_array($mds_hash, 'id');


(with adapted return schema), as this makes it similar to list mon (and mgr upcoming
from Dominik) and will be used for the WebUI. While I dislike the 'we're on a node level
but return cluster wide info' here, it makes it easier and consistent.

We could clean this all up for a bigger API adaption, e.g. 6.0