[pve-devel] [PATCH cluster v3] add generic data broadcast interface

[Thomas Lamprecht]

On 5/14/19 11:15 AM, Dominik Csapak wrote:
> similar to how we handle the cluster wide tasklist and rrd data,
> have an interface that can sync data across the cluster
> 
> this data is only transient and will not be written to disk
> 
> we can use this for a number of things, e.g. listing ceph services
> across the cluster, etc.
> 
> Signed-off-by: Dominik Csapak <d.csapak at proxmox.com>
> ---
> changes from v2:
> * user proper sub ($sub->() vs $get_node_data->())
> * encode data only once
> * add comment about maximum pmxcfs size
> * die instead of eval/warn/syslog (caller must handle failure)
> * remove post if variable declaration
> 

implementation looks OK, but like we talked offline I'd have two changes in
behaviour:

* ability to "delete" a key - optimal, real deletion from the hashtable, else
  at leas a way to set it to '' or the like.
* "intelligent" data encoding, simplified and untested, something like:

    $data = encode_json($data) if (ref($data));
    die "to big" if length($data) > 32 * 1024;
    ...

 

>  data/PVE/Cluster.pm | 44 ++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 44 insertions(+)
> 
> diff --git a/data/PVE/Cluster.pm b/data/PVE/Cluster.pm
> index 5af11e6..4c719eb 100644
> --- a/data/PVE/Cluster.pm
> +++ b/data/PVE/Cluster.pm
> @@ -540,6 +540,50 @@ sub get_nodelist {
>      return [ keys %$nodelist ];
>  }
>  
> +# best effort data store for cluster
> +# this data is gone if the pmxcfs is restarted, but only the local data,
> +# so we should not use this for very important data
> +sub broadcast_node_kv {
> +    my ($key, $data) = @_;
> +
> +    my $enc_data = encode_json($data);
> +    my $size = length($enc_data);
> +    # pmxcfs has an upper bound of 32k for each entry
> +    if ($size >= (32 * 1024)) {
> +	warn "data for '$key' too big\n";
> +	return;
> +    }
> +
> +    eval {
> +	$ipcc_update_status->("kv/$key", $enc_data);
> +    };
> +
> +    warn $@ if $@;
> +}
> +
> +sub get_node_kv {
> +    my ($key, $nodename) = @_;
> +
> +    my $res = {};
> +    my $get_node_data = sub {
> +	my ($node) = @_;
> +	my $raw = $ipcc_get_status->("kv/$key", $node);
> +	$res->{$node} = decode_json($raw) if $raw;
> +    };
> +
> +    if ($nodename) {
> +	$get_node_data->($nodename);
> +    } else {
> +	my $nodelist = get_nodelist();
> +
> +	foreach my $node (@$nodelist) {
> +	    $get_node_data->($node);
> +	}
> +    }
> +
> +    return $res;
> +}
> +
>  # $data must be a chronological descending ordered array of tasks
>  sub broadcast_tasklist {
>      my ($data) = @_;
>