[pve-devel] applied: [PATCH manager 2/2] status/graphite: refactor write_graphite to send all at once

[Thomas Lamprecht]

Instead of doing multiple sends, for each status metric line one,
assemble it all in a string and send it out in a single go.
Per VM/CT/Node we had >10 lines to send, so this is quite the
reduction. But, also note that thanks to Nagler's delay algorithm
this may not had a big effect for TCP, as it buffered those small
writes anyhow.
For UDP it can reduce the packet count on the line dramatically,
though.

Signed-off-by: Thomas Lamprecht <t.lamprecht at proxmox.com>
---

note, this improves things sligthly, but our current usager of the plugin
modules in pvestatd is so full of overhead that it does not makes it good.

we create a new connection for each VM/CT/NODE/Storage, for each! and that on
every update loop iteration newly. While UDP doesn't even cares to much about
that, TCP really does, and it shows.

So there'll be some other patches comming to get this a bit perfomanter..

 PVE/Status/Graphite.pm | 52 ++++++++++++++++++++++--------------------
 1 file changed, 27 insertions(+), 25 deletions(-)

diff --git a/PVE/Status/Graphite.pm b/PVE/Status/Graphite.pm
index 8afadb53..06168cea 100644
--- a/PVE/Status/Graphite.pm
+++ b/PVE/Status/Graphite.pm
@@ -55,14 +55,6 @@ sub options {
     };
 }
 
-# we do not want boolean/state information to export to graphite
-my $key_blacklist = {
-    'template' => 1,
-    'pid' => 1,
-    'agent' => 1,
-    'serial' => 1,
-};
-
 # Plugin implementation
 sub update_node_status {
     my ($class, $plugin_config, $node, $data, $ctime) = @_;
@@ -119,25 +111,35 @@ sub write_graphite_hash {
 sub write_graphite {
     my ($carbon_socket, $d, $ctime, $path) = @_;
 
-    for my $key (keys %$d) {
-
-	my $value = $d->{$key};
-	my $oldpath = $path;
-	$key =~ s/\./-/g;
-	$path .= ".$key";
-
-	if ( defined $value ) {
-	    if ( ref $value eq 'HASH' ) {
-		write_graphite($carbon_socket, $value, $ctime, $path);
-	    } elsif ($value =~ m/^[+-]?[0-9]*\.?[0-9]+$/ &&
-		!$key_blacklist->{$key}) {
-		$carbon_socket->send( "$path $value $ctime\n" );
-	    } else {
-		# do not send blacklisted or non-numeric values
+    # we do not want boolean/state information to export to graphite
+    my $key_blacklist = {
+	'template' => 1,
+	'pid' => 1,
+	'agent' => 1,
+	'serial' => 1,
+    };
+
+    my $graphite_data = '';
+    my $assemble_graphite_data;
+    $assemble_graphite_data = sub {
+	my ($metric, $path) = @_;
+
+	for my $key (sort keys %$metric) {
+	    my $value = $d->{$key} // next;
+
+	    $key =~ s/\./-/g;
+	    my $metricpath = $path . ".$key";
+
+	    if (ref($value) eq 'HASH') {
+		$assemble_graphite_data->($value, $metricpath);
+	    } elsif ($value =~ m/^[+-]?[0-9]*\.?[0-9]+$/ && !$key_blacklist->{$key}) {
+		$graphite_data .= "$metricpath $value $ctime\n";
 	    }
 	}
-	$path = $oldpath;
-    }
+    };
+    $assemble_graphite_data->($d, $path);
+
+    $carbon_socket->send($graphite_data) if $graphite_data ne '';
 }
 
 PVE::JSONSchema::register_format('graphite-path', \&pve_verify_graphite_path);
-- 
2.20.1