[pmg-devel] [PATCH pmg-api] adapt journalctl invocation to buster

[Fabian Grünbichler]

On August 5, 2019 7:54 pm, Stoiko Ivanov wrote:
> With Debian Buster the behavior of `journalctl` has changed when it finds
> no entries for a given selection:
> * The exit code was 0 in stretch, but is 1 in buster
> * The output changed slightly - a header got added
> 
> Since PMG::Utils::scan_journal_for_rbl_rejects uses journalctl for reading the
> IPs blocked by postscreen it needs to adapt for the new behavior
> (otherwise the journal gets filled with spurious log-messages)
> 
> The patch addresses the problem by checking the die-message for
> 'failed: exit code 1`, and ignoring the error if the '-- No entries --' line
> was found and no rbl-counter got incremented

maybe it would make sense to switch this code to mini-journalreader?
we already use it for the regular log API endpoint.

it would require a patch to filter by identifier, but the default 
behaviour is much nicer since mini-journalreader will just print the 
start and end cursor if no messages are available, and exit with 0 ;)

it's also faster :-P

alternatively, journalctl has the following options

--output=json
--output-fields=MESSAGE

that would make this parsing a bit more sane, and remove the need for 
--show-cursor (every line contains a json object representing a journal 
entry, including the cursor, boot_id and timestamps). there is no 
special 'No entries' output in that case, so empty stderr + no output + 
exit code 1 => ignore error.

MESSAGE can contain newlines after all, so line-based parsing of plain 
journalctl output is a bit icky (that would also apply to 
mini-journalreader though, and I am not sure if we want json-encoding 
there?).

> Signed-off-by: Stoiko Ivanov <s.ivanov at proxmox.com>
> ---
> Thanks @Fabian for reporting the issue initially. I verified that the spamming
> in the journal started after an experimental upgrade to buster.
> 
>  src/PMG/Utils.pm | 10 +++++++++-
>  1 file changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/src/PMG/Utils.pm b/src/PMG/Utils.pm
> index aa6aac7..385b6d3 100644
> --- a/src/PMG/Utils.pm
> +++ b/src/PMG/Utils.pm
> @@ -1298,6 +1298,7 @@ sub scan_journal_for_rbl_rejects {
>  
>      my $rbl_count = 0;
>      my $pregreet_count = 0;
> +    my $noentries = 0;
>  
>      my $parser = sub {
>  	my $line = shift;
> @@ -1305,6 +1306,8 @@ sub scan_journal_for_rbl_rejects {
>  	if ($line =~ m/^--\scursor:\s(\S+)$/) {
>  	    $rbl_scan_last_cursor = $1;
>  	    return;
> +	} elsif ($line =~ m/^-- No entries --$/) {
> +	    $noentries = 1;
>  	}
>  
>  	if ($line =~ m/\s$identifier\[\d+\]:\sNOQUEUE:\sreject:.*550 5.7.1 Service unavailable;/) {
> @@ -1324,7 +1327,12 @@ sub scan_journal_for_rbl_rejects {
>  	push @$cmd, "--since=@" . $rbl_scan_start_time;
>      }
>  
> -    PVE::Tools::run_command($cmd, outfunc => $parser);
> +    eval{ PVE::Tools::run_command($cmd, outfunc => $parser); };
> +    my $err = $@;
> +    if ($err) {
> +	return if ( $err =~ m/failed: exit code 1$/ && $noentries == 1 && $rbl_count == 0 && $pregreet_count == 0);
> +	die "$err\n";
> +    }
>  
>      return ($rbl_count, $pregreet_count);
>  }
> -- 
> 2.20.1
> 
> 
> _______________________________________________
> pmg-devel mailing list
> pmg-devel at pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pmg-devel
> 
>