[pmg-devel] [PATCH pmg-api v2] utils: fix mailflow if smtputf8 is disabled

[Stoiko Ivanov]

with the recent addition of smtputf8 support for the rulesystem setups
explicitly disabling smtputf8 in postfix got broken.

This is mostly noticeable for the spamreports (the receivers are taken
from the database and potentially decoded from utf-8, which sets the
'is_utf8' flag, and then tries to use the smtputf8 extension when
reinjecting the mail, which fails (since smtputf8 is disabled)

Instead of checking for the internal flag, we check for occurence of
characters which are not ascii printable (everything excluding
controlcharacters - '[\x20-\x7E]') in the envelope-addresses and
headers (there also for [\r\n\t], due to searching all headers and
folding). - see
https://perldoc.perl.org/perlunifaq#What-is-%22the-UTF8-flag%22?  and
https://perldoc.perl.org/perlrecharclass#POSIX-Character-Classes

The only diversion from the requirements in the smptutf8 rfc
https://www.rfc-editor.org/rfc/rfc6531
is that we do not check the headers of all parts of a multipart
message (think suggested filename for an attachment), but I assume
that this should not be an issue in mail-transit

the addresses now always get encoded as UTF-8, as this is robust for
aascii-only addresses.

reported in our community forum:
https://forum.proxmox.com/threads/.119387/

issue is reproducible by setting
`smtputf8_enable = no` in postfix main.cf
and sending a spamreport using `pmgqm`

regular mailflow should not be affected in those setups (as no utf-8
addresses would come into the system)

Signed-off-by: Stoiko Ivanov <s.ivanov at proxmox.com>
---
v1->v2:
* as suggested by Dominik (huge thanks for the thorough review and the
  suggestions!) the (top-level) mail headers are also scanned for non-ascii
  printable characters (and \n\r\t, since those occur in headers as strings)
* put the test in a sub of its own
* addresses are now always encoded as utf-8 (since for ascii only addresses
  this should be identity

 src/PMG/Utils.pm | 42 +++++++++++++++++++++---------------------
 1 file changed, 21 insertions(+), 21 deletions(-)

diff --git a/src/PMG/Utils.pm b/src/PMG/Utils.pm
index 10193f6..825b8d9 100644
--- a/src/PMG/Utils.pm
+++ b/src/PMG/Utils.pm
@@ -221,6 +221,24 @@ sub subst_values_for_header {
     return $res;
 }
 
+sub mail_needs_smtputf8 {
+    my ($entity, $sender, $targets) = @_;
+
+    return 1 if ($sender =~ /[^\p{PosixPrint}]/);
+
+    foreach my $target (@$targets) {
+	if ($target =~ /[^\p{PosixPrint}]/) {
+	    return 1;
+	}
+    }
+
+    if ($entity->head()->as_string() =~ /([^\p{PosixPrint}\n\r\t])/) {
+	return 1;
+    }
+
+    return 0;
+}
+
 sub reinject_mail {
     my ($entity, $sender, $targets, $xforward, $me, $params) = @_;
 
@@ -245,23 +263,9 @@ sub reinject_mail {
 	    }
 	}
 
-	my $has_utf8_targets = 0;
-	foreach my $target (@$targets) {
-	    if (utf8::is_utf8($target)) {
-		$has_utf8_targets = 1;
-		last;
-	    }
-	}
-
 	my $mail_opts = " BODY=8BITMIME";
-	my $sender_addr;
-	if (utf8::is_utf8($sender)) {
-	    $sender_addr = encode('UTF-8', $smtp->_addr($sender));
-	    $mail_opts .= " SMTPUTF8";
-	} else {
-	    $sender_addr = $smtp->_addr($sender);
-	    $mail_opts .= " SMTPUTF8" if $has_utf8_targets;
-	}
+	$mail_opts .= " SMTPUTF8" if mail_needs_smtputf8($entity, $sender, $targets);
+	my $sender_addr = encode('UTF-8', $smtp->_addr($sender));
 
 	if (defined($params->{mail})) {
 	    my $mailparams = $params->{mail};
@@ -284,12 +288,8 @@ sub reinject_mail {
 		    $rcpt_opts .= " $p=$rcptparams->{$p}";
 		}
 	    }
+	    $rcpt_addr = encode('UTF-8', $smtp->_addr($target));
 
-	    if (utf8::is_utf8($target)) {
-		$rcpt_addr = encode('UTF-8', $smtp->_addr($target));
-	    } else {
-		$rcpt_addr = $smtp->_addr($target);
-	    }
 	    if (!$smtp->_RCPT("TO:" . $rcpt_addr . $rcpt_opts)) {
 		syslog ('err', "smtp error - got: %s %s", $smtp->code, scalar($smtp->message));
 		die "smtp to: ERROR";
-- 
2.30.2