[pmg-devel] [PATCH pmg-api/gui/docs v2] enhance greylist (configurable mask + ipv6 support)

[Stoiko Ivanov]

changes v1 -> v2:
* after some discussion off-list with Dietmar and Dominik (mostly some
  excellent feedback) v2 takes a simpler and more robust approach:
** the definition of the cgreylist table is only changed, by making the ipnet
   column wider (49 instead of 16 characters)
** the data is still saved as text and postgres inet manipulation functions are
   used for matching an ip to a saved network
* fixed the default value in the GUI for the default v6 greylisting netmask
  (v1 had /24 there now it's a more sensible /64).
* did some testing with artificial greylist data (mostly with 1 million row)
  the upgrade of the existing data does block for ~1 minute on my system
  so maybe we could alternatively just not upgrade the greylist data and
  start filling it anew (it would get deleted by the expiry mechanism in
  pmgpolicy)

cover letter for v1:
This patchset started out as adding support for configuring the netmask used
for comparing triples of (ipnet,sender,receiver) while greylisting, because
some cloud providers send out the same mail from different outbound ips
(from a network which is larger then /19 - e.g. office365)

While looking through the code it seemed worthwhile to also add support for
greylisting ipv6 addresses.

As a sideeffect the use_spf flag in pmgpolicy now also works for ipv6
addresses.

One potential caveat compared to the current code is that it now could happen
that we have 2 triples with the same sender+receiver but different ips added to
the table in case those 2 ips send the mails to 2 different clusternodes
within 2 minutes (clustersync intervall).

I tested the changes in my (limited, but clustered) environment:
* sending from an ipv6 address not covered by the SPF record with hard fail
* sending from an ipv6 address covered by the SPF record with hard fail
* sending from different ipv6 addresses in the same configured network
* syncing between an updated master and old node
(all of the tests worked)

additionally the first patch for pmg-api fixes a glitch in test_greylist.pl
(not stopping the pmgpolicy server used for testing when an error is
encountered)


pmg-api:
Stoiko Ivanov (5):
  test_greylist: exit pmgpolicy on failed test
  use postgres inet functions for greylist matching
  pmgpolicy: add IPv6 support
  greylist: make netmasks configurable
  add tests for greylisting ipv6

 src/PMG/Cluster.pm         |  7 ++--
 src/PMG/Config.pm          | 24 +++++++++++++-
 src/PMG/DBTools.pm         | 43 ++++++++++++++++++++++--
 src/bin/pmgpolicy          | 67 ++++++++++++++++++++++++--------------
 src/tests/test_greylist.pl | 39 ++++++++++++++++++++--
 5 files changed, 148 insertions(+), 32 deletions(-)

pmg-gui:
Stoiko Ivanov (1):
  MailProxyOptions: add greylist enhancements

 js/MailProxyOptions.js | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

pmg-docs:
Stoiko Ivanov (1):
  add new greylist params gen-pmg.conf.5.-opts.pl

 gen-pmg.conf.5-opts.pl | 3 +++
 1 file changed, 3 insertions(+)

-- 
2.20.1