[pmg-devel] applied-series: [PATCH pmg-api/gui/docs v2] enhance greylist (configurable mask + ipv6 support)

[Thomas Lamprecht]

On 4/20/20 1:22 PM, Stoiko Ivanov wrote:
> changes v1 -> v2:
> * after some discussion off-list with Dietmar and Dominik (mostly some
>   excellent feedback) v2 takes a simpler and more robust approach:
> ** the definition of the cgreylist table is only changed, by making the ipnet
>    column wider (49 instead of 16 characters)
> ** the data is still saved as text and postgres inet manipulation functions are
>    used for matching an ip to a saved network
> * fixed the default value in the GUI for the default v6 greylisting netmask
>   (v1 had /24 there now it's a more sensible /64).
> * did some testing with artificial greylist data (mostly with 1 million row)
>   the upgrade of the existing data does block for ~1 minute on my system
>   so maybe we could alternatively just not upgrade the greylist data and
>   start filling it anew (it would get deleted by the expiry mechanism in
>   pmgpolicy)
> 
> cover letter for v1:
> This patchset started out as adding support for configuring the netmask used
> for comparing triples of (ipnet,sender,receiver) while greylisting, because
> some cloud providers send out the same mail from different outbound ips
> (from a network which is larger then /19 - e.g. office365)
> 
> While looking through the code it seemed worthwhile to also add support for
> greylisting ipv6 addresses.
> 
> As a sideeffect the use_spf flag in pmgpolicy now also works for ipv6
> addresses.
> 
> One potential caveat compared to the current code is that it now could happen
> that we have 2 triples with the same sender+receiver but different ips added to
> the table in case those 2 ips send the mails to 2 different clusternodes
> within 2 minutes (clustersync intervall).
> 
> I tested the changes in my (limited, but clustered) environment:
> * sending from an ipv6 address not covered by the SPF record with hard fail
> * sending from an ipv6 address covered by the SPF record with hard fail
> * sending from different ipv6 addresses in the same configured network
> * syncing between an updated master and old node
> (all of the tests worked)
> 
> additionally the first patch for pmg-api fixes a glitch in test_greylist.pl
> (not stopping the pmgpolicy server used for testing when an error is
> encountered)
> 
> 
> pmg-api:
> Stoiko Ivanov (5):
>   test_greylist: exit pmgpolicy on failed test
>   use postgres inet functions for greylist matching
>   pmgpolicy: add IPv6 support
>   greylist: make netmasks configurable
>   add tests for greylisting ipv6
> 
>  src/PMG/Cluster.pm         |  7 ++--
>  src/PMG/Config.pm          | 24 +++++++++++++-
>  src/PMG/DBTools.pm         | 43 ++++++++++++++++++++++--
>  src/bin/pmgpolicy          | 67 ++++++++++++++++++++++++--------------
>  src/tests/test_greylist.pl | 39 ++++++++++++++++++++--
>  5 files changed, 148 insertions(+), 32 deletions(-)
> 
> pmg-gui:
> Stoiko Ivanov (1):
>   MailProxyOptions: add greylist enhancements
> 
>  js/MailProxyOptions.js | 11 ++++++++++-
>  1 file changed, 10 insertions(+), 1 deletion(-)
> 
> pmg-docs:
> Stoiko Ivanov (1):
>   add new greylist params gen-pmg.conf.5.-opts.pl
> 
>  gen-pmg.conf.5-opts.pl | 3 +++
>  1 file changed, 3 insertions(+)
> 

applied series, thanks! Followed up for the maxValue and indentation stuff Dominik
noted (thanks!) - the sync update issue shouldn't have serious effects and no real
good alternative is currently proposed - so ignoring it for now.