[pmg-devel] [PATCH pmg-api v3] config: adjust max_filters calculation to reflect current memory usage

[Markus Frank]

Hello,

thanks for looking into this.

On  2024-02-19 19:04, Alexander Zeidler wrote:
> On Thu, 2024-01-18 at 15:55 +0100, Markus Frank wrote:
> 
>>       my $max_servers = 5;
>>       my $servermem = 120;
>> +    my $base;
>>       my $memory = physical_memory();
>> -    my $add_servers = int(($memory - 512)/$servermem);
>> +    if ($memory < 3840) {
>> +	warn "low amount of system memory installed, recommended is 4+ GB\n"
>> +	    ."to prevent OOM kills, it is better to set max_filters manually\n";
>> +	$base = $memory > 1536 ? 1024 : 512;
>> +    } else {
>> +	$base = 2816;
>> +	$servermem = 150;
>> +    }
>> +    my $add_servers = int(($memory - $base)/$servermem);
>>       $max_servers += $add_servers if $add_servers > 0;
>>       $max_servers = 40 if  $max_servers > 40;
> 1. Perhaps we also want to increase the current max_filters limit of 40
> minus 2?
What would be the improvement? 38 processes seem enough to me.
Could you maybe trigger this much process at the same time?
> 
> 
> 2. With the recommended 4GB setup, the patch now only returns
> max_filters = 10 instead of the previous 32 as marked in the table
> below.
Since 4GB is lowest recommended memory setup, 10 processes seem fine to me.
Also I do not think a 2-4 Core 4GB setup could reach more 10 processes at the same time.
I could only trigger 3-6 processes running simultaneously in my testing with a flood of emails.
> 
> (table with temporarily removed 38-limit for the patch columns)
> 
> [1]     [2]    [3]  [4]   [5]
> RAM *0.975=    yet *120  left     patch       left
> 0.5     499      3        139         3 *120   139
>    1     998      7        158         7 *120   158
>    2    1996     15        196        11 *120   676
>    3    2995     23        235        19 *120   715
> 
>    4    3993     32        153    ==> 10 *150  2493
>    8    7987     38       3427        37 *150  2437
>   16   15974     38      11414        90 *150  2474
>   32   31948     38      27388       197 *150  2398
>   64   63897     38      59337       410 *150  2397
> 
Thanks for this calculation.
It shows that the remaining memory would be similar across
multiple memory configurations with this new calculation.

> [1] installed
> [2] ca. MemTotal according to /proc/meminfo, hence used for calculation
> [3] max_filters returned by current code
> [4] expected servermem used in code and table for calculation
> [5] (RAM * 0.975) - (max_filters * servermem)