[pve-devel] [PATCH firewall] make nfct_catch non-blocking

[David Limbeck]

Hi,

Did you test it without modifying the patch or only with your syslog change?

On 1/30/19 2:31 PM, Alexandre DERUMIER wrote:
> Hi,
>
> I have done some tests, and can't reproduce it.
>
> I wonder if it could be related to syslog, the only thing I have change, is dropping pve-firewall log in rsyslog.
>
>   218     // also log to syslog
>   219
>   220     vsyslog(loglevel, fmt, ap2);
>
>
> It's quite possible than /dev/log was overloaded with the rate, rsyslog was not able to spool it. (I also forward log to central syslog with tcp, could be related).
> I known if /dev/log buffer is full, syslog call are blocking.
>
> don't known how vsyslog() is working in this case.
>
> Could it be possible to have an option to disable syslog logging ? (or maybe add an option to use udp to send mail).
>
>
> Also, I have notice that we don't have timestamp in pve-firewall.log  for conntrack log.
> and maybe could we log them in a separate file ? (not sure how the gui will react if we need to filter a vm log, with the rate of new log coming)
>
>
>
> ----- Mail original -----
> De: "aderumier" <aderumier at odiso.com>
> À: "David Limbeck" <d.limbeck at proxmox.com>
> Cc: "pve-devel" <pve-devel at pve.proxmox.com>
> Envoyé: Samedi 26 Janvier 2019 08:07:43
> Objet: Re: [pve-devel] [PATCH firewall] make nfct_catch non-blocking
>
> Thanks !
>
> I'll test it Monday.
>
> ----- Mail original -----
> De: "David Limbeck" <d.limbeck at proxmox.com>
> À: "aderumier" <aderumier at odiso.com>, "Wolfgang Bumiller" <w.bumiller at proxmox.com>
> Cc: "pve-devel" <pve-devel at pve.proxmox.com>
> Envoyé: Vendredi 25 Janvier 2019 14:31:30
> Objet: Re: [pve-devel] [PATCH firewall] make nfct_catch non-blocking
>
> Hi,
>
> A new commit was pushed that enables building of debug symbols for
> pve-firewall. Please build and install it again with that commit
> included and run it again.
>
> This might help narrow it down some more.
>
> On 1/14/19 11:42 AM, Alexandre DERUMIER wrote:
>> Hi,
>>
>> I have able to reproduce, after 1hour.
>>
>> I have enable debug to get it run in foreground.
>>
>> This time, the process was not crashed, but was hanging.
>>
>> output was simply hanging, and no more write in /var/log/pve-firewall.log
>>
>> Also, memory was pretty huge, and still increasing during the hang (not sure if it's related to debug mode)
>>
>>
>> ps -aux|grep logger
>> root 19434 26.2 0.4 1770688 1679136 pts/1 Rl+ 10:44 11:27 ./pvefw-logger
>>
>> after some minutes
>>
>> root 19434 24.8 0.8 3625024 3533496 pts/1 Sl+ 10:44 12:20 ./pvefw-logger
>>
>>
>> I was able to do a coredump with gdb
>> http://odisoweb1.odiso.net/core.19434.gz
>>
>> Hope it's help.
>>
>>
>> ----- Mail original -----
>> De: "Wolfgang Bumiller" <w.bumiller at proxmox.com>
>> À: "aderumier" <aderumier at odiso.com>
>> Cc: "David Limbeck" <d.limbeck at proxmox.com>, "pve-devel" <pve-devel at pve.proxmox.com>
>> Envoyé: Lundi 14 Janvier 2019 08:01:54
>> Objet: Re: [pve-devel] [PATCH firewall] make nfct_catch non-blocking
>>
>> On Fri, Jan 11, 2019 at 06:05:36PM +0100, Alexandre DERUMIER wrote:
>>>>> Do you have any additional information as to why it stopped?
>>> no sorry.
>>>
>>>>> Maybe we could increase the buffer size via nfnl_set_rcv_buffer_size by
>>>>> default and continue to ignore ENOBUFS?
>>> I'll try next week. maybe doing strace on the process to have some clues ? (I'ts crashing after 30min-1h)
>> A coredump should work and produce less noise, perhaps?
>>
>>
> _______________________________________________
> pve-devel mailing list
> pve-devel at pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>
>