[pve-devel] [PATCH firewall] make nfct_catch non-blocking

[Alexandre DERUMIER]

Hi,

I have done some tests, and can't reproduce it.

I wonder if it could be related to syslog, the only thing I have change, is dropping pve-firewall log in rsyslog.

 218     // also log to syslog
 219 
 220     vsyslog(loglevel, fmt, ap2);


It's quite possible than /dev/log was overloaded with the rate, rsyslog was not able to spool it. (I also forward log to central syslog with tcp, could be related).
I known if /dev/log buffer is full, syslog call are blocking.

don't known how vsyslog() is working in this case.

Could it be possible to have an option to disable syslog logging ? (or maybe add an option to use udp to send mail).


Also, I have notice that we don't have timestamp in pve-firewall.log  for conntrack log.
and maybe could we log them in a separate file ? (not sure how the gui will react if we need to filter a vm log, with the rate of new log coming)



----- Mail original -----
De: "aderumier" <aderumier at odiso.com>
À: "David Limbeck" <d.limbeck at proxmox.com>
Cc: "pve-devel" <pve-devel at pve.proxmox.com>
Envoyé: Samedi 26 Janvier 2019 08:07:43
Objet: Re: [pve-devel] [PATCH firewall] make nfct_catch non-blocking

Thanks ! 

I'll test it Monday. 

----- Mail original ----- 
De: "David Limbeck" <d.limbeck at proxmox.com> 
À: "aderumier" <aderumier at odiso.com>, "Wolfgang Bumiller" <w.bumiller at proxmox.com> 
Cc: "pve-devel" <pve-devel at pve.proxmox.com> 
Envoyé: Vendredi 25 Janvier 2019 14:31:30 
Objet: Re: [pve-devel] [PATCH firewall] make nfct_catch non-blocking 

Hi, 

A new commit was pushed that enables building of debug symbols for 
pve-firewall. Please build and install it again with that commit 
included and run it again. 

This might help narrow it down some more. 

On 1/14/19 11:42 AM, Alexandre DERUMIER wrote: 
> Hi, 
> 
> I have able to reproduce, after 1hour. 
> 
> I have enable debug to get it run in foreground. 
> 
> This time, the process was not crashed, but was hanging. 
> 
> output was simply hanging, and no more write in /var/log/pve-firewall.log 
> 
> Also, memory was pretty huge, and still increasing during the hang (not sure if it's related to debug mode) 
> 
> 
> ps -aux|grep logger 
> root 19434 26.2 0.4 1770688 1679136 pts/1 Rl+ 10:44 11:27 ./pvefw-logger 
> 
> after some minutes 
> 
> root 19434 24.8 0.8 3625024 3533496 pts/1 Sl+ 10:44 12:20 ./pvefw-logger 
> 
> 
> I was able to do a coredump with gdb 
> http://odisoweb1.odiso.net/core.19434.gz 
> 
> Hope it's help. 
> 
> 
> ----- Mail original ----- 
> De: "Wolfgang Bumiller" <w.bumiller at proxmox.com> 
> À: "aderumier" <aderumier at odiso.com> 
> Cc: "David Limbeck" <d.limbeck at proxmox.com>, "pve-devel" <pve-devel at pve.proxmox.com> 
> Envoyé: Lundi 14 Janvier 2019 08:01:54 
> Objet: Re: [pve-devel] [PATCH firewall] make nfct_catch non-blocking 
> 
> On Fri, Jan 11, 2019 at 06:05:36PM +0100, Alexandre DERUMIER wrote: 
>>>> Do you have any additional information as to why it stopped? 
>> no sorry. 
>> 
>>>> Maybe we could increase the buffer size via nfnl_set_rcv_buffer_size by 
>>>> default and continue to ignore ENOBUFS? 
>> I'll try next week. maybe doing strace on the process to have some clues ? (I'ts crashing after 30min-1h) 
> A coredump should work and produce less noise, perhaps? 
> 
> 

_______________________________________________ 
pve-devel mailing list 
pve-devel at pve.proxmox.com 
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel