[pve-devel] pve-firewall: using NFLOG

[Alexandre DERUMIER]

>>The whole point is that I want to have local files with "easy to parse" format, so that we can view them 
>>easily and "fast" (those files can have many entries!). 
yes, no problem for local file

>>So I would like to have code to send data direct to log servers (using syslog protocol, or json, ...) 
>>We need that for pveproxy and pvefw-logger. 
>>
>>What do you think? 
Yes, I really need a central log server, it's easier to debug, with the firewalls on each host, in case of a ddos for example.


I'll check about logstash and input formats.

----- Mail original ----- 

De: "Dietmar Maurer" <dietmar at proxmox.com> 
À: "Alexandre DERUMIER" <aderumier at odiso.com> 
Cc: "Eric Blevins" <eric at netwalk.com>, pve-devel at pve.proxmox.com 
Envoyé: Vendredi 14 Mars 2014 09:02:28 
Objet: RE: [pve-devel] pve-firewall: using NFLOG 

> can now output in json format, and then logstash or splunk can read them 
> easily. 
> 
> code is here : 
> http://git.netfilter.org/ulogd2/tree/output/ulogd_output_JSON.c?id=2b39df 
> 550fbad944b4aab77617d4272c5d62ba70 
> 
> 
> It could be wonderfull to add this kind of output format 

I am quite unsure about that. Let me explain. 

1.) logstash, splunk, nxlog can read any format anyways (regex support to parse files) 

2.) using json blows up space usage 

3.) json is harder to read for humans 

4.) json require a parsers which is slow down things 

And finally, we can easily add code to send data directly to a log server (maybe in json format). 

The whole point is that I want to have local files with "easy to parse" format, so that we can view them 
easily and "fast" (those files can have many entries!). 

So I would like to have code to send data direct to log servers (using syslog protocol, or json, ...) 

We need that for pveproxy and pvefw-logger. 

What do you think?