[pve-devel] [PATCH corosync] corosync.service: add patch to reduce log spam in broken network setups

[Friedrich Weber]

On 04/04/2025 10:55, Thomas Lamprecht wrote:
> Am 04.04.25 um 10:14 schrieb Maximiliano Sandoval:
>> Friedrich Weber <f.weber at proxmox.com> writes:
> 
>  ...
> 
>> An option that might require lower maintenance would be to ship a
>> service file override, e.g. at
>> /lib/systemd/system/corosync.service.d/set-log-rate-limit.conf with
>> contents:
>>
>> ```
>> [Service]
>> LogRateLimitIntervalSec=1s
>> LogRateLimitBurst=200
>> ```
>>
>> No strong feelings, it is just a matter of taste.
> 
> Would be more fitting if we did not package corosync our self, as is
> this integrated way would be fine to me. That sasid yours could be too.

Hmm, is this cut off?

> But ...
> 
>>
>>> + 1 file changed, 2 insertions(+)
>>> +
>>> +diff --git a/init/corosync.service.in b/init/corosync.service.in
>>> +index bd2a48a9..3d7ea2db 100644
>>> +--- a/init/corosync.service.in
>>> ++++ b/init/corosync.service.in
>>> +@@ -10,6 +10,8 @@ EnvironmentFile=- at INITCONFIGDIR@/corosync
>>> + ExecStart=@SBINDIR@/corosync -f $COROSYNC_OPTIONS
>>> + ExecStop=@SBINDIR@/corosync-cfgtool -H --force
>>> + Type=notify
>>> ++LogRateLimitIntervalSec=1s
>>> ++LogRateLimitBurst=200
>>
>> 200 hundred messages per second might be a bit too many. Since we are
>> not sure how many messages a unlucky user might see, I would suggest to
>> lower it a bit for the time being, 100 is a good round number.
>>
> 
> ... well, this is a core cluster service, having more available from a
> log burst is IMO really justified here.
> 
> That's also why I won't apply this patch for now, systemd already has
> default rate limiting for _very_ noisy stuff, it can also handle high
> log rates just fine and this only affects broken setups until they got
> fixed.

If I read the journald.conf docs [1] right, the default interval is 30s
and the burst value is 10000 multiplied by a factor depending on the
free disk space, I guess 4-6 on reasonable setups -- this is a lot of
messages, but as you mention probably fine for limiting really noisy
services. I was more thinking about this from a technical support
point-of-view, where I'd fear that having extreme corosync logspam over
days or weeks would cause the actually interesting stuff to be rotated
away more quickly than I'd like. :)

But as we have no idea how many broken setups are out there, this is all
somewhat hypothetical, so I'm also fine with not applying this -- if we
get many user reports seeing logspam I guess we can still do this.

[1]
https://www.freedesktop.org/software/systemd/man/latest/journald.conf.html#RateLimitIntervalSec=