[pve-devel] applied: [PATCH ceph master 1/3] fix #5213: ceph-osd postinst: add patch to avoid connection freezes

[Thomas Lamprecht]

Am 15/02/2024 um 10:40 schrieb Friedrich Weber:
> Assume there is an open TCP connection to a VM, and ceph-osd is
> installed/upgraded on the host on which the PVE firewall is active.
> Currently, ceph-osd postinst reloads all sysctl settings. Thus,
> installing/upgrading ceph-osd will set the sysctl setting
> `net.bridge.bridge-nf-call-iptables` to 0. The PVE firewall will flip
> the setting back to 1 in its next iteration (in <10 seconds). But
> while the setting is 0, conntrack will not see packets of the existing
> TCP connection. When the setting is flipped back to 1, conntrack will
> see packets again, but may consider the seq/ack numbers of new packets
> out-of-window, mark them as invalid and drop them. This will freeze
> the TCP connection.
> 
> To avoid this, add a patch that modifies the ceph-osd postinst to only
> apply settings from the sysctl settings file shipped with ceph-osd,
> and only apply them on fresh install. As the ceph-osd sysctl settings
> do not set `net.bridge.bridge-nf-call-iptables`, this will avoid the
> temporary flip to 0 when installing/upgrading ceph-osd.
> 
> Signed-off-by: Friedrich Weber <f.weber at proxmox.com>
> ---
>  ...t-avoid-reloading-all-sysctl-setting.patch | 47 +++++++++++++++++++
>  patches/series                                |  1 +
>  2 files changed, 48 insertions(+)
>  create mode 100644 patches/0015-ceph-osd-postinst-avoid-reloading-all-sysctl-setting.patch
> 
>

applied, thanks!

as talked off-list, ceph is really not trying to reduce confusion potential
doing things like:

 install -D -m 644 etc/sysctl/90-ceph-osd.conf $(DESTDIR)/etc/sysctl.d/30-ceph-osd.conf

I.e., having it checked in as 90-... but installing it as 30-..

And while I think the argument for "admin could have overrides that this
affects", which you mentioned that Fabian brought up off-list, is fine,
but is just as true on initial installation.

What might be better is one (or some) of:
- do nothing, just install the file and be done, a reboot sorts this out
  sooner or later anyway.
- a script that checks if there are any overrides and only sets it up if
  there are none else warns.
- just warns visible in general if lower values are detected.
- drop our odd disabling of the `net.bridge.bridge-nf-call-iptables`