[PVE-User] MTU size changed on a running cluster

[Wolfgang Link]

This is a openvswitch bug.
The workaround is to use openvswitch 2.6, it is on testing repo and set
mtu_reqest on the interface.

https://github.com/openvswitch/ovs/blob/master/FAQ.rst

Q: How can I configure the bridge internal interface MTU? Why does Open
vSwitch keep changing internal ports MTU?

A: By default Open vSwitch overrides the internal interfaces (e.g. br0)
MTU. If you have just an internal interface (e.g. br0) and a physical
interface (e.g. eth0), then every change in MTU to eth0 will be
reflected to br0. Any manual MTU configuration using ip or ifconfig on
internal interfaces is going to be overridden by Open vSwitch to match
the current bridge minimum.

Sometimes this behavior is not desirable, for example with tunnels. The
MTU of an internal interface can be explicitly set using the following
command:

$ ovs-vsctl set int br0 mtu_request=1450
After this, Open vSwitch will configure br0 MTU to 1450. Since this
setting is in the database it will be persistent (compared to what
happens with ip or ifconfig).

The MTU configuration can be removed to restore the default behavior with:

$ ovs-vsctl set int br0 mtu_request=[]
The mtu_request column can be used to configure MTU even for physical
interfaces (e.g. eth0).



On 11/23/2016 04:16 PM, Marco M. Gabriel wrote:
> Hi there,
> 
> on a productive 5 node Proxmox VE Ceph cluster, we experienced some strange
> behaviour:
> 
> Based on
> http://pve.proxmox.com/wiki/Open_vSwitch#Example_2:_Bond_.2B_Bridge_.2B_Internal_Ports
> we
> have an internal network for cluster/corosync communication and another
> internal network for Ceph Storage traffic. The Ceph OVS bridge was set to
> MTU 9000 in /etc/network/interfaces and ran without a problem since a week.
> 
> Today we've seen Ceph errors like "x requests are blocked > 32 sec".
> 
> After a troubleshooting, we's seen that packets got dropped because they
> were > 1500 bytes on the Ceph interface. That was strange as we had set
> them to MTU 9000 and it was running since a week.
> 
> We checked the Interfaces and on two nodes, we saw a MTU of 1500 while the
> other three nodes still had MTU 9000.
> 
> Has anybody experiences something like that? I read that an OVS bridge
> automatically sets it's own MTU according to the lowest MTU of the member
> interfaces, but I am not sure if this could be a problem here.
> 
> Any hints appreciated,
> Marco
> _______________________________________________
> pve-user mailing list
> pve-user at pve.proxmox.com
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>