[pve-devel] BUG in vlan aware bridge

Wed Oct 13 11:22:32 CEST 2021

HI,

Yes i think it has nothing to do with the bonds but with the vlan aware bridge interface.

I see this with ping -s 1500

On tap interface: 
11:19:35.141414 62:47:e0:fe:f9:31 > 54:e0:32:27:6e:50, ethertype IPv4 (0x0800), length 1514: (tos 0x0, ttl 64, id 39999, offset 0, flags [+], proto ICMP (1), length 1500)
    37.16.72.52 > 77.244.240.131: ICMP echo request, id 2182, seq 4, length 1480
11:19:35.141430 62:47:e0:fe:f9:31 > 54:e0:32:27:6e:50, ethertype IPv4 (0x0800), length 562: (tos 0x0, ttl 64, id 39999, offset 1480, flags [none], proto ICMP (1), length 548)
    37.16.72.52 > 77.244.240.131: ip-proto-1

On vmbr0:
11:19:35.141442 62:47:e0:fe:f9:31 > 54:e0:32:27:6e:50, ethertype 802.1Q (0x8100), length 2046: vlan 350, p 0, ethertype IPv4 (0x0800), (tos 0x0, ttl 64, id 39999, offset 0, flags [none], proto ICMP (1), length 2028)
    37.16.72.52 > 77.244.240.131: ICMP echo request, id 2182, seq 4, length 2008

On bond0 its gone....

But who is in charge of fragementing the packets normally? The bridge itself? Netfilter?

-----Ursprüngliche Nachricht-----
Von: pve-devel <pve-devel-bounces at lists.proxmox.com> Im Auftrag von Stoyan Marinov
Gesendet: Mittwoch, 13. Oktober 2021 00:46
An: Proxmox VE development discussion <pve-devel at lists.proxmox.com>
Betreff: Re: [pve-devel] BUG in vlan aware bridge

OK, I have just verified it has nothing to do with bonds. I get the same behavior with vlan aware bridge, bridge-nf-call-iptables=1 with regular eth0 being part of the bridge. Packets arrive fragmented on tap, reassembled by netfilter and then re-injected in bridge assembled (full size).

I did have limited success by setting net.bridge.bridge-nf-filter-vlan-tagged to 1. Now packets seem to get fragmented on the way out and back in, but there are still issues:

1. I'm testing with ping -s 2000 (1500 mtu everywhere) to an external box. I do see reply packets arrive on the vm nic, but ping doesn't see them. Haven't analyzed much further.
2. While watching with tcpdump (inside the vm) i notice "ip reassembly time exceeded" messages being generated from the vm.

I'll try to investigate a bit further tomorrow.

> On 12 Oct 2021, at 11:26 PM, Stoyan Marinov <stoyan at marinov.us> wrote:
> 
> That's an interesting observation. Now that I think about it, it could be caused by bonding and not the underlying device. When I tested this (about an year ago) I was using bonding on the mlx adapters and not using bonding on intel ones.
> 
>> On 12 Oct 2021, at 3:36 PM, VELARTIS Philipp Dürhammer <p.duerhammer at velartis.at> wrote:
>> 
>> HI,
>> 
>> we use HP Server with Intel Cards or the standard hp nic ( ithink also intel)
>> 
>> Also I see the I did a mistake:
>> 
>> Setup working:
>> tapX (UNtagged) <- -> vmbr0 <- - > bond0
>> 
>> is correct. (before I had also tagged) 
>> 
>> it should be :
>> 
>> Setup not working:
>> tapX (tagged) <- -> vmbr0 <- - > bond0
>> 
>> Setup working:
>> tapX (untagged) <- -> vmbr0 <- - > bond0
>> 
>> Setup also working:
>> tapX < - - > vmbr0v350 < -- > bond0.350 < -- > bond0
>> 
>> -----Ursprüngliche Nachricht-----
>> Von: pve-devel <pve-devel-bounces at lists.proxmox.com> Im Auftrag von Stoyan Marinov
>> Gesendet: Dienstag, 12. Oktober 2021 13:16
>> An: Proxmox VE development discussion <pve-devel at lists.proxmox.com>
>> Betreff: Re: [pve-devel] BUG in vlan aware bridge
>> 
>> I'm having the very same issue with Mellanox ethernet adapters. I don't see this behavior with Intel nics. What network cards do you have?
>> 
>>> On 12 Oct 2021, at 1:48 PM, VELARTIS Philipp Dürhammer <p.duerhammer at velartis.at> wrote:
>>> 
>>> HI,
>>> 
>>> i am playing around since days because we have strange packet losses.
>>> Finally I can report following (Linux 5.11.22-4-pve, Proxmox 7, all devices MTU 1500):
>>> 
>>> Packet with sizes > 1500 without VLAN working well but at the moment they are Tagged they are dropped by the bond device.
>>> Netfilter (set to 1) always reassembles the packets when they arrive a bridge. But they don't get fragmented again I they are VLAN tagged. So the bond device drops them. If the bridge is NOT Vlan aware they also get fragmented and it works well.
>>> 
>>> Setup not working:
>>> 
>>> tapX (tagged) <- -> vmbr0 <- - > bond0
>>> 
>>> Setup working:
>>> 
>>> tapX (tagged) <- -> vmbr0 <- - > bond0
>>> 
>>> Setup also working:
>>> 
>>> tapX < - - > vmbr0v350 < -- > bond0.350 < -- > bond0
>>> 
>>> Have you got any idea where to search? I don't understand who is in charge of fragmenting packages again if they get reassembled by netfilter. (and why it is not working with vlan aware bridges)
>>> 
>>> 
>>> _______________________________________________
>>> pve-devel mailing list
>>> pve-devel at lists.proxmox.com
>>> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>>> 
>> 
>> 
>> _______________________________________________
>> pve-devel mailing list
>> pve-devel at lists.proxmox.com
>> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>> _______________________________________________
>> pve-devel mailing list
>> pve-devel at lists.proxmox.com
>> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
> 
> _______________________________________________
> pve-devel mailing list
> pve-devel at lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

_______________________________________________
pve-devel mailing list
pve-devel at lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel