[pve-devel] new bridge code doesn't work with redhat kernel
Stefan Priebe - Profihost AG
s.priebe at profihost.ag
Thu Feb 7 14:56:28 CET 2013
Hi,
as Patrick says - it should / must be on top of the bridge. So my code
is fine but the RHEL kernel is buggy. Strangely GVRP works correctly
with RHEL kernel... so GVRP stays up with my patches but in your case
network isn't pingable.
---------------------------------------------------
What seems to be happening is that the switch is constantly requesting
GVRP updates. Once eth0 is added to the bridge, the bridge has preference
over local protocol (LLC) delivery. Since the packet has a link local
destination and your bridge doesn't seem to have STP enabled, the
bridge decides to forward the packet. Since its a multicast destination,
the packet is finally passed up to LLC, but the receiving device is
the bridge, not eth0. When the packet reaches the GARP input handler,
no garp_port is associated to the device and the packet is dropped.
This result is that the GARP protocol doesn't notice that the switch
expects an update and the switch finally removes the VLAN.
So the solution is: put the VLAN on the bridge device.
------------------------------------------------------
Stefan
Am 07.02.2013 13:44, schrieb Alexandre DERUMIER:
>>> But ovirt does it also the other way.
> By other way, you mean vlan on interface ?
>
> because ovirt ~= opensource rhev, and rhev use redhat kernel. S
> o it make sense that they put vlan on interface, if it's not working to put vlan on bridge for redhat.
>
> Maybe they doesn't care about gvrp
>
>
> ----- Mail original -----
>
> De: "Stefan Priebe - Profihost AG" <s.priebe at profihost.ag>
> À: "Alexandre DERUMIER" <aderumier at odiso.com>
> Cc: pve-devel at pve.proxmox.com
> Envoyé: Jeudi 7 Février 2013 13:14:45
> Objet: Re: [pve-devel] new bridge code doesn't work with redhat kernel
>
> But ovirt does it also the other way.
>
> Stefan
> Am 07.02.2013 13:13, schrieb Alexandre DERUMIER:
>> I think it must a be a bug in redhat kernel, as it's working fine with newer kernels.
>>
>>
>>
>>
>> ----- Mail original -----
>>
>> De: "Stefan Priebe - Profihost AG" <s.priebe at profihost.ag>
>> À: "Alexandre DERUMIER" <aderumier at odiso.com>
>> Cc: pve-devel at pve.proxmox.com
>> Envoyé: Jeudi 7 Février 2013 12:59:21
>> Objet: Re: [pve-devel] new bridge code doesn't work with redhat kernel
>>
>> Hi,
>>
>> patrick on the kernel mailing list also suggests to add the vlan on top
>> of the bridge like my patch does for PVE and not on top of the ethernet
>> device. It seems nobody knows what is the correct way. So on kernel is
>> broken - RHEL or vanilla ;-)
>>
>> Stefan
>>
>> Am 07.02.2013 12:20, schrieb Alexandre DERUMIER:
>>> ok, so you are sure that is the bridge which cause the problem ?
>>>
>>> I'll try to find some infos/bugreport about it.
>>>
>>>
>>> ----- Mail original -----
>>>
>>> De: "Stefan Priebe - Profihost AG" <s.priebe at profihost.ag>
>>> À: "Alexandre DERUMIER" <aderumier at odiso.com>
>>> Cc: pve-devel at pve.proxmox.com
>>> Envoyé: Jeudi 7 Février 2013 11:57:40
>>> Objet: Re: [pve-devel] new bridge code doesn't work with redhat kernel
>>>
>>> Hi,
>>>
>>> the kernel just stops sending out GVRP packets after adding a bridge or
>>> does not even start to sending packets when the device already has a
>>> bridge except the initial packet when creating the vlan.
>>>
>>> Stefan
>>> Am 07.02.2013 11:44, schrieb Alexandre DERUMIER:
>>>>>> OK my findings are wrong. The gvrp bridge disappears even without adding
>>>>>> a bridge. It seems the first initial gvrp package is send but then the
>>>>>> "keep-alive" packages aren't.
>>>>
>>>> How does work gvrp exactly ? (Don't have switch with gvrp to test).
>>>>
>>>> is it sending constantly vlans info to the switch ?
>>>>
>>>>
>>>> I have checked the redhat init script when they have implemeted gvrp
>>>> https://bugzilla.redhat.com/attachment.cgi?id=417903&action=diff
>>>>
>>>> ip link add dev ${DEVICE} link ${PHYSDEV} type vlan id ${VID} ${FLAG_REORDER_HDR} ${FLAG_GVRP}
>>>>
>>>> so, it should work
>>>>
>>>>
>>>> Maybe wireshark could help ?
>>>>
>>>> ----- Mail original -----
>>>>
>>>> De: "Stefan Priebe - Profihost AG" <s.priebe at profihost.ag>
>>>> À: "Alexandre DERUMIER" <aderumier at odiso.com>
>>>> Cc: pve-devel at pve.proxmox.com
>>>> Envoyé: Jeudi 7 Février 2013 11:37:46
>>>> Objet: Re: [pve-devel] new bridge code doesn't work with redhat kernel
>>>>
>>>> Hi,
>>>>
>>>> OK my findings are wrong. The gvrp bridge disappears even without adding
>>>> a bridge. It seems the first initial gvrp package is send but then the
>>>> "keep-alive" packages aren't.
>>>>
>>>> Stefan
>>>> Am 07.02.2013 11:34, schrieb Stefan Priebe - Profihost AG:
>>>>> Hi Alexandre,
>>>>>
>>>>> i've discovered something even more strange:
>>>>>
>>>>> # ip link add link eth1 name eth1.99 type vlan id 99 reorder_hdr on gvrp on
>>>>> # ip link set eth1.99 up
>>>>>
>>>>> Switch:
>>>>>
>>>>> # show vlan
>>>>> VLAN ID Name Status Jumbo
>>>>> ------- -------------------- ------------ -----
>>>>> 99 GVRP_99 Dynamic
>>>>>
>>>>>
>>>>> Now i add a bridge just the bridge i do not even assign an interface:
>>>>> # brctl addbr vmbr1v99
>>>>>
>>>>> Switch:
>>>>>
>>>>> # show vlan
>>>>> VLAN ID Name Status Jumbo
>>>>> ------- -------------------- ------------ -----
>>>>>
>>>>> The gvrp vlan disappears when i just add a bridge to the host.
>>>>>
>>>>>
>>>>>> so, GVRP doesn't work when vlan are defined on interfaces or bond
>>>>>> with redhat kernel and vanilla kernel ?
>>>>>>
>>>>>> gvrp seem available on redhat kernel
>>>>>> CONFIG_VLAN_8021Q=m
>>>>>> CONFIG_VLAN_8021Q_GVRP=y
>>>>>
>>>>> Yes - but i seems it should work. It disappears after using brctl. Maybe
>>>>> a bug in brctl code? I'll try to find out more.
>>>>>
>>>>> Stefan
>>>>>
>>>>> Am 07.02.2013 11:25, schrieb Alexandre DERUMIER:
>>>>>>>> yes while rechecking the code i saw the same. But then GVRP does not
>>>>>>>> work. The relevant packet does not leave the network interface. So maybe
>>>>>>>> it's a vanilla kernel bug? Right now i've no more ideas how / what to test.
>>>>>>
>>>>>> so, GVRP doesn't work when vlan are defined on interfaces or bond with redhat kernel and vanilla kernel ?
>>>>>>
>>>>>> gvrp seem available on redhat kernel
>>>>>> CONFIG_VLAN_8021Q=m
>>>>>> CONFIG_VLAN_8021Q_GVRP=y
>>>>>>
>>>>>>
>>>>>>
>>>>>> ----- Mail original -----
>>>>>>
>>>>>> De: "Stefan Priebe - Profihost AG" <s.priebe at profihost.ag>
>>>>>> À: "Alexandre DERUMIER" <aderumier at odiso.com>
>>>>>> Cc: pve-devel at pve.proxmox.com
>>>>>> Envoyé: Jeudi 7 Février 2013 11:10:42
>>>>>> Objet: Re: [pve-devel] new bridge code doesn't work with redhat kernel
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> yes while rechecking the code i saw the same. But then GVRP does not
>>>>>> work. The relevant packet does not leave the network interface. So maybe
>>>>>> it's a vanilla kernel bug? Right now i've no more ideas how / what to test.
>>>>>>
>>>>>> Greets,
>>>>>> Stefan
>>>>>> Am 07.02.2013 10:59, schrieb Alexandre DERUMIER:
>>>>>>>>> Right now I'm not sure how it works under PVE. I've to recheck the code.
>>>>>>>
>>>>>>> It's work like redhat
>>>>>>>
>>>>>>> eth0 ---vmbrx
>>>>>>> eth0 -- eth0.10 --- vmbrxV10
>>>>>>>
>>>>>>> eth0 --bond0 -- bond0.10 --- vmbrxV10
>>>>>>> eth1 -|
>>>>>>>
>>>>>>> ----- Mail original -----
>>>>>>>
>>>>>>> De: "Stefan Priebe - Profihost AG" <s.priebe at profihost.ag>
>>>>>>> À: "Alexandre DERUMIER" <aderumier at odiso.com>
>>>>>>> Cc: pve-devel at pve.proxmox.com
>>>>>>> Envoyé: Jeudi 7 Février 2013 10:12:14
>>>>>>> Objet: Re: [pve-devel] new bridge code doesn't work with redhat kernel
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> oh i think it's clear.
>>>>>>>
>>>>>>> They put the vlan on top of the bond and then the bridge on top of the vlan.
>>>>>>>
>>>>>>> without a bond they add the vlan on top of the nic and then on top of
>>>>>>> the vlan the bridge.
>>>>>>>
>>>>>>> Right now I'm not sure how it works under PVE. I've to recheck the code.
>>>>>>>
>>>>>>> Stefan
>>>>>>> Am 06.02.2013 16:15, schrieb Alexandre DERUMIER:
>>>>>>>>>> mhm strange i was sure that i've tested it with latest proxmox kernel.
>>>>>>>>>> Will retest. Didn't redhat backport all those changes done in vanilla?
>>>>>>>>
>>>>>>>> Not all new features are backported. (I think we need to wait for rhel7 to have a big jump in kernel version)
>>>>>>>>
>>>>>>>>
>>>>>>>>>> mean what's the correct workflow under redhat to use bond + bridge + vlan.
>>>>>>>>
>>>>>>>> here the ovirt (and rhev) network setup
>>>>>>>>
>>>>>>>> http://www.ovirt.org/Vdsm_Network
>>>>>>>>
>>>>>>>> But it's not clear if they put vlan on bridge or on interface.
>>>>>>>>
>>>>>>>> ----- Mail original -----
>>>>>>>>
>>>>>>>> De: "Stefan Priebe - Profihost AG" <s.priebe at profihost.ag>
>>>>>>>> À: "Alexandre DERUMIER" <aderumier at odiso.com>
>>>>>>>> Cc: pve-devel at pve.proxmox.com
>>>>>>>> Envoyé: Mercredi 6 Février 2013 14:48:11
>>>>>>>> Objet: Re: [pve-devel] new bridge code doesn't work with redhat kernel
>>>>>>>>
>>>>>>>> Hio,
>>>>>>>>
>>>>>>>> mhm strange i was sure that i've tested it with latest proxmox kernel.
>>>>>>>> Will retest. Didn't redhat backport all those changes done in vanilla? I
>>>>>>>> mean what's the correct workflow under redhat to use bond + bridge + vlan.
>>>>>>>>
>>>>>>>> Stefan
>>>>>>>>
>>>>>>>> Am 06.02.2013 14:35, schrieb Alexandre DERUMIER:
>>>>>>>>>>> Maybe, I'll do test without bond.
>>>>>>>>>
>>>>>>>>> Doesn't work without bond too :(
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ----- Mail original -----
>>>>>>>>>
>>>>>>>>> De: "Alexandre DERUMIER" <aderumier at odiso.com>
>>>>>>>>> À: "Stefan Priebe - Profihost AG" <s.priebe at profihost.ag>
>>>>>>>>> Cc: pve-devel at pve.proxmox.com
>>>>>>>>> Envoyé: Mercredi 6 Février 2013 14:22:08
>>>>>>>>> Objet: Re: [pve-devel] new bridge code doesn't work with redhat kernel
>>>>>>>>>
>>>>>>>>>>> Urgh... that's pretty sad. It was working fine my test. But maybe i
>>>>>>>>>>> didn't test everything. Could you desribe me how exactly you've done the
>>>>>>>>>>> test?
>>>>>>>>>
>>>>>>>>> simply start the vm with old code and new code. (I have reboot the host to be sure)
>>>>>>>>> can't ping the vm with new code.
>>>>>>>>>
>>>>>>>>> But it's work fine with kernel 3.7.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> I've no bond running - just plain eth => bridge. Maybe that's the
>>>>>>>>>>> important difference?
>>>>>>>>>
>>>>>>>>> Maybe, I'll do test without bond.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> It's really possible it's a kernel problem, because like I said it, a lot of work has been done in kernel around 2.6.39 on the vlan code.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ----- Mail original -----
>>>>>>>>>
>>>>>>>>> De: "Stefan Priebe - Profihost AG" <s.priebe at profihost.ag>
>>>>>>>>> À: "Alexandre DERUMIER" <aderumier at odiso.com>
>>>>>>>>> Cc: "Dietmar Maurer" <dietmar at proxmox.com>, pve-devel at pve.proxmox.com
>>>>>>>>> Envoyé: Mercredi 6 Février 2013 13:56:08
>>>>>>>>> Objet: Re: new bridge code doesn't work with redhat kernel
>>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Urgh... that's pretty sad. It was working fine my test. But maybe i
>>>>>>>>> didn't test everything. Could you desribe me how exactly you've done the
>>>>>>>>> test?
>>>>>>>>>
>>>>>>>>> I've no bond running - just plain eth => bridge. Maybe that's the
>>>>>>>>> important difference?
>>>>>>>>>
>>>>>>>>> Stefan
>>>>>>>>>
>>>>>>>>> Am 06.02.2013 09:47, schrieb Alexandre DERUMIER:
>>>>>>>>>> Hi,
>>>>>>>>>> I'm testing the new bridge code, and it doesn't work for me !
>>>>>>>>>>
>>>>>>>>>> setup: bridge vmbr1, on top of bond0
>>>>>>>>>> guest vm (id 115) have an interface in vlan95
>>>>>>>>>> host kernel 2.6.32-18-pve
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> But it's working fine with 3.7 kernel
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Also I notice than we update the pve-common code, the old tagged network interface remain in bridge,
>>>>>>>>>> I don't known if it's can do network loop. (bond0.95 and vmbr1.95 in same bridge in my example)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> old
>>>>>>>>>> --------------
>>>>>>>>>> #brctl show
>>>>>>>>>>
>>>>>>>>>> vmbr1 8000.001aa03c98c5 no bond0
>>>>>>>>>>
>>>>>>>>>> vmbr1v95 8000.001aa03c98c5 no bond0.95
>>>>>>>>>> tap115i0
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> update to last pve-common, restart the vm
>>>>>>>>>> -------------------------------------------
>>>>>>>>>> #qm stop 115
>>>>>>>>>> #qm start 115
>>>>>>>>>> #brctl show
>>>>>>>>>>
>>>>>>>>>> vmbr1 8000.001aa03c98c5 no bond0
>>>>>>>>>> vmbr1v95 8000.001aa03c98c5 no bond0.95 -->old bond0.95 always attached (don't known if it can cause network loop ?)
>>>>>>>>>> tap115i0
>>>>>>>>>> vmbr1.95
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> (I have also restart the server to have a clean bridge, doesn't work too)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ----- Mail original -----
>>>>>>>>>>
>>>>>>>>>> De: "Stefan Priebe" <s.priebe at profihost.ag>
>>>>>>>>>> À: "Dietmar Maurer" <dietmar at proxmox.com>
>>>>>>>>>> Cc: pve-devel at pve.proxmox.com
>>>>>>>>>> Envoyé: Lundi 28 Janvier 2013 17:45:27
>>>>>>>>>> Objet: Re: [pve-devel] [PATCH] pve-common: PVE/Network: rework of activate_bridge_vlan
>>>>>>>>>>
>>>>>>>>>> Thanks!
>>>>>>>>>> Am 28.01.2013 12:00, schrieb Dietmar Maurer:
>>>>>>>>>>> applied, thanks!
>>>>>>>>>>>
>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>> From: pve-devel-bounces at pve.proxmox.com [mailto:pve-devel-
>>>>>>>>>>>> bounces at pve.proxmox.com] On Behalf Of Stefan Priebe
>>>>>>>>>>>> Sent: Freitag, 25. Jänner 2013 22:16
>>>>>>>>>>>> To: pve-devel at pve.proxmox.com
>>>>>>>>>>>> Subject: [pve-devel] [PATCH] pve-common: PVE/Network: rework of
>>>>>>>>>>>> activate_bridge_vlan
>>>>>>>>>>>>
>>>>>>>>>>>> - use ip command instead of old vconfig
>>>>>>>>>>>> - activate gvrp by default (it doesn't harm if the switch does not support it or
>>>>>>>>>>>> it is disabled)
>>>>>>>>>>>> - use bridge instead of raw ethernet device as vlan bridge
>>>>>>>>>>>>
>>>>>>>>>>>> Signed-off-by: Stefan Priebe <s.priebe at profihost.ag>
>>>>>>>>>>>> ---
>>>>>>>>>>>> data/PVE/Network.pm | 22 ++++------------------
>>>>>>>>>>>> 1 file changed, 4 insertions(+), 18 deletions(-)
>>>>>>>>>>>>
>>>>>>>>>>>> diff --git a/data/PVE/Network.pm b/data/PVE/Network.pm index
>>>>>>>>>>>> 2c356eb..71045fe 100644
>>>>>>>>>>>> --- a/data/PVE/Network.pm
>>>>>>>>>>>> +++ b/data/PVE/Network.pm
>>>>>>>>>>>> @@ -72,28 +72,14 @@ sub activate_bridge_vlan {
>>>>>>>>>>>> die "got strange vlan tag '$tag_param'\n" if $tag < 1 || $tag > 4094;
>>>>>>>>>>>>
>>>>>>>>>>>> my $bridgevlan = "${bridge}v$tag";
>>>>>>>>>>>> -
>>>>>>>>>>>> - my $dir = "/sys/class/net/$bridge/brif";
>>>>>>>>>>>> -
>>>>>>>>>>>> - #check if we have an only one ethX or bondX interface in the bridge
>>>>>>>>>>>> -
>>>>>>>>>>>> - my $iface;
>>>>>>>>>>>> - PVE::Tools::dir_glob_foreach($dir, '((eth|bond)\d+)', sub {
>>>>>>>>>>>> - my ($slave) = @_;
>>>>>>>>>>>> -
>>>>>>>>>>>> - die "more then one physical interfaces on bridge '$bridge'\n" if
>>>>>>>>>>>> $iface;
>>>>>>>>>>>> - $iface = $slave;
>>>>>>>>>>>> -
>>>>>>>>>>>> - });
>>>>>>>>>>>> -
>>>>>>>>>>>> - die "no physical interface on bridge '$bridge'\n" if !$iface;
>>>>>>>>>>>> -
>>>>>>>>>>>> + my $iface = $bridge;
>>>>>>>>>>>> my $ifacevlan = "${iface}.$tag";
>>>>>>>>>>>> + my $vlanflags = "reorder_hdr on gvrp on";
>>>>>>>>>>>>
>>>>>>>>>>>> # create vlan on $iface is not already exist
>>>>>>>>>>>> if (! -d "/sys/class/net/$ifacevlan") {
>>>>>>>>>>>> - system("/sbin/vconfig add $iface $tag") == 0 ||
>>>>>>>>>>>> - die "can't add vlan tag $tag to interface $iface\n";
>>>>>>>>>>>> + system("/sbin/ip link add link $iface name $ifacevlan type vlan id $tag
>>>>>>>>>>>> $vlanflags") == 0 ||
>>>>>>>>>>>> + die "can't add vlan tag $tag to interface $iface\n";
>>>>>>>>>>>> }
>>>>>>>>>>>>
>>>>>>>>>>>> # be sure to have the $ifacevlan up
>>>>>>>>>>>> --
>>>>>>>>>>>> 1.7.10.4
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> pve-devel mailing list
>>>>>>>>>>>> pve-devel at pve.proxmox.com
>>>>>>>>>>>> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> pve-devel mailing list
>>>>>>>>>> pve-devel at pve.proxmox.com
>>>>>>>>>> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> pve-devel mailing list
>>>>>>>>> pve-devel at pve.proxmox.com
>>>>>>>>> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>>>>>>>>>
>>>>> _______________________________________________
>>>>> pve-devel mailing list
>>>>> pve-devel at pve.proxmox.com
>>>>> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>>>>>
More information about the pve-devel
mailing list