[PVE-User] bond0 Issues

[Jeff Saxe]

Ah, I just looked into it a bit more. 802.3ad is dynamic link  
aggregation, and in absence of a vendor-proprietary protocol like  
Cisco's PAgP, this pretty much has to be the standard Link Aggregation  
Control Protocol, LACP. A data sheet In found on the ProCurve 4200vl  
switch says it does support that method. So go ahead and make your  
bond0 interface on Proxmox use 802.3ad, and on the switch set the four  
ports into one channel group or LAG group or something, and turn LACP  
to "active" mode on those four ports -- this means the switch is  
actively seeking to form a group, and the Linux server on the other  
side just needs to respond affirmatively to the special LACP frames.  
Assuming it works, this will be safer, and you'll be able to monitor  
it from the switch side -- in the GUI or CLI or even SNMP queries, you  
can see which links are part of the bundle, how long they have been in  
the bundle, etc. And it does provide distribution of frames across the  
links, in a manner very similar to the "balance-xor" mode.

http://www.mjmwired.net/kernel/Documentation/networking/bonding.txt

I'm afraid I have never configured a ProCurve switch, despite my  
networking engineering experience, so I can't follow through on my  
promise to help you. Sorry! Look in the menus for options about  
"channel bonding", "LAG groups", "LACP active mode", etc.

-- JeffS


On Jun 12, 2009, at 10:34 AM, Andrew Niemantsverdriet wrote:

> Jeff,
>
> It is my understanding that you DO NOT need special switch
> configuration for using balance-rr, balance-xor, active-backup,
> balance-tlb or balance-alb. You DO need special switch configuration
> for 802.3ad and broadcast. Am I wrong? I could very well be just
> started looking into bonding 2 days ago when I migrated a server and
> caused issues.
>
> The switch I am using is a HP Procurve 4200vl
>
> Thanks,
> _
> /-\ ndrew
>
> On Fri, Jun 12, 2009 at 6:22 AM, Jeff Saxe<jsaxe at briworks.com> wrote:
>> I think the more pressing question is: Did you configure the  
>> Ethernet switch on the other side of your four bonded NIC cables to  
>> create a link aggregate, i.e., "port channel" or "EtherChannel"? I  
>> can certainly see why one would bond 4 NICs together -- it gives 4  
>> times the bandwidth, as long as a lot of conversations are  
>> happening over it so that the layer 2 load-balancing rules can take  
>> effect. And you can throw them all in the same bridge, and all the  
>> IP addresses just magically float between all four links, and the  
>> failover time when one link fails (i.e., one switch module in a  
>> multi-module switch fails) is extremely fast, about a second or so.  
>> But the devices on both sides of the aggregate have to agree that  
>> it is going to happen, otherwise bridge frame-flooding behavior and  
>> Spanning Tree protocols will get very confused!
>>
>> I have not tried the NIC bonding in Proxmox yet (the test servers I  
>> am using happen to have only one NIC), but I know how it works in  
>> general. If your switch defaults to using a link aggregation  
>> protocol, like LACP or PAgP, you will have to turn it off for these  
>> four ports, since I don't think the Debian Linux side of it will  
>> perform LACP. Just set those four ports on the Ethernet switch to  
>> the same channel group number and to "channeling mode ON", i.e.,  
>> always on, not negotiating whether to channel or not. And be sure  
>> not to accidentally plug in any other servers into any of those  
>> four ports later until you take them out of the aggregate, or again  
>> you will knock one or both servers off the network... mode "ON" is  
>> easier, but LACP is safer, since it verifies that the devices on  
>> both sides of the cables agree on who's plugged into the channeling  
>> ports before forming the aggregate.
>>
>> Good luck, Andrew. If you tell me what model of switch you have, I  
>> can probably hack up the config statements for you.
>>
>> -- Jeff Saxe
>> Network Engineer, Blue Ridge InternetWorks
>> Charlottesville, Virginia
>>