[PVE-User] bond0 Issues
jsaxe at briworks.com
Fri Jun 12 17:24:04 CEST 2009
Ah, I just looked into it a bit more. 802.3ad is dynamic link
aggregation, and in absence of a vendor-proprietary protocol like
Cisco's PAgP, this pretty much has to be the standard Link Aggregation
Control Protocol, LACP. A data sheet In found on the ProCurve 4200vl
switch says it does support that method. So go ahead and make your
bond0 interface on Proxmox use 802.3ad, and on the switch set the four
ports into one channel group or LAG group or something, and turn LACP
to "active" mode on those four ports -- this means the switch is
actively seeking to form a group, and the Linux server on the other
side just needs to respond affirmatively to the special LACP frames.
Assuming it works, this will be safer, and you'll be able to monitor
it from the switch side -- in the GUI or CLI or even SNMP queries, you
can see which links are part of the bundle, how long they have been in
the bundle, etc. And it does provide distribution of frames across the
links, in a manner very similar to the "balance-xor" mode.
I'm afraid I have never configured a ProCurve switch, despite my
networking engineering experience, so I can't follow through on my
promise to help you. Sorry! Look in the menus for options about
"channel bonding", "LAG groups", "LACP active mode", etc.
On Jun 12, 2009, at 10:34 AM, Andrew Niemantsverdriet wrote:
> It is my understanding that you DO NOT need special switch
> configuration for using balance-rr, balance-xor, active-backup,
> balance-tlb or balance-alb. You DO need special switch configuration
> for 802.3ad and broadcast. Am I wrong? I could very well be just
> started looking into bonding 2 days ago when I migrated a server and
> caused issues.
> The switch I am using is a HP Procurve 4200vl
> /-\ ndrew
> On Fri, Jun 12, 2009 at 6:22 AM, Jeff Saxe<jsaxe at briworks.com> wrote:
>> I think the more pressing question is: Did you configure the
>> Ethernet switch on the other side of your four bonded NIC cables to
>> create a link aggregate, i.e., "port channel" or "EtherChannel"? I
>> can certainly see why one would bond 4 NICs together -- it gives 4
>> times the bandwidth, as long as a lot of conversations are
>> happening over it so that the layer 2 load-balancing rules can take
>> effect. And you can throw them all in the same bridge, and all the
>> IP addresses just magically float between all four links, and the
>> failover time when one link fails (i.e., one switch module in a
>> multi-module switch fails) is extremely fast, about a second or so.
>> But the devices on both sides of the aggregate have to agree that
>> it is going to happen, otherwise bridge frame-flooding behavior and
>> Spanning Tree protocols will get very confused!
>> I have not tried the NIC bonding in Proxmox yet (the test servers I
>> am using happen to have only one NIC), but I know how it works in
>> general. If your switch defaults to using a link aggregation
>> protocol, like LACP or PAgP, you will have to turn it off for these
>> four ports, since I don't think the Debian Linux side of it will
>> perform LACP. Just set those four ports on the Ethernet switch to
>> the same channel group number and to "channeling mode ON", i.e.,
>> always on, not negotiating whether to channel or not. And be sure
>> not to accidentally plug in any other servers into any of those
>> four ports later until you take them out of the aggregate, or again
>> you will knock one or both servers off the network... mode "ON" is
>> easier, but LACP is safer, since it verifies that the devices on
>> both sides of the cables agree on who's plugged into the channeling
>> ports before forming the aggregate.
>> Good luck, Andrew. If you tell me what model of switch you have, I
>> can probably hack up the config statements for you.
>> -- Jeff Saxe
>> Network Engineer, Blue Ridge InternetWorks
>> Charlottesville, Virginia
More information about the pve-user