[PVE-User] bond0 Issues

[Jeff Saxe]

I think the more pressing question is: Did you configure the Ethernet switch on the other side of your four bonded NIC cables to create a link aggregate, i.e., "port channel" or "EtherChannel"? I can certainly see why one would bond 4 NICs together -- it gives 4 times the bandwidth, as long as a lot of conversations are happening over it so that the layer 2 load-balancing rules can take effect. And you can throw them all in the same bridge, and all the IP addresses just magically float between all four links, and the failover time when one link fails (i.e., one switch module in a multi-module switch fails) is extremely fast, about a second or so. But the devices on both sides of the aggregate have to agree that it is going to happen, otherwise bridge frame-flooding behavior and Spanning Tree protocols will get very confused!

I have not tried the NIC bonding in Proxmox yet (the test servers I am using happen to have only one NIC), but I know how it works in general. If your switch defaults to using a link aggregation protocol, like LACP or PAgP, you will have to turn it off for these four ports, since I don't think the Debian Linux side of it will perform LACP. Just set those four ports on the Ethernet switch to the same channel group number and to "channeling mode ON", i.e., always on, not negotiating whether to channel or not. And be sure not to accidentally plug in any other servers into any of those four ports later until you take them out of the aggregate, or again you will knock one or both servers off the network... mode "ON" is easier, but LACP is safer, since it verifies that the devices on both sides of the cables agree on who's plugged into the channeling ports before forming the aggregate.

Good luck, Andrew. If you tell me what model of switch you have, I can probably hack up the config statements for you.

-- Jeff Saxe
Network Engineer, Blue Ridge InternetWorks
Charlottesville, Virginia



-----Original Message-----
From: pve-user-bounces at pve.proxmox.com on behalf of Dietmar Maurer
Sent: Fri 6/12/2009 6:42 AM
To: Andrew Niemantsverdriet; proxmoxve
Subject: Re: [PVE-User] bond0 Issues
 
Do you use AoE?


Besides, I do not think it makes sense to use 4 nics in one bond.

> -----Original Message-----
> From: pve-user-bounces at pve.proxmox.com [mailto:pve-user-
> bounces at pve.proxmox.com] On Behalf Of Andrew Niemantsverdriet
> Sent: Donnerstag, 11. Juni 2009 21:20
> To: proxmoxve
> Subject: [PVE-User] bond0 Issues
> 
> I have setup a bond0 interface containing eth0 eth1 eth2 eth3. I setup
> the bond interface with balance-rr and then set vmbr0 to contain
> bond0. When I rebooted the machine for the change to take effect I saw
> this scroll across the console:
> 
> bond0 received packet with own address as source address
> 
> It also caused the network to be very unstable with 70% packet loss.
> 

_______________________________________________
pve-user mailing list
pve-user at pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user