[PVE-User] 802.3ad Bonding Proxmox 1.4

[Jeff Saxe]

I'm not sure why 943Mb/sec "sucks" -- personally, I'd be pretty  
pleased to get such bandwidth, if I asked for a virtual machine and  
you hosted it for me on your Proxmox server.  :-)

But seriously, you may not be taking into account precisely what  
happens when you use a layer 2 Ethernet aggregate (EtherChannel, or  
port channel). The accepted standards of layer 2 say that frames  
should not arrive out of order from how they were transmitted, so the  
way a port-channeling device treats each frame is to run it through  
some quick hash algorithm (based on either source or destination MAC,  
IP, or layer 4 port numbers, or some combination), and whatever the  
hash comes up with, it sends the frame on that link out of the bundle.  
The result of this is that a single long-running conversation between  
two endpoints (for instance, one long FTP transfer) is always going to  
choose the same Ethernet port over and over for each frame, so even if  
you bond together 5 gig Ethernets, one file transfer is going to go  
through only one of the five. So a speed of 943Mb/sec is not  
surprising -- likely, you are nearly saturating just one gig port  
while the others remain idle.

An Ethernet port channel gives you good redundancy, fast failover,  
easy expansion, no need to use a routing protocol, no need to think  
hard about spanning tree, safety from accidentally plugging into wrong  
ports (when you use 802.3ad protocol), etc. But it does not  
automatically give you high bandwidth for focused transmissions. It  
only gives you high average bandwidth in the larger case, where you  
have many hosts (or several IP addresses on the same hosts, or many  
TCP conversations, again depending on the frame distribution algorithm  
in use on each side of the aggregate). Sorry if that messes up your  
plans.

-- Jeff Saxe, Network Engineer
Blue Ridge InternetWorks, Charlottesville, VA
434-817-0707 ext. 2024  /  JSaxe at briworks.com




On Nov 11, 2009, at 5:13 PM, Andrew Niemantsverdriet wrote:

> I just went in and enabled STP the bridge is now and able to
> communicate. It is slow though. Still can't see more than 943Mbits/sec
> through the bond0 interface.
>
> # network interface settings
> auto lo
> iface lo inet loopback
>
> iface eth0 inet manual
>
> iface eth1 inet manual
>
> iface eth2 inet manual
>
> iface eth3 inet manual
>
> iface eth4 inet manual
>
> auto eth5
> iface eth5 inet static
> 	address  192.168.3.4
> 	netmask  255.255.255.0
>
> auto bond0
> iface bond0 inet manual
> 	slaves eth0 eth1 eth2 eth3 eth4
> 	bond_miimon 100
> 	bond_mode 802.3ad
>
> auto vmbr0
> iface vmbr0 inet static
> 	address  192.0.2.6
> 	netmask  255.255.255.0
> 	gateway 192.0.2.1
> 	bridge_ports bond0
> 	bridge_stp on
> 	bridge_fd 0
>
> The switch shows 802.3ad partners so that is working however the speed
> sucks although that is better than not working.
>
> Any ideas?
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.proxmox.com/pipermail/pve-user/attachments/20091111/99234ae5/attachment.htm>