<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div>I'm not sure why 943Mb/sec "sucks" -- personally, I'd be pretty pleased to get such bandwidth, if I asked for a virtual machine and you hosted it for me on your Proxmox server. :-)</div><div><br></div><div>But seriously, you may not be taking into account precisely what happens when you use a layer 2 Ethernet aggregate (EtherChannel, or port channel). The accepted standards of layer 2 say that frames should not arrive out of order from how they were transmitted, so the way a port-channeling device treats each frame is to run it through some quick hash algorithm (based on either source or destination MAC, IP, or layer 4 port numbers, or some combination), and whatever the hash comes up with, it sends the frame on that link out of the bundle. The result of this is that <i>a single long-running conversation</i> between two endpoints (for instance, one long FTP transfer) is always going to choose the same Ethernet port over and over for each frame, so even if you bond together 5 gig Ethernets, one file transfer is going to go through only one of the five. So a speed of 943Mb/sec is not surprising -- likely, you are nearly saturating just one gig port while the others remain idle.</div><div><br></div><div>An Ethernet port channel gives you good redundancy, fast failover, easy expansion, no need to use a routing protocol, no need to think hard about spanning tree, safety from accidentally plugging into wrong ports (when you use 802.3ad protocol), etc. But it does not automatically give you high bandwidth for focused transmissions. It only gives you high average bandwidth in the larger case, where you have many hosts (or several IP addresses on the same hosts, or many TCP conversations, again depending on the frame distribution algorithm in use on each side of the aggregate). Sorry if that messes up your plans.</div><div apple-content-edited="true"> <span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div><br class="Apple-interchange-newline">-- Jeff Saxe, Network Engineer</div><div>Blue Ridge InternetWorks, Charlottesville, VA</div><div>434-817-0707 ext. 2024 / <a href="mailto:JSaxe@briworks.com">JSaxe@briworks.com</a></div><div><br></div></span></span></span></span></div></span></div></span></div></span></div></span><br class="Apple-interchange-newline"></div></span><br class="Apple-interchange-newline"> </div><br><div><div>On Nov 11, 2009, at 5:13 PM, Andrew Niemantsverdriet wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>I just went in and enabled STP the bridge is now and able to<br>communicate. It is slow though. Still can't see more than 943Mbits/sec<br>through the bond0 interface.<br><br># network interface settings<br>auto lo<br>iface lo inet loopback<br><br>iface eth0 inet manual<br><br>iface eth1 inet manual<br><br>iface eth2 inet manual<br><br>iface eth3 inet manual<br><br>iface eth4 inet manual<br><br>auto eth5<br>iface eth5 inet static<br><span class="Apple-tab-span" style="white-space:pre"> </span>address 192.168.3.4<br><span class="Apple-tab-span" style="white-space:pre"> </span>netmask 255.255.255.0<br><br>auto bond0<br>iface bond0 inet manual<br><span class="Apple-tab-span" style="white-space:pre"> </span>slaves eth0 eth1 eth2 eth3 eth4<br><span class="Apple-tab-span" style="white-space:pre"> </span>bond_miimon 100<br><span class="Apple-tab-span" style="white-space:pre"> </span>bond_mode 802.3ad<br><br>auto vmbr0<br>iface vmbr0 inet static<br><span class="Apple-tab-span" style="white-space:pre"> </span>address 192.0.2.6<br><span class="Apple-tab-span" style="white-space:pre"> </span>netmask 255.255.255.0<br><span class="Apple-tab-span" style="white-space:pre"> </span>gateway 192.0.2.1<br><span class="Apple-tab-span" style="white-space:pre"> </span>bridge_ports bond0<br><span class="Apple-tab-span" style="white-space:pre"> </span>bridge_stp on<br><span class="Apple-tab-span" style="white-space:pre"> </span>bridge_fd 0<br><br>The switch shows 802.3ad partners so that is working however the speed<br>sucks although that is better than not working.<br><br>Any ideas?<br><br></div></blockquote></div><br></body></html>