[pve-devel] [PATCH pve-docs] add documentation for vxlan layer 2 network
Alexandre Derumier
aderumier at odiso.com
Fri Jul 13 17:50:57 CEST 2018
This is documentation for layer2 vxlan (without routing),
with multicast,anycast and bgp evpn
for vlan aware and vlan unware bridge
(Fixme : set svg width/height)
svg images break patch send by email (more than 998 character on 1 line),
they are available here:
http://odisoweb1.odiso.net:/vxlan-l2-vlanaware.svg
http://odisoweb1.odiso.net:/vxlan-l2-vlanunaware.svg
---
pve-network.adoc | 844 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 844 insertions(+)
diff --git a/pve-network.adoc b/pve-network.adoc
index 198f2d1..6cf11cf 100644
--- a/pve-network.adoc
+++ b/pve-network.adoc
@@ -468,6 +468,850 @@ iface vmbr0 inet manual
----
+
+VXLAN layer2 with vlan unware linux bridges
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+VXLAN is an overlay network to carry Ethernet traffic over an existing IP network
+while accommodating a very large number of tenants. It is defined in RFC 7348.
+Each overlay network is known as a VXLAN Segment and identified by a unique
+24-bit segment ID called a VXLAN Network Identifier (VNI).
+
+For BUM traffic (broadcast / unknown unicast traffic, multicast),
+we have 3 differents vxlan setup modes : multicast, unicast, bgp-evpn
+
+image::images/vxlan-l2-vlanunaware.svg["vxlan l2 bridge vlan unaware",align="center"]
+
+multicast mode
+^^^^^^^^^^^^^^
+
+This scenario relies in head end replication, meaning that end host in case
+of not having any entry for the destination MAC address will send out an ARP
+to other devices / VTEPs in the VXLAN network.
+This is done by sending the request to the VXLAN multicast group,
+remote VTEPs will get the packet and answer accordingly direct to the originating VTEP.
+
+
+* node1
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.1
+ netmask 255.255.255.0
+ bridge_ports eno1
+ bridge_stp off
+ bridge_fd 0
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan-svcnodeip 225.20.1.1
+ vxlan-physdev eno1
+
+auto vmbr2
+iface vmbr2 inet manual
+ bridge_ports vxlan2
+ bridge_stp off
+ bridge_fd 0
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan-svcnodeip 225.20.1.1
+ vxlan-physdev eno1
+
+auto vmbr3
+iface vmbr3 inet manual
+ bridge_ports vxlan3
+ bridge_stp off
+ bridge_fd 0
+----
+
+
+* node2
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.2
+ netmask 255.255.255.0
+ bridge_ports eno1
+ bridge_stp off
+ bridge_fd 0
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan-svcnodeip 225.20.1.1
+ vxlan-physdev eno1
+
+auto vmbr2
+iface vmbr2 inet manual
+ bridge_ports vxlan2
+ bridge_stp off
+ bridge_fd 0
+
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan-svcnodeip 225.20.1.1
+ vxlan-physdev eno1
+
+auto vmbr3
+iface vmbr3 inet manual
+ bridge_ports vxlan3
+ bridge_stp off
+ bridge_fd 0
+----
+
+
+* node3
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.3
+ netmask 255.255.255.0
+ bridge_ports eno1
+ bridge_stp off
+ bridge_fd 0
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan-svcnodeip 225.20.1.1
+ vxlan-physdev eno1
+
+auto vmbr2
+iface vmbr2 inet manual
+ bridge_ports vxlan2
+ bridge_stp off
+ bridge_fd 0
+
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan-svcnodeip 225.20.1.1
+ vxlan-physdev eno1
+
+auto vmbr3
+iface vmbr3 inet manual
+ bridge_ports vxlan3
+ bridge_stp off
+ bridge_fd 0
+----
+
+
+unicast mode
+^^^^^^^^^^^^
+
+We can replace multicast by head-end replication of BUM frames to a statically configured lists of remote VTEPs.
+The VXLAN is defined without a remote multicast group.
+Instead, all the remote VTEPs are associated with the all-zero address:
+a BUM frame will be duplicated to all these destinations.
+The VXLAN device will still learn remote addresses automatically using source-address learning.
+
+* node1
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.1
+ netmask 255.255.255.0
+ bridge_ports eno1
+ bridge_stp off
+ bridge_fd 0
+
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan_remoteip 192.168.0.2
+ vxlan_remoteip 192.168.0.3
+
+
+auto vmbr2
+iface vmbr2 inet manual
+ bridge_ports vxlan2
+ bridge_stp off
+ bridge_fd 0
+
+
+auto vxlan3
+iface vxlan2 inet manual
+ vxlan_remoteip 192.168.0.2
+ vxlan_remoteip 192.168.0.3
+
+
+auto vmbr3
+iface vmbr3 inet manual
+ bridge_ports vxlan3
+ bridge_stp off
+ bridge_fd 0
+----
+
+
+* node2
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.2
+ netmask 255.255.255.0
+ bridge_ports eno1
+ bridge_stp off
+ bridge_fd 0
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan_remoteip 192.168.0.1
+ vxlan_remoteip 192.168.0.3
+
+
+
+auto vmbr2
+iface vmbr2 inet manual
+ bridge_ports vxlan2
+ bridge_stp off
+ bridge_fd 0
+
+auto vxlan3
+iface vxlan2 inet manual
+ vxlan_remoteip 192.168.0.1
+ vxlan_remoteip 192.168.0.3
+
+
+auto vmbr3
+iface vmbr3 inet manual
+ bridge_ports vxlan3
+ bridge_stp off
+ bridge_fd 0
+----
+
+
+* node3
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.3
+ netmask 255.255.255.0
+ bridge_ports eno1
+ bridge_stp off
+ bridge_fd 0
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan_remoteip 192.168.0.2
+ vxlan_remoteip 192.168.0.3
+
+
+
+auto vmbr2
+iface vmbr2 inet manual
+ bridge_ports vxlan2
+ bridge_stp off
+ bridge_fd 0
+
+auto vxlan3
+iface vxlan2 inet manual
+ vxlan_remoteip 192.168.0.2
+ vxlan_remoteip 192.168.0.3
+
+
+auto vmbr3
+iface vmbr3 inet manual
+ bridge_ports vxlan3
+ bridge_stp off
+ bridge_fd 0
+----
+
+
+bgp-evpn
+^^^^^^^^
+
+VTEPs use control plane learning/distribution via BGP for remote MAC addresses instead of data plane learning.
+VTEPs have the ability to suppress ARP flooding over VXLAN tunnels.
+
+The control plane used here is FRR, a bgp routing software.
+Each node in the proxmox cluster peer with each others nodes.
+For bigger networks, or multiple proxmox clusters,
+it's possible to use external bgp route reflector servers.
+
+* node1
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.1
+ netmask 255.255.255.0
+ bridge_ports eno1
+ bridge_stp off
+ bridge_fd 0
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan-local-tunnelip 192.168.0.1
+ bridge-learning off
+ bridge-arp-nd-suppress on
+ bridge-unicast-flood off
+ bridge-multicast-flood off
+
+
+auto vmbr2
+iface vmbr2 inet manual
+ bridge_ports vxlan2
+ bridge_stp off
+ bridge_fd 0
+
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan-local-tunnelip 192.168.0.1
+ bridge-learning off
+ bridge-arp-nd-suppress on
+ bridge-unicast-flood off
+ bridge-multicast-flood off
+
+
+auto vmbr3
+iface vmbr3 inet manual
+ bridge_ports vxlan3
+ bridge_stp off
+ bridge_fd 0
+----
+
+
+/etc/frr/frr.conf
+
+----
+router bgp 1234
+ no bgp default ipv4-unicast
+ coalesce-time 1000
+ neighbor 192.168.0.2 remote-as 1234
+ neighbor 192.168.0.3 remote-as 1234
+ !
+ address-family l2vpn evpn
+ neighbor 192.168.0.2 activate
+ neighbor 192.168.0.3 activate
+ advertise-all-vni
+ exit-address-family
+!
+line vty
+!
+----
+
+
+* node2
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.2
+ netmask 255.255.255.0
+ bridge_ports eno1
+ bridge_stp off
+ bridge_fd 0
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan-local-tunnelip 192.168.0.2
+ bridge-learning off
+ bridge-arp-nd-suppress on
+ bridge-unicast-flood off
+ bridge-multicast-flood off
+
+
+auto vmbr2
+iface vmbr2 inet manual
+ bridge_ports vxlan2
+ bridge_stp off
+ bridge_fd 0
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan-local-tunnelip 192.168.0.2
+ bridge-learning off
+ bridge-arp-nd-suppress on
+ bridge-unicast-flood off
+ bridge-multicast-flood off
+
+
+auto vmbr3
+iface vmbr3 inet manual
+ bridge_ports vxlan3
+ bridge_stp off
+ bridge_fd 0
+----
+
+
+/etc/frr/frr.conf
+
+----
+router bgp 1234
+ no bgp default ipv4-unicast
+ coalesce-time 1000
+ neighbor 192.168.0.1 remote-as 1234
+ neighbor 192.168.0.3 remote-as 1234
+ !
+ address-family l2vpn evpn
+ neighbor 192.168.0.1 activate
+ neighbor 192.168.0.3 activate
+ advertise-all-vni
+ exit-address-family
+!
+line vty
+!
+----
+
+
+* node3
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.2
+ netmask 255.255.255.0
+ bridge_ports eno1
+ bridge_stp off
+ bridge_fd 0
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan-local-tunnelip 192.168.0.3
+ bridge-learning off
+ bridge-arp-nd-suppress on
+ bridge-unicast-flood off
+ bridge-multicast-flood off
+
+
+auto vmbr2
+iface vmbr2 inet manual
+ bridge_ports vxlan2
+ bridge_stp off
+ bridge_fd 0
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan-local-tunnelip 192.168.0.3
+ bridge-learning off
+ bridge-arp-nd-suppress on
+ bridge-unicast-flood off
+ bridge-multicast-flood off
+
+
+auto vmbr3
+iface vmbr3 inet manual
+ bridge_ports vxlan3
+ bridge_stp off
+ bridge_fd 0
+----
+
+
+/etc/frr/frr.conf
+
+
+----
+router bgp 1234
+ no bgp default ipv4-unicast
+ coalesce-time 1000
+ neighbor 192.168.0.1 remote-as 1234
+ neighbor 192.168.0.2 remote-as 1234
+ !
+ address-family l2vpn evpn
+ neighbor 192.168.0.1 activate
+ neighbor 192.168.0.2 activate
+ advertise-all-vni
+ exit-address-family
+!
+line vty
+!
+----
+
+
+VXLAN layer2 with vlan aware linux bridges
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+We use 1 vmbr bridge, each vxlan is mapped to a vlan
+
+image::images/vxlan-l2-vlanaware.svg["vxlan l2 bridge vlan aware",align="center"]
+
+multicast mode
+^^^^^^^^^^^^^^
+
+* node1
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.1
+ netmask 255.255.255.0
+ bridge_ports eno1 vxlan2 vxlan3
+ bridge_stp off
+ bridge_fd 0
+ bridge_vlan_aware yes
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan-svcnodeip 225.20.1.1
+ vxlan-physdev eno1
+ bridge-access 2
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan-svcnodeip 225.20.1.1
+ vxlan-physdev eno1
+ bridge-access 3
+----
+
+
+* node2
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.2
+ netmask 255.255.255.0
+ bridge_ports eno1 vxlan2 vxlan3
+ bridge_stp off
+ bridge_fd 0
+ bridge_vlan_aware yes
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan-svcnodeip 225.20.1.1
+ vxlan-physdev eno1
+ bridge-access 2
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan-svcnodeip 225.20.1.1
+ vxlan-physdev eno1
+ bridge-access 3
+----
+
+
+* node3
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.3
+ netmask 255.255.255.0
+ bridge_ports eno1 vxlan2 vxlan3
+ bridge_stp off
+ bridge_fd 0
+ bridge_vlan_aware yes
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan-svcnodeip 225.20.1.1
+ vxlan-physdev eno1
+ bridge-access 2
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan-svcnodeip 225.20.1.1
+ vxlan-physdev eno1
+ bridge-access 3
+----
+
+
+unicast mode
+^^^^^^^^^^^^
+
+* node1
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.1
+ netmask 255.255.255.0
+ bridge_ports eno1 vxlan2 vxlan3
+ bridge_stp off
+ bridge_fd 0
+ bridge_vlan_aware yes
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan_remoteip 192.168.0.2
+ vxlan_remoteip 192.168.0.3
+ bridge-access 2
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan_remoteip 192.168.0.2
+ vxlan_remoteip 192.168.0.3
+ bridge-access 3
+----
+
+
+* node2
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.2
+ netmask 255.255.255.0
+ bridge_ports eno1 vxlan2 vxlan3
+ bridge_stp off
+ bridge_fd 0
+ bridge_vlan_aware yes
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan_remoteip 192.168.0.1
+ vxlan_remoteip 192.168.0.3
+ bridge-access 2
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan_remoteip 192.168.0.1
+ vxlan_remoteip 192.168.0.3
+ bridge-access 3
+----
+
+
+* node3
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.3
+ netmask 255.255.255.0
+ bridge_ports eno1 vxlan2 vxlan3
+ bridge_stp off
+ bridge_fd 0
+ bridge_vlan_aware yes
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan_remoteip 192.168.0.2
+ vxlan_remoteip 192.168.0.3
+ bridge-access 2
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan_remoteip 192.168.0.2
+ vxlan_remoteip 192.168.0.3
+ bridge-access 3
+----
+
+
+bgp-evpn
+^^^^^^^^
+
+Note: currently FRR is working only with 1 vlan aware bridge
+
+* node1
+
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.1
+ netmask 255.255.255.0
+ bridge_ports eno1 vxlan2 vxlan3
+ bridge_stp off
+ bridge_fd 0
+ bridge_vlan_aware yes
+
+auto vxlan0
+iface vxlan0 inet manual
+ vxlan-local-tunnelip 192.168.0.1
+ bridge-learning off
+ bridge-arp-nd-suppress on
+ bridge-unicast-flood off
+ bridge-multicast-flood off
+ bridge-access 2
+
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan-local-tunnelip 192.168.0.1
+ bridge-learning off
+ bridge-arp-nd-suppress on
+ bridge-unicast-flood off
+ bridge-multicast-flood off
+ bridge-access 3
+----
+
+
+/etc/frr/frr.conf
+
+----
+router bgp 1234
+ no bgp default ipv4-unicast
+ coalesce-time 1000
+ neighbor 192.168.0.2 remote-as 1234
+ neighbor 192.168.0.3 remote-as 1234
+ !
+ address-family l2vpn evpn
+ neighbor 192.168.0.2 activate
+ neighbor 192.168.0.3 activate
+ advertise-all-vni
+ exit-address-family
+!
+line vty
+!
+----
+
+
+* node2
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.2
+ netmask 255.255.255.0
+ bridge_ports eno1 vxlan2 vxlan3
+ bridge_stp off
+ bridge_fd 0
+ bridge_vlan_aware yes
+
+auto vxlan0
+iface vxlan0 inet manual
+ vxlan-local-tunnelip 192.168.0.2
+ bridge-learning off
+ bridge-arp-nd-suppress on
+ bridge-unicast-flood off
+ bridge-multicast-flood off
+ bridge-access 2
+
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan-local-tunnelip 192.168.0.2
+ bridge-learning off
+ bridge-arp-nd-suppress on
+ bridge-unicast-flood off
+ bridge-multicast-flood off
+ bridge-access 3
+----
+
+
+/etc/frr/frr.conf
+
+----
+router bgp 1234
+ no bgp default ipv4-unicast
+ coalesce-time 1000
+ neighbor 192.168.0.1 remote-as 1234
+ neighbor 192.168.0.3 remote-as 1234
+ !
+ address-family l2vpn evpn
+ neighbor 192.168.0.1 activate
+ neighbor 192.168.0.3 activate
+ advertise-all-vni
+ exit-address-family
+!
+line vty
+!
+----
+
+
+* node3
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.3
+ netmask 255.255.255.0
+ bridge_ports eno1 vxlan2 vxlan3
+ bridge_stp off
+ bridge_fd 0
+ bridge_vlan_aware yes
+
+auto vxlan0
+iface vxlan0 inet manual
+ vxlan-local-tunnelip 192.168.0.3
+ bridge-learning off
+ bridge-arp-nd-suppress on
+ bridge-unicast-flood off
+ bridge-multicast-flood off
+ bridge-access 2
+
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan-local-tunnelip 192.168.0.3
+ bridge-learning off
+ bridge-arp-nd-suppress on
+ bridge-unicast-flood off
+ bridge-multicast-flood off
+ bridge-access 3
+----
+
+
+/etc/frr/frr.conf
+----
+router bgp 1234
+ no bgp default ipv4-unicast
+ coalesce-time 1000
+ neighbor 192.168.0.1 remote-as 1234
+ neighbor 192.168.0.2 remote-as 1234
+ !
+ address-family l2vpn evpn
+ neighbor 192.168.0.1 activate
+ neighbor 192.168.0.2 activate
+ advertise-all-vni
+ exit-address-family
+!
+line vty
+!
+----
+
+
////
TODO: explain IPv6 support?
TODO: explain OVS
--
2.11.0
More information about the pve-devel
mailing list