[pve-devel] [PATCH v2 pve-docs 6/6] sdn: Examples
Stefan Lendl
s.lendl at proxmox.com
Fri Nov 17 14:55:31 CET 2023
Signed-off-by: Stefan Lendl <s.lendl at proxmox.com>
---
pvesdn.adoc | 486 +++++++++++++++++++---------------------------------
1 file changed, 180 insertions(+), 306 deletions(-)
diff --git a/pvesdn.adoc b/pvesdn.adoc
index 450955d..17135cc 100644
--- a/pvesdn.adoc
+++ b/pvesdn.adoc
@@ -508,74 +508,96 @@ key:: An API access key
ttl:: The default TTL for records
+[[pvesdn_setup_examples]]
Examples
---------
+-------------
-[[pvesdn_setup_example_vlan]]
-VLAN Setup Example
-~~~~~~~~~~~~~~~~~~
+This section presents multiple configuration examples tailored for common SDN
+use cases. It aims to offer tangible implementations, providing additional
+details to enhance comprehension of the available configuration options.
-TIP: While we show plaintext configuration content here, almost everything
-should be configurable using the web-interface only.
-Node1: /etc/network/interfaces
+[[pvesdn_setup_example_simple]]
+Simple Zone Example
+~~~~~~~~~~~~~
-----
-auto vmbr0
-iface vmbr0 inet manual
- bridge-ports eno1
- bridge-stp off
- bridge-fd 0
- bridge-vlan-aware yes
- bridge-vids 2-4094
+Simple zone networks create an isolated network for quests on a single host to
+connect to each other.
-#management ip on vlan100
-auto vmbr0.100
-iface vmbr0.100 inet static
- address 192.168.0.1/24
+TIP: connection between quests are possible if all quests reside on a same host
+but cannot be reached on other nodes.
-source /etc/network/interfaces.d/*
-----
+* Create a simple zone named `simple`.
+* Add a VNet names `vnet1`.
+* Create a Subnet with a gateway and the SNAT option enabled.
+* This creates a network bridge `vnet1` on the node. Assign this bridge to the
+ quests that shall join the network and configure an IP address.
-Node2: /etc/network/interfaces
+The network interface configuration in two VMs may look like this which allows
+them to communicate via the 10.0.1.0/24 network.
----
-auto vmbr0
-iface vmbr0 inet manual
- bridge-ports eno1
- bridge-stp off
- bridge-fd 0
- bridge-vlan-aware yes
- bridge-vids 2-4094
+allow-hotplug ens19
+iface ens19 inet static
+ address 10.0.1.14/24
+----
-#management ip on vlan100
-auto vmbr0.100
-iface vmbr0.100 inet static
- address 192.168.0.2/24
+----
+allow-hotplug ens19
+iface ens19 inet static
+ address 10.0.1.15/24
+----
-source /etc/network/interfaces.d/*
+
+[[pvesdn_setup_example_nat]]
+Source NAT Example
+~~~~~~~~~~~~~
+
+If you want to allow outgoing connections for quests in the simple network zone
+the simple zone offers a Source NAT (SNAT) option.
+
+Starting from the configuration xref:pvesdn_setup_example_simple[above], Add a
+Subnet to the VNet `vnet1`, set a gateway IP and enable the SNAT option.
+
+----
+Subnet: 172.16.0.0/24
+Gateway: 172.16.0.1
+SNAT: checked
----
-Create a VLAN zone named `myvlanzone':
+In the quests configure the static IP address inside the subnet's IP range.
+
+The node itself will join this network with the Gateway IP '172.16.0.1' and
+function as the NAT gateway for quests within the subnet range.
+
+
+[[pvesdn_setup_example_vlan]]
+VLAN Setup Example
+~~~~~~~~~~~~~
+
+When VMs on different nodes need to communicate through an isolated network, the
+VLAN zone allows network level isolation using VLAN tags.
+
+Create a VLAN zone named `myvlanzone`:
----
-id: myvlanzone
-bridge: vmbr0
+ID: myvlanzone
+Bridge: vmbr0
----
-Create a VNet named `myvnet1' with `vlan-id` `10' and the previously created
-`myvlanzone' as its zone.
+Create a VNet named `myvnet1` with VLAN tag 10 and the previously created
+`myvlanzone`.
----
-id: myvnet1
-zone: myvlanzone
-tag: 10
+ID: myvnet1
+Zone: myvlanzone
+Tag: 10
----
Apply the configuration through the main SDN panel, to create VNets locally on
each node.
-Create a Debian-based virtual machine (vm1) on node1, with a vNIC on `myvnet1'.
+Create a Debian-based virtual machine ('vm1') on node1, with a vNIC on `myvnet1`.
Use the following network configuration for this VM:
@@ -585,8 +607,8 @@ iface eth0 inet static
address 10.0.3.100/24
----
-Create a second virtual machine (vm2) on node2, with a vNIC on the same VNet
-`myvnet1' as vm1.
+Create a second virtual machine ('vm2') on node2, with a vNIC on the same VNet
+`myvnet1` as vm1.
Use the following network configuration for this VM:
@@ -596,234 +618,124 @@ iface eth0 inet static
address 10.0.3.101/24
----
-Following this, you should be able to ping between both VMs over that network.
+Following this, you should be able to ping between both VMs using that network.
[[pvesdn_setup_example_qinq]]
QinQ Setup Example
-~~~~~~~~~~~~~~~~~~
-
-TIP: While we show plaintext configuration content here, almost everything
-should be configurable using the web-interface only.
-
-Node1: /etc/network/interfaces
-
-----
-auto vmbr0
-iface vmbr0 inet manual
- bridge-ports eno1
- bridge-stp off
- bridge-fd 0
- bridge-vlan-aware yes
- bridge-vids 2-4094
-
-#management ip on vlan100
-auto vmbr0.100
-iface vmbr0.100 inet static
- address 192.168.0.1/24
+~~~~~~~~~~~~~
-source /etc/network/interfaces.d/*
-----
-Node2: /etc/network/interfaces
+This example configures two QinQ zones and adds two VMs to each zone to
+demonstrate the additional layer of VLAN tags which allows the configuration of
+more isolated VLANs.
-----
-auto vmbr0
-iface vmbr0 inet manual
- bridge-ports eno1
- bridge-stp off
- bridge-fd 0
- bridge-vlan-aware yes
- bridge-vids 2-4094
+A typical use case for this configuration is a hosting provider that provides an
+isolated network to customers for VM communication but isolates the VMs from
+other customers.
-#management ip on vlan100
-auto vmbr0.100
-iface vmbr0.100 inet static
- address 192.168.0.2/24
+Create a QinQ zone named `qinqzone1` with service VLAN 20
-source /etc/network/interfaces.d/*
----
-
-Create a QinQ zone named `qinqzone1' with service VLAN 20
-
+ID: qinqzone1
+Bridge: vmbr0
+Service VLAN: 20
----
-id: qinqzone1
-bridge: vmbr0
-service vlan: 20
-----
-
-Create another QinQ zone named `qinqzone2' with service VLAN 30
+Create another QinQ zone named `qinqzone2` with service VLAN 30
----
-id: qinqzone2
-bridge: vmbr0
-service vlan: 30
+ID: qinqzone2
+Bridge: vmbr0
+Service VLAN: 30
----
-Create a VNet named `myvnet1' with customer VLAN-ID 100 on the previously
-created `qinqzone1' zone.
+Create a VNet named `myvnet1` with VLAN-ID 100 on the previously created
+`qinqzone1` zone.
----
-id: myvnet1
-zone: qinqzone1
-tag: 100
+ID: qinqvnet1
+Zone: qinqzone1
+Tag: 100
----
-Create a `myvnet2' with customer VLAN-ID 100 on the previously created
-`qinqzone2' zone.
+Create a `myvnet2` with VLAN-ID 100 on the `qinqzone2` zone.
----
-id: myvnet2
-zone: qinqzone2
-tag: 100
+ID: qinqvnet2
+Zone: qinqzone2
+Tag: 100
----
Apply the configuration on the main SDN web-interface panel to create VNets
-locally on each nodes.
-
-Create a Debian-based virtual machine (vm1) on node1, with a vNIC on `myvnet1'.
-
-Use the following network configuration for this VM:
-
-----
-auto eth0
-iface eth0 inet static
- address 10.0.3.100/24
-----
-
-Create a second virtual machine (vm2) on node2, with a vNIC on the same VNet
-`myvnet1' as vm1.
-
-Use the following network configuration for this VM:
-
-----
-auto eth0
-iface eth0 inet static
- address 10.0.3.101/24
-----
-
-Create a third virtual machine (vm3) on node1, with a vNIC on the other VNet
-`myvnet2'.
-
-Use the following network configuration for this VM:
-
-----
-auto eth0
-iface eth0 inet static
- address 10.0.3.102/24
-----
+locally on each node.
-Create another virtual machine (vm4) on node2, with a vNIC on the same VNet
-`myvnet2' as vm3.
+Create four Debian-bases virtual machines (vm1, vm2, vm3, vm4) and add network
+interfaces to vm1 and vm2 with bridge `qinqvnet1` and vm3 and vm4 with bridge
+`qinqvnet2`.
-Use the following network configuration for this VM:
+Inside the VM, configure the IP addresses of the interfaces, for example via
+`/etc/network/interfaces`:
----
auto eth0
iface eth0 inet static
- address 10.0.3.103/24
+ address 10.0.3.101/24
----
+Configure the four VMs to have IP addresses from 10.0.3.101 to 10.0.3.104.
-Then, you should be able to ping between the VMs 'vm1' and 'vm2', as well as
+Now you should be able to ping between the VMs 'vm1' and 'vm2', as well as
between 'vm3' and 'vm4'. However, neither of VMs 'vm1' or 'vm2' can ping VMs
-'vm3' or 'vm4', as they are on a different zone with a different service-vlan.
+'vm3' or 'vm4', as they are on a different zone with a different service-VLAN.
[[pvesdn_setup_example_vxlan]]
VXLAN Setup Example
-~~~~~~~~~~~~~~~~~~~
-
-TIP: While we show plaintext configuration content here, almost everything
-is configurable through the web-interface.
-
-node1: /etc/network/interfaces
-
-----
-auto vmbr0
-iface vmbr0 inet static
- address 192.168.0.1/24
- gateway 192.168.0.254
- bridge-ports eno1
- bridge-stp off
- bridge-fd 0
- mtu 1500
-
-source /etc/network/interfaces.d/*
-----
-
-node2: /etc/network/interfaces
-
-----
-auto vmbr0
-iface vmbr0 inet static
- address 192.168.0.2/24
- gateway 192.168.0.254
- bridge-ports eno1
- bridge-stp off
- bridge-fd 0
- mtu 1500
-
-source /etc/network/interfaces.d/*
-----
-
-node3: /etc/network/interfaces
+~~~~~~~~~~~~~
-----
-auto vmbr0
-iface vmbr0 inet static
- address 192.168.0.3/24
- gateway 192.168.0.254
- bridge-ports eno1
- bridge-stp off
- bridge-fd 0
- mtu 1500
+The example assumes a cluster with three nodes, with the node IP addresses
+192.168.0.1, 192.168.0.2 and 192.168.0.3.
-source /etc/network/interfaces.d/*
-----
-
-Create a VXLAN zone named `myvxlanzone', using a lower MTU to ensure the extra
-50 bytes of the VXLAN header can fit. Add all previously configured IPs from
-the nodes to the peer address list.
+Create a VXLAN zone named `myvxlanzone` and add all IPs from the nodes to the
+peer address list. Use the default MTU of 1450 or configure accordingly.
----
-id: myvxlanzone
-peers address list: 192.168.0.1,192.168.0.2,192.168.0.3
-mtu: 1450
+ID: myvxlanzone
+Peers Address List: 192.168.0.1,192.168.0.2,192.168.0.3
----
-Create a VNet named `myvnet1' using the VXLAN zone `myvxlanzone' created
+Create a VNet named `vxvnet1` using the VXLAN zone `myvxlanzone` created
previously.
----
-id: myvnet1
-zone: myvxlanzone
-tag: 100000
+ID: vxvnet1
+Zone: myvxlanzone
+Tag: 100000
----
Apply the configuration on the main SDN web-interface panel to create VNets
locally on each nodes.
-Create a Debian-based virtual machine (vm1) on node1, with a vNIC on `myvnet1'.
+Create a Debian-based virtual machine ('vm1') on node1, with a vNIC on `vxvnet1`.
Use the following network configuration for this VM (note the lower MTU).
----
auto eth0
iface eth0 inet static
- address 10.0.3.100/24
- mtu 1450
+ address 10.0.3.100/24
+ mtu 1450
----
-Create a second virtual machine (vm2) on node3, with a vNIC on the same VNet
-`myvnet1' as vm1.
+Create a second virtual machine ('vm2') on node3, with a vNIC on the same VNet
+`vxvnet1` as vm1.
Use the following network configuration for this VM:
----
auto eth0
iface eth0 inet static
- address 10.0.3.101/24
- mtu 1450
+ address 10.0.3.101/24
+ mtu 1450
----
Then, you should be able to ping between between 'vm1' and 'vm2'.
@@ -831,137 +743,94 @@ Then, you should be able to ping between between 'vm1' and 'vm2'.
[[pvesdn_setup_example_evpn]]
EVPN Setup Example
-~~~~~~~~~~~~~~~~~~
-
-node1: /etc/network/interfaces
-
-----
-auto vmbr0
-iface vmbr0 inet static
- address 192.168.0.1/24
- gateway 192.168.0.254
- bridge-ports eno1
- bridge-stp off
- bridge-fd 0
- mtu 1500
-
-source /etc/network/interfaces.d/*
-----
-
-node2: /etc/network/interfaces
-
-----
-auto vmbr0
-iface vmbr0 inet static
- address 192.168.0.2/24
- gateway 192.168.0.254
- bridge-ports eno1
- bridge-stp off
- bridge-fd 0
- mtu 1500
-
-source /etc/network/interfaces.d/*
-----
-
-node3: /etc/network/interfaces
-
-----
-auto vmbr0
-iface vmbr0 inet static
- address 192.168.0.3/24
- gateway 192.168.0.254
- bridge-ports eno1
- bridge-stp off
- bridge-fd 0
- mtu 1500
+~~~~~~~~~~~~~
-source /etc/network/interfaces.d/*
-----
+The example assumes a cluster with three nodes (node1, node2, node3) with IP
+addresses 192.168.0.1, 192.168.0.2 and 192.168.0.3.
Create an EVPN controller, using a private ASN number and the above node
addresses as peers.
----
-id: myevpnctl
-asn: 65000
-peers: 192.168.0.1,192.168.0.2,192.168.0.3
+ID: myevpnctl
+ASN#: 65000
+Peers: 192.168.0.1,192.168.0.2,192.168.0.3
----
-Create an EVPN zone named `myevpnzone', using the previously created
-EVPN-controller. Define 'node1' and 'node2' as exit nodes.
+Create an EVPN zone named `myevpnzone`, assign the previously created
+EVPN-controller and define 'node1' and 'node2' as exit nodes.
----
-id: myevpnzone
-vrf vxlan tag: 10000
-controller: myevpnctl
-mtu: 1450
-vnet mac address: 32:F4:05:FE:6C:0A
-exitnodes: node1,node2
+ID: myevpnzone
+VRF VXLAN Tag: 10000
+Controller: myevpnctl
+MTU: 1450
+VNet MAC Address: 32:F4:05:FE:6C:0A
+Exit Nodes: node1,node2
----
-Create the first VNet named `myvnet1' using the EVPN zone `myevpnzone'.
+Create the first VNet named `myvnet1` using the EVPN zone `myevpnzone`.
+
----
-id: myvnet1
-zone: myevpnzone
-tag: 11000
+ID: myvnet1
+Zone: myevpnzone
+Tag: 11000
----
-Create a subnet 10.0.1.0/24 with 10.0.1.1 as gateway on `myvnet1`.
+Create a subnet on `myvnet1`:
----
-subnet: 10.0.1.0/24
-gateway: 10.0.1.1
+Subnet: 10.0.1.0/24
+Gateway: 10.0.1.1
----
-Create the second VNet named `myvnet2' using the same EVPN zone `myevpnzone', a
-different IPv4 CIDR network.
+Create the second VNet named `myvnet2` using the same EVPN zone `myevpnzone`.
----
-id: myvnet2
-zone: myevpnzone
-tag: 12000
+ID: myvnet2
+Zone: myevpnzone
+Tag: 12000
----
-Create a different subnet 10.0.2.0/24 with 10.0.2.1 as gateway on vnet2
+Create a different subnet on `myvnet2``:
----
-subnet: 10.0.2.0/24
-gateway: 10.0.2.1
+Subnet: 10.0.2.0/24
+Gateway: 10.0.2.1
----
-
Apply the configuration from the main SDN web-interface panel to create VNets
-locally on each node and generate the FRR config.
+locally on each node and generate the FRR configuration.
-Create a Debian-based virtual machine (vm1) on node1, with a vNIC on `myvnet1'.
+Create a Debian-based virtual machine ('vm1') on node1, with a vNIC on `myvnet1`.
-Use the following network configuration for this VM:
+Use the following network configuration for 'vm1':
----
auto eth0
iface eth0 inet static
- address 10.0.1.100/24
- gateway 10.0.1.1 #this is the ip of the vnet1
- mtu 1450
+ address 10.0.1.100/24
+ gateway 10.0.1.1
+ mtu 1450
----
-Create a second virtual machine (vm2) on node2, with a vNIC on the other VNet
-`myvnet2'.
+Create a second virtual machine ('vm2') on node2, with a vNIC on the other VNet
+`myvnet2`.
-Use the following network configuration for this VM:
+Use the following network configuration for 'vm2':
----
auto eth0
iface eth0 inet static
- address 10.0.2.100/24
- gateway 10.0.2.1 #this is the ip of the myvnet2
- mtu 1450
+ address 10.0.2.100/24
+ gateway 10.0.2.1
+ mtu 1450
----
-Then, you should be able to ping vm2 from vm1, and vm1 from vm2.
+Now you should be able to ping vm2 from vm1, and vm1 from vm2.
-If you ping an external IP from 'vm2' on the non-gateway 'node3', the packet
+If you ping an external IP from 'vm2' on the non-gateway node3, the packet
will go to the configured 'myvnet2' gateway, then will be routed to the exit
nodes ('node1' or 'node2') and from there it will leave those nodes over the
default gateway configured on node1 or node2.
@@ -974,38 +843,43 @@ If you have configured an external BGP router, the BGP-EVPN routes (10.0.1.0/24
and 10.0.2.0/24 in this example), will be announced dynamically.
+[[pvesdn_notes]]
Notes
------
+-------------
Multiple EVPN Exit Nodes
-~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~
If you have multiple gateway nodes, you should disable the `rp_filter` (Strict
Reverse Path Filter) option, because packets can arrive at one node but go out
from another node.
-.sysctl.conf disabling `rp_filter`
+Add the following to `/etc/sysctl.conf`:
+
-----
net.ipv4.conf.default.rp_filter=0
net.ipv4.conf.all.rp_filter=0
-----
VXLAN IPSEC Encryption
-~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~
+
+To add IPSEC encryption on top of a VXLAN, this example shows how to use
+`strongswan`.
-If you need to add encryption on top of a VXLAN, it's possible to do so with
-IPSEC, through `strongswan`. You'll need to reduce the 'MTU' by 60 bytes (IPv4)
-or 80 bytes (IPv6) to handle encryption.
+You`ll need to reduce the 'MTU' by additional 60 bytes for IPv4 or 80 bytes for
+IPv6 to handle encryption.
So with default real 1500 MTU, you need to use a MTU of 1370 (1370 + 80 (IPSEC)
+ 50 (VXLAN) == 1500).
-.Install strongswan
+Install strongswan on the host.
+
----
apt install strongswan
----
-Add configuration to `/etc/ipsec.conf'. We only need to encrypt traffic from
+Add configuration to `/etc/ipsec.conf`. We only need to encrypt traffic from
the VXLAN UDP port '4789'.
----
@@ -1028,16 +902,16 @@ conn input
auto=route
----
-Then generate a pre-shared key with:
+Generate a pre-shared key with:
----
openssl rand -base64 128
----
-and add the key to `/etc/ipsec.secrets', so that the file contents looks like:
+and add the key to `/etc/ipsec.secrets`, so that the file contents looks like:
----
: PSK <generatedbase64key>
----
-You need to copy the PSK and the configuration onto the other nodes.
+Copy the PSK and the configuration to all nodes participating in the VXLAN network.
--
2.42.0
More information about the pve-devel
mailing list