[pve-devel] [PATCH pve-docs 1/1] fabrics: add initial documentation for sdn fabrics

[Shannon Sterz]

On Fri Mar 28, 2025 at 6:13 PM CET, Gabriel Goller wrote:
> Add initial documentation for the openfabric and ospf options.
>
> Signed-off-by: Gabriel Goller <g.goller at proxmox.com>
> ---
>  pvesdn.adoc | 155 ++++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 155 insertions(+)
>
> diff --git a/pvesdn.adoc b/pvesdn.adoc
> index 5d5d27bfbc1f..feb6898beaa5 100644
> --- a/pvesdn.adoc
> +++ b/pvesdn.adoc
> @@ -517,6 +517,161 @@ Loopback:: Use a loopback or dummy interface as the source of the EVPN network
>    (for multipath).
>
>
> +[[pvesdn_config_fabrics]]
> +Fabrics
> +-------
> +
> +Fabrics in {pve} SDN provide automated routing between nodes in a cluster. They
> +simplify the configuration of underlay networks between nodes to form the
> +foundation for SDN deployments.
> +
> +Fabrics in {pve} automatically configure routing protocols on your physical
> +network interfaces to establish connectivity between nodes in the cluster. This
> +creates a resilient, auto-configuring network fabric that adapts to changes in
> +network topology.

feels strange to have two adjacent paragraphs start with "Fabrics in
Proxmox VE" you could do something "You can use Fabrics to
automatically..." or just "They" imo.

> +
> +Installation
> +~~~~~~~~~~~~
> +
> +We use the FRR implementations of OpenFabric and OSPF, so first you need to
> +install `frr` and the `frr-pythontools` package:
> +
> +----
> +apt update
> +apt install frr frr-pythontools
> +----
> +
> +Configuration
> +~~~~~~~~~~~~~
> +
> +To create a Fabric, head over to Datacenter->SDN->Fabrics and click "Add
> +Fabric" to create a new fabric. After selecting the preferred protocol, the

starting and ending the sentence with "To create a (new) Fabric" is
redundant. lose the "to create a new fabric." at the end.

> +fabric is created. With the "+" button you can select the nodes which you want
> +to add to the fabric, you also have to select the interfaces used to
> +communicate to the other nodes.

nit: with the other nodes.

> +
> +Router-ID Selection
> +^^^^^^^^^^^^^^^^^^^
> +
> +Each node in a fabric needs a unique router ID, which is an IPv4 (or IPv6 in
> +OpenFabric) address in dotted-decimal notation (e.g., 192.168.1.1).

does this apply to v6 too? i've never seen IPv6 represented in dotted
decimal notation. imo if you mention IPv6 here, you should specify it's
representation. maybe:

Each node in a fabric needs a unique router ID, which is an IPv4 address
in dotted decimal notation (e.g. 192.168.1.1). In OpenFabric this can
also be an IPv6 address in the typical hexadecimal representation
separated by colons (e.g., 2001:db8::1428:57ab).

> +
> +Loopback Prefix
> +^^^^^^^^^^^^^^^
> +
> +You can specify a CIDR network range (e.g., 10.0.0.0/24) as a loopback prefix for the fabric.
> +When configured, the system will automatically verify that all router-IDs are contained within
> +this prefix. This ensures consistency in your addressing scheme and helps prevent addressing
> +conflicts or errors. The loopback prefix applies to the entire fabric and is optional.
> +
> +[[pvesdn_openfabric]]
> +OpenFabric
> +~~~~~~~~~~
> +
> +OpenFabric is a routing protocol specifically designed for data center fabrics.
> +It's based on IS-IS and optimized for the leaf-spine topology common in data
> +centers.
> +
> +Configuration options:
> +
> +[[pvesdn_openfabric_fabric]]
> +On the Fabric
> +^^^^^^^^^^^^^
> +
> +Name:: This is the name of the OpenFabric fabric and can be at most 8 characters long.
> +
> +Loopback Prefix:: CIDR (IPv4 or IPv6) network range (e.g., 10.0.0.0/24) used to verify that
> +all router-IDs in the fabric are contained within this prefix.
> +
> +Hello Interval:: Controls how frequently (in seconds) hello packets are sent to
> +discover and maintain connections with neighboring nodes. Lower values detect
> +failures faster but increase network traffic. If empty, the default value will

what is the default value? would it make sense to mention that here?

> +be used. This option is global on the fabric, meaning every interface on every
> +node in this fabric will inherit this hello-interval property.
> +
> +[[pvesdn_openfabric_node]]
> +On the Node
> +^^^^^^^^^^^
> +
> +Node:: Select the node which will be added to the fabric. Only nodes that
> +currently are in the cluster will be shown.
> +
> +Router-ID:: A unique IPv4 or IPv6 address used to generate the OpenFabric
> +Network Entity Title (NET). Each node in the same fabric must have a different
> +Router-ID, while a single node must use the same NET address across all fabrics
> +(this consistency is automatically managed by {pve}).
> +
> +NOTE: When using IPv6 addresses, we use the last 3 segments to generate the
> +NET. Ensure these segments differ between nodes.

would it make sense to make this a `WARNING` instead of a `NOTE`? sounds
like this is a bit more important to get right.

> +Interfaces:: Specify the interfaces used to establish peering connections with
> +other OpenFabric nodes. Preferably select interfaces without pre-assigned IP
> +addresses, then configure addresses in the IPv4/IPv6 column if needed. A dummy
> +"loopback" interface with the router-id is automatically created.
> +Interface-specific settings override the global fabric settings.
> +
> +The following optional parameters can be configured per interface when enabling
> +the additional columns:
> +
> +Passive::: When enabled, the interface will not form OpenFabric adjacencies but
> +its networks will still be advertised.
> +
> +Hello Interval::: Controls how frequently (in seconds) hello packets are sent
> +on this specific interface. Lower values detect failures faster but increase
> +network traffic.
> +
> +CSNP Interval::: Controls how frequently (in seconds) Complete Sequence Number
> +PDUs are sent. CSNPs contain a complete list of all LSPs in the database.
> +
> +Hello Multiplier::: Defines how many missed hello packets constitute a failed
> +connection. Higher values make the connection more resilient to packet loss but
> +slow down failure detection.
> +
> +[[pvesdn_ospf]]
> +OSPF
> +~~~~
> +
> +OSPF (Open Shortest Path First) is a widely-used link-state routing protocol
> +that efficiently calculates the shortest path for routing traffic through IP
> +networks.
> +
> +[[pvesdn_ospf_fabric]]
> +On the Fabric
> +^^^^^^^^^^^^^
> +
> +Area:: This specifies the OSPF area identifier, which can be either an integer
> +(i32) or an IP address. Areas are a way to organize and structure OSPF networks

i32 is super intuitive for Rust programmers but "32-bit signed integer"
would be clearer to everyone else ;)

> +hierarchically, with Area 0 (or 0.0.0.0) serving as the backbone area.
> +
> +Loopback Prefix:: CIDR (only IPv4) network range (e.g., 10.0.0.0/24) used to
> +verify that all router-IDs in the fabric are contained within this prefix.
> +
> +Area:: This specifies the OSPF area identifier, which can be either an integer
> +(i32) or an IP address. Areas are a way to organize and structure OSPF networks

same here :)

> +hierarchically, with Area 0 (or 0.0.0.0) serving as the backbone area.
> +
> +[[pvesdn_ospf_node]]
> +On the Node
> +^^^^^^^^^^^
> +
> +Node:: Select the node which will be added to the fabric. Only nodes that
> +currently are in the cluster will be shown.

"that are currently in..."

> +
> +Router-ID:: A unique IPv4 address used to identify this router within the OSPF
> +network. Each node in the same fabric must have a different Router-ID.
> +
> +Interfaces:: Specify the interfaces used to establish peering connections with
> +other OSPF nodes. Preferably select interfaces without pre-assigned IP
> +addresses, then configure addresses in the IPv4 column if needed. A dummy
> +"loopback" interface with the router-id is automatically created.
> +
> +The following optional parameter can be configured per interface when enabling
> +the additional columns:
> +
> +Passive::: When enabled, the interface will not form OSPF adjacencies but
> +its networks will still be advertised.
> +
> +
>  [[pvesdn_config_ipam]]
>  IPAM
>  ----