[pdm-devel] [RFC proxmox{-api-types, -yew-comp, -datacenter-manager} 00/26] Add initial SDN / EVPN integration

Fri Feb 28 16:17:37 CET 2025

## Introduction

This patch series introduces the first SDN features to the Proxmox Datacenter
Manager. I sent it as an RFC, since I wanted to get some early feedback and
nothing presented here is set in stone.

This patch series is mainly intended for laying the groundwork on how to handle
SDN configuration changes across multiple remotes. To demonstrate the concept it
implements two, relatively simple, features using the newly introduced lock
functionality:

* Creating a new VRF (= zone) across multiple remotes
* Creating a new VNet across multiple remotes

Additionally there is an aggregated overview of all EVPN zones / controllers of
all PVE remotes.

Once everything has been properly fleshed out, the groundwork laid here will be
used to build the full SDN/EVPN integration into PDM with more complex
configuration changes.

## Prerequisites

This patch series relies on the following preparatory patch series, that I've
sent separately:

https://lore.proxmox.com/pve-devel/20250228130549.100357-1-s.hanreich@proxmox.com/T/
https://lore.proxmox.com/pve-devel/20250228140136.124286-1-s.hanreich@proxmox.com/T/
https://lore.proxmox.com/pbs-devel/20250129104250.934927-1-s.hanreich@proxmox.com/T/

I've sent others, but as far as I can tell they have already been merged. Please
tell me if I missed something and you cannot build this patch series! Not all of
them have been bumped yet, so make sure to apply the patches on top of the
current master.

## How it works

I introduced a new locking mechanism to the SDN API, that locks the
configuration and enables subsequent modifications only when the lock secret is
provided. Locking the configuration only succeeds, if there are no pending
changes, but there's a knob for tuning that behavior (that we could expose in
the PDM UI as well). See my other patch series for more detailed information.

The two PDM API calls introduced in this patch series do the following sequence
of actions when trying to change the SDN configuration:

* lock the sdn configuration of all involved remotes
  * if locking at least one remote fails, release the lock on all remotes and
    return unsuccessfully without making any changes
* invoke the API endpoints to make the changes to the SDN configuration
  * if a call fails, no further changes will be made to the remote and the
    configuration will stay locked - no rollbacks
  * no further changes will be made to the remote after one call fails
* apply the configuration on all remotes where the changes were successful
  * any errors during applying the configuration on a remote will be logged
  * the configuration will stay locked if applying the configuration fails
* reload the network configuration on all nodes where applying was succesful
  * if reloading was unsuccessful, the configuration will be unlocked, but the
    user will get an error message (This is also how applying the configuration
    on a single cluster currently works).
  * the task will wait for all remotes to finish reloading and only return
    successfully if changing the configuration on all remotes was successful

We are doing no automatic rollback of configuration changes. If we want to
introduce automatic rollback from PDM, this is how we could do it with how the
new SDN lock functionality works: We lock the SDN configuration from PDM if
there are no pending changes. We then proceed to make our changes to the config.
If we run into any error, we can be sure that all pending changes were made by
PDM. This fact enables us to safely rollback the configuration changes we made
and unlock the SDN configuration. This could be opt-in as a global PDM setting
(or when starting the task).

In the case of network issues, we cannot really proceed with making
configuration changes or rolling back the configuration. The configuration
changes will stay on PVE side, and users can choose to unlock & rollback.

## Open issues

Currently, we need information about 3 types of entities in the UI (zones,
vnets, controllers). The PDM API calls make an API call for each of those types
to the remotes and aggregates them before returning them. This makes the calls
really expensive, since we need to make 3 API calls to every remote configured
in PDM.

In this series, the yew component makes this call to PDM once and stores all the
results, passing the entities to every child component. This way we do not need
to repeat those calls when opening an EditWindow, for instance. In the future I
want to add a proper caching mechanism on the server side. Since this patch
series was already quite extensive I've decided to cut it out of scope for now.
I will think a bit on the best way to implement caching for this and implement
it in a follow-up patch series or a v2.

The matching of zones to a VRF is based solely on their VRF VXLAN VNI, in the
future we should also consider the rt-import configuration key for matching VRFs
to each other.

## Dependencies
* proxmox-api-types depends on pve-network (from the separate patch series)
* proxmox-datacenter-manager depends on proxmox-api-types / proxmox-yew-comp

proxmox-api-types:

Stefan Hanreich (12):
  sdn: add list/create zone endpoints
  sdn: generate zones endpoints
  sdn: add list/create vnet endpoints
  sdn: generate vnet endpoints
  sdn: add list/create controller endpoints
  sdn: generate controller endpoints
  sdn: add acquire/release lock endpoints
  sdn: generate lock endpoints
  sdn: add apply configuration endpoint
  sdn: generate apply configuration endpoint
  tasks: add helper for querying successfully finished tasks
  sdn: add helpers for pending values

 pve-api-types/generate.pl            |   36 +
 pve-api-types/src/generated/code.rs  |  150 +-
 pve-api-types/src/generated/types.rs | 5259 +++++++++++++++++---------
 pve-api-types/src/lib.rs             |    1 +
 pve-api-types/src/sdn.rs             |   33 +
 pve-api-types/src/types/mod.rs       |    4 +
 6 files changed, 3609 insertions(+), 1874 deletions(-)
 create mode 100644 pve-api-types/src/sdn.rs

proxmox-yew-comp:

Stefan Hanreich (1):
  sdn: add descriptions for sdn tasks

 src/utils.rs | 3 +++
 1 file changed, 3 insertions(+)

proxmox-datacenter-manager:

Stefan Hanreich (13):
  server: add locked sdn client and helper methods
  api: sdn: add list_zones endpoint
  api: sdn: add create_zone endpoint
  api: sdn: add list_vnets endpoint
  api: sdn: add create_vnet endpoint
  api: sdn: add list_controllers endpoint
  ui: add VrfTree component
  ui: sdn: add RouterTable component
  ui: sdn: add AddVnetWindow component
  ui: sdn: add AddZoneWindow component
  ui: sdn: add EvpnPanel
  ui: sdn: add EvpnPanel to main menu
  pve: sdn: add descriptions for sdn tasks

 lib/pdm-api-types/Cargo.toml      |   2 +
 lib/pdm-api-types/src/lib.rs      |   2 +
 lib/pdm-api-types/src/sdn.rs      | 168 +++++++++++++++++
 lib/pdm-client/src/lib.rs         |  64 +++++++
 server/src/api/mod.rs             |   2 +
 server/src/api/sdn/controllers.rs |  78 ++++++++
 server/src/api/sdn/mod.rs         |  17 ++
 server/src/api/sdn/vnets.rs       | 151 ++++++++++++++++
 server/src/api/sdn/zones.rs       | 177 ++++++++++++++++++
 server/src/lib.rs                 |   1 +
 server/src/sdn_client.rs          | 234 ++++++++++++++++++++++++
 ui/src/lib.rs                     |   2 +
 ui/src/main_menu.rs               |  10 +
 ui/src/sdn/evpn/add_vnet.rs       | 216 ++++++++++++++++++++++
 ui/src/sdn/evpn/add_zone.rs       | 229 +++++++++++++++++++++++
 ui/src/sdn/evpn/evpn_panel.rs     | 249 +++++++++++++++++++++++++
 ui/src/sdn/evpn/mod.rs            |  14 ++
 ui/src/sdn/evpn/router_table.rs   | 125 +++++++++++++
 ui/src/sdn/evpn/vrf_tree.rs       | 291 ++++++++++++++++++++++++++++++
 ui/src/sdn/mod.rs                 |   1 +
 ui/src/tasks.rs                   |   2 +
 21 files changed, 2035 insertions(+)
 create mode 100644 lib/pdm-api-types/src/sdn.rs
 create mode 100644 server/src/api/sdn/controllers.rs
 create mode 100644 server/src/api/sdn/mod.rs
 create mode 100644 server/src/api/sdn/vnets.rs
 create mode 100644 server/src/api/sdn/zones.rs
 create mode 100644 server/src/sdn_client.rs
 create mode 100644 ui/src/sdn/evpn/add_vnet.rs
 create mode 100644 ui/src/sdn/evpn/add_zone.rs
 create mode 100644 ui/src/sdn/evpn/evpn_panel.rs
 create mode 100644 ui/src/sdn/evpn/mod.rs
 create mode 100644 ui/src/sdn/evpn/router_table.rs
 create mode 100644 ui/src/sdn/evpn/vrf_tree.rs
 create mode 100644 ui/src/sdn/mod.rs

Summary over all repositories:
  28 files changed, 5647 insertions(+), 1874 deletions(-)

-- 
Generated by git-murpp 0.8.0