[!!ACHTUNG extern!!] - Re: SPAM: [PATCH pve-network v4 1/2] ipam: add Nautobot plugin - range workaround questions
Lou.Lecrivain at wdz.de
Lou.Lecrivain at wdz.de
Fri May 9 11:34:56 CEST 2025
Re,
Ok, thanks for clarifying. Indeed the offset parameter is ineffective (I have looked at the Nautobot code this morning). Tbqh I don't really like having to create then delete an object in Nautobot for this, but I guess there's no other way, given the limitations we're faced with.
Alternatively as a middle ground (and I think this would be much quicker than implementing IP ranges in Nautobot), we could implement the offset parameter in Nautobot and change the original workaround code to fit this, and then require a minimal Nautobot version (for ranges or the whole plugin). But IMO this doesn't mean we can't merge this. We could change it afterwards.
I have no other remarks. Thanks again for your work.
BR
Lou
________________________________________
De : Hannes Duerr <h.duerr at proxmox.com>
Envoyé : vendredi 9 mai 2025 10:46
À : Lecrivain, Lou (WDZ) <Lou.Lecrivain at wdz.de>; pve-devel at lists.proxmox.com <pve-devel at lists.proxmox.com>
Objet : [!!ACHTUNG extern!!] - Re: SPAM: [PATCH pve-network v4 1/2] ipam: add Nautobot plugin - range workaround questions
On 5/7/25 11:17, Lou.Lecrivain at wdz.de wrote:
Hannes,
Thank you for helping with getting the Netbox plugin in good shape, it's appreciated, really :)
I just have two questions regarding the lack of range support in Nautobot:
- Was there an issue with the original workaround code? I find this one way more complex,
but also I guess I could've missed some corner case(s).
Yes, there was the problem that the old version would possibly output
invalid free IP addresses if the IP range did not start directly at
the beginning of the prefix. Assuming you have 192.168.0.0/16 as
prefix, all IP addresses are still free and you want free ip addresses
in the range 192.168.6.1/16 - 192.168.7.255/16 then the old
implementation would still have received the addresses
192.168.0.[0-50]/24 from the API. This could have been remedied by an
offset parameter, but this is not implemented for the API endpoint.
Increasing the `limit` is also not an option, as there is a maximum
limit of 1000 and you can still run into problems.
I have therefore opted for the current implementation.
- What happens if we use a prefix with a CIDR greater than /24 in IPv4 or /120 in IPv6?
Does not being able to slice it means that ranges with small prefixes such as these
would be forbidden/not working?
No, all prefixes should work. For smaller prefixes such as /26, we
still create a /24 pool and search it for free ips, but then check
again whether the ip is actually in the desired range in /26.
BR
Lou
________________________________________
De : Hannes Duerr <h.duerr at proxmox.com>
Envoyé : mardi 6 mai 2025 11:00
À : pve-devel at lists.proxmox.com <pve-devel at lists.proxmox.com>
Cc : Hannes Duerr <h.duerr at proxmox.com>; Lecrivain, Lou (WDZ) <Lou.Lecrivain at wdz.de>
Objet : [!!ACHTUNG extern!!] - [PATCH pve-network v4 1/2] ipam: add Nautobot plugin
Nautobot is a network source of truth [0] and as such this plugin offers
the possibility to automatically enter IP addresses and subnets
(prefixes in Nautobot jargon) into Nautobot as soon as they are created
in Proxmox VE.
Limitations:
* The IPAM plugin currently does not recognize whether VMs/CTs are
active/online and initially sets the status to Active but when the
VMs/CTs are switched off the status is not deactivated.
* Nautobot does not support ranges that can be searched for free ip
addresses, so we map as many pool prefixes that we search until we
have searched the entire range for free ip addresses
* Nautobot has the possibility to map DNS names to IP addresses.
However, since we have no standardized way to set the DNS names in
Proxmox VE (for containers the container name in Proxmox VE is set as
the host name in the container, but this is not possible for VMs) we
refrain from setting a DNS name at the moment.
* Nautobot does not have a field in IP address objects for a MAC address
or to mark them as gateway, so we write this in the comment.
[0] https://networktocode.com/nautobot/
Co-authored-by: lou lecrivain <lou.lecrivain at wdz.de>
Signed-off-by: Hannes Duerr <h.duerr at proxmox.com>
---
Notes:
Changes from v3 -> v4:
* Merge the check for empty prefixes/subnets into this commit
* Create nautobot_api_request to unify API calls
* add update_subnet function
* fix add_range_next_freeip
* fix and unify error handling
* Fix perl style
src/PVE/API2/Network/SDN/Ipams.pm | 1 +
src/PVE/Network/SDN/Ipams.pm | 3 +
src/PVE/Network/SDN/Ipams/Makefile | 2 +-
src/PVE/Network/SDN/Ipams/NautobotPlugin.pm | 513 ++++++++++++++++++++
4 files changed, 518 insertions(+), 1 deletion(-)
create mode 100644 src/PVE/Network/SDN/Ipams/NautobotPlugin.pm
diff --git a/src/PVE/API2/Network/SDN/Ipams.pm b/src/PVE/API2/Network/SDN/Ipams.pm
index 27ead02..8074512 100644
--- a/src/PVE/API2/Network/SDN/Ipams.pm
+++ b/src/PVE/API2/Network/SDN/Ipams.pm
@@ -12,6 +12,7 @@ use PVE::Network::SDN::Ipams::Plugin;
use PVE::Network::SDN::Ipams::PVEPlugin;
use PVE::Network::SDN::Ipams::PhpIpamPlugin;
use PVE::Network::SDN::Ipams::NetboxPlugin;
+use PVE::Network::SDN::Ipams::NautobotPlugin;
use PVE::Network::SDN::Dhcp;
use PVE::Network::SDN::Vnets;
use PVE::Network::SDN::Zones;
diff --git a/src/PVE/Network/SDN/Ipams.pm b/src/PVE/Network/SDN/Ipams.pm
index c689b8f..2ecb75e 100644
--- a/src/PVE/Network/SDN/Ipams.pm
+++ b/src/PVE/Network/SDN/Ipams.pm
@@ -12,11 +12,14 @@ use PVE::Network;
use PVE::Network::SDN::Ipams::PVEPlugin;
use PVE::Network::SDN::Ipams::NetboxPlugin;
+use PVE::Network::SDN::Ipams::NautobotPlugin;
use PVE::Network::SDN::Ipams::PhpIpamPlugin;
use PVE::Network::SDN::Ipams::Plugin;
+
PVE::Network::SDN::Ipams::PVEPlugin->register();
PVE::Network::SDN::Ipams::NetboxPlugin->register();
+PVE::Network::SDN::Ipams::NautobotPlugin->register();
PVE::Network::SDN::Ipams::PhpIpamPlugin->register();
PVE::Network::SDN::Ipams::Plugin->init();
diff --git a/src/PVE/Network/SDN/Ipams/Makefile b/src/PVE/Network/SDN/Ipams/Makefile
index 4e7d65f..75e5b9a 100644
--- a/src/PVE/Network/SDN/Ipams/Makefile
+++ b/src/PVE/Network/SDN/Ipams/Makefile
@@ -1,4 +1,4 @@
-SOURCES=Plugin.pm PhpIpamPlugin.pm NetboxPlugin.pm PVEPlugin.pm
+SOURCES=Plugin.pm PhpIpamPlugin.pm NetboxPlugin.pm PVEPlugin.pm NautobotPlugin.pm
PERL5DIR=${DESTDIR}/usr/share/perl5
diff --git a/src/PVE/Network/SDN/Ipams/NautobotPlugin.pm b/src/PVE/Network/SDN/Ipams/NautobotPlugin.pm
new file mode 100644
index 0000000..2897025
--- /dev/null
+++ b/src/PVE/Network/SDN/Ipams/NautobotPlugin.pm
@@ -0,0 +1,513 @@
+package PVE::Network::SDN::Ipams::NautobotPlugin;
+
+use strict;
+use warnings;
+use PVE::INotify;
+use PVE::Cluster;
+use PVE::Tools;
+use NetAddr::IP;
+use Net::Subnet qw(subnet_matcher);
+
+use base('PVE::Network::SDN::Ipams::Plugin');
+
+sub type {
+ return 'nautobot';
+}
+
+sub properties {
+ return {
+ namespace => {
+ type => 'string',
+ },
+ };
+}
+
+sub options {
+ return {
+ url => { optional => 0 },
+ token => { optional => 0 },
+ namespace => { optional => 0 },
+ fingerprint => { optional => 1 },
+ };
+}
+
+sub default_ip_status {
+ return 'Active';
+}
+
+sub nautobot_api_request {
+ my ($config, $method, $path, $params) = @_;
+
+ return PVE::Network::SDN::api_request(
+ $method,
+ "$config->{url}${path}",
+ [
+ 'Content-Type' => 'application/json; charset=UTF-8',
+ 'Authorization' => "token $config->{token}",
+ 'Accept' => "application/json"
+ ],
+ $params,
+ $config->{fingerprint},
+ );
+}
+
+sub add_subnet {
+ my ($class, $config, undef, $subnet, $noerr) = @_;
+
+ my $cidr = $subnet->{cidr};
+ my $namespace = $config->{namespace};
+
+ my $internalid = get_prefix_id($config, $cidr, $noerr);
+ if ($internalid) {
+ return if $noerr;
+ die "could not add the subnet $subnet because it already exists in nautobot\n"
+ }
+
+ my $params = {
+ prefix => $cidr,
+ namespace => $namespace,
+ status => default_ip_status()
+ };
+
+ eval {
+ nautobot_api_request($config, "POST", "/ipam/prefixes/", $params);
+ };
+ if ($@) {
+ return if $noerr;
+ die "error adding the subnet $subnet to nautobot $@\n";
+ }
+}
+
+sub update_subnet {
+ my ($class, $plugin_config, $subnetid, $subnet, $old_subnet, $noerr) = @_;
+ # dhcp ranges are not supported in nautobot so we don't have to update them
+}
+
+sub del_subnet {
+ my ($class, $config, $subnetid, $subnet, $noerr) = @_;
+
+ my $cidr = $subnet->{cidr};
+
+ my $internalid = get_prefix_id($config, $cidr, $noerr);
+ if (!$internalid) {
+ warn("could not find delete the subnet $cidr because it does not exist in nautobot\n");
+ return;
+ }
+
+ if (!subnet_is_deletable($config, $subnetid, $subnet, $internalid, $noerr)) {
+ return if $noerr;
+ die "could not delete the subnet $cidr, it still contains ip addresses!\n";
+ }
+
+ # delete associated gateway IP addresses
+ $class->empty_subnet( $config, $subnetid, $subnet, $internalid, $noerr);
+
+ eval {
+ nautobot_api_request($config, "DELETE", "/ipam/prefixes/$internalid/");
+ };
+ if ($@) {
+ return if $noerr;
+ die "error deleting subnet from nautobot: $@\n";
+ }
+ return 1;
+}
+
+sub add_ip {
+ my ($class, $config, undef, $subnet, $ip, $hostname, $mac, undef,
+ $is_gateway, $noerr) = @_;
+
+ my $mask = $subnet->{mask};
+ my $namespace = $config->{namespace};
+
+ my $description = undef;
+ if ($is_gateway) {
+ $description = 'gateway';
+ } elsif ($mac) {
+ $description = "mac:$mac";
+ }
+
+ my $params = {
+ address => "$ip/$mask",
+ type => "dhcp",
+ description => $description,
+ namespace => $namespace,
+ status => default_ip_status()
+ };
+
+ eval {
+ nautobot_api_request($config, "POST", "/ipam/ip-addresses/", $params);
+ };
+
+ if ($@) {
+ if ($is_gateway) {
+ die "error add subnet ip to ipam: ip $ip already exist: $@"
+ if !is_ip_gateway($config, $ip, $noerr);
+ } elsif (!$noerr) {
+ die "error add subnet ip to ipam: ip already exist: $@";
+ }
+ }
+}
+
+sub add_next_freeip {
+ my ($class, $config, undef, $subnet, $hostname, $mac, undef, $noerr) = @_;
+
+ my $cidr = $subnet->{cidr};
+ my $namespace = $config->{namespace};
+
+ my $internalid = get_prefix_id($config, $cidr, $noerr);
+ if (!defined($internalid)) {
+ return if $noerr;
+ die "could not find prefix $cidr in nautobot\n";
+ }
+
+ my $description = undef;
+ $description = "mac:$mac" if $mac;
+
+ my $params = {
+ type => "dhcp",
+ description => $description,
+ namespace => $namespace,
+ status => default_ip_status()
+ };
+
+ my $response = eval {
+ return nautobot_api_request(
+ $config,
+ "POST",
+ "/ipam/prefixes/$internalid/available-ips/",
+ $params
+ );
+ };
+ if ($@ || !$response) {
+ return if $noerr;
+ die "could not allocate ip in subnet $cidr: $@\n";
+ }
+
+ my $ip = NetAddr::IP->new($response->{address});
+
+ return $ip->addr;
+}
+
+sub add_range_next_freeip {
+ my ($class, $config, $subnet, $range, $data, $noerr) = @_;
+
+ my $cidr = NetAddr::IP->new($subnet->{cidr});
+ my $namespace = $config->{namespace};
+
+ # Nautobot does not support IP ranges only prefixes.
+ # We therefore divide the range into smaller pool-prefixes,
+ # each containing 256 addresses, and search for available IPs in them
+ my $prefix_size = $cidr->version == 4 ? 24 : 120;
+ my $increment = 256;
+ my $found_ip = undef;
+
+ my $start_range = NetAddr::IP->new($range->{'start-address'}, $prefix_size);
+ my $end_range = NetAddr::IP->new($range->{'end-address'}, $prefix_size);
+ my $matcher = subnet_matcher($end_range->cidr);
+ my $current_ip = $start_range;
+
+ while (1) {
+ my $current_cidr = $current_ip->addr . "/$prefix_size";
+
+ my $params = {
+ prefix => $current_cidr,
+ namespace => $namespace,
+ status => default_ip_status(),
+ type => "pool"
+ };
+
+ # create temporary pool prefix
+ my $temp_prefix = eval {
+ return nautobot_api_request($config, "POST", "/ipam/prefixes/", $params);
+ };
+
+ # skip if it is not possible to create it
+ next if $@;
+
+ my $temp_prefix_id = $temp_prefix->{id};
+
+ # Fetch available IPs from the temporary pool and find a matching IP
+ my $result = eval {
+ return nautobot_api_request(
+ $config,
+ "GET",
+ "/ipam/prefixes/$temp_prefix_id/available-ips/?limit=$increment"
+ );
+ };
+
+ # search list for IPs in actual range
+ if (!$@) {
+ foreach my $entry (@$result) {
+ my $ip = NetAddr::IP->new($entry->{address});
+ # comparison is only possible because they are in the same subnet
+ if ($start_range <= $ip && $ip <= $end_range) {
+ $found_ip = $ip->addr;
+ last;
+ }
+ }
+ }
+
+ # Delete temporary prefix pool
+ eval {
+ nautobot_api_request($config, "DELETE", "/ipam/prefixes/$temp_prefix_id/");
+ };
+
+ last if $found_ip;
+
+ # we searched the last pool prefix
+ last if $matcher->($current_ip->addr);
+
+ $current_ip = $current_ip->plus($increment);
+ }
+
+ if (!$found_ip) {
+ return if $noerr;
+ die "could not allocate ip in the range " . $start_range->addr . " - "
+ . $end_range->addr . ": $@\n";
+ }
+
+ $class->add_ip($config, undef, $subnet, $found_ip, $data->{hostname}, $data->{mac}, undef, 0, $noerr);
+
+ return $found_ip;
+}
+
+sub update_ip {
+ my ($class, $config, $subnetid, $subnet, $ip, $hostname, $mac,
+ undef, $is_gateway, $noerr) = @_;
+
+ my $mask = $subnet->{mask};
+ my $namespace = $config->{namespace};
+
+ my $description = undef;
+ if ($is_gateway) {
+ $description = 'gateway'
+ } elsif ($mac) {
+ $description = "mac:$mac";
+ }
+
+ my $params = {
+ address => "$ip/$mask",
+ type => "dhcp",
+ description => $description,
+ namespace => $namespace,
+ status => default_ip_status()
+ };
+
+ my $ip_id = get_ip_id($config, $ip, $noerr);
+ if (!defined($ip_id)) {
+ return if $noerr;
+ die "could not find the ip $ip in nautobot\n";
+ }
+
+ eval {
+ nautobot_api_request($config, "PATCH", "/ipam/ip-addresses/$ip_id/", $params);
+ };
+ if ($@) {
+ return if $noerr;
+ die "error updating ip $ip: $@";
+ }
+}
+
+
+sub del_ip {
+ my ($class, $config, undef, undef, $ip, $noerr) = @_;
+
+ return if !$ip;
+
+ my $ip_id = get_ip_id($config, $ip, $noerr);
+ if (!defined($ip_id)) {
+ warn("could not find the ip $ip in nautobot\n");
+ return;
+ }
+
+ eval {
+ nautobot_api_request($config, "DELETE", "/ipam/ip-addresses/$ip_id/");
+ };
+ if ($@) {
+ return if $noerr;
+ die "error deleting ip $ip : $@\n";
+ }
+
+ return 1;
+}
+
+sub empty_subnet {
+ my ($class, $config, $subnetid, $subnet, $subnetuuid, $noerr) = @_;
+
+ my $namespace = $config->{namespace};
+
+ my $response = eval {
+ return nautobot_api_request(
+ $config,
+ "GET",
+ "/ipam/ip-addresses/?namespace=$namespace&parent=$subnetuuid"
+ );
+ };
+ if ($@) {
+ return if $noerr;
+ die "could not find the subnet $subnet in nautobot: $@\n";
+ }
+
+ for my $ip (@{$response->{results}}) {
+ del_ip($class, $config, undef, undef, $ip->{host}, $noerr);
+ }
+
+ return 1;
+}
+
+sub subnet_is_deletable {
+ my ($config, $subnetid, $subnet, $subnetuuid, $noerr) = @_;
+
+ my $namespace = $config->{namespace};
+
+ my $response = eval {
+ return nautobot_api_request(
+ $config,
+ "GET",
+ "/ipam/ip-addresses/?namespace=$namespace&parent=$subnetuuid"
+ );
+ };
+ if ($@) {
+ return if $noerr;
+ die "error querying prefix $subnet: $@\n";
+ }
+ my $n_ips = scalar $response->{results}->@*;
+
+ # least costly check operation 1st
+ return 1 if ($n_ips == 0);
+
+ for my $ip (values $response->{results}->@*) {
+ if (!is_ip_gateway($config, $ip->{host}, $noerr)) {
+ # some remaining IP is not a gateway so we can't delete the subnet
+ return 0;
+ }
+ }
+ #all remaining IPs are gateways
+ return 1;
+}
+
+sub verify_api {
+ my ($class, $config) = @_;
+
+ my $namespace = $config->{namespace};
+
+ # check if the namespace and the status "Active" exist
+ eval {
+ get_namespace_id($config, $namespace)
+ // die "namespace $namespace does not exist";
+ get_status_id($config, default_ip_status())
+ // die "the status " . default_ip_status() . " does not exist";
+ };
+ if ($@) {
+ die "could not use nautobot api: $@\n";
+ }
+}
+
+sub get_ips_from_mac {
+ my ($class, $config, $mac, $zone) = @_;
+
+ my $ip4 = undef;
+ my $ip6 = undef;
+
+ my $data = eval {
+ nautobot_api_request($config, "GET", "/ipam/ip-addresses/?q=$mac");
+ };
+ if ($@) {
+ die "could not query ip address entry for mac $mac: $@";
+ }
+
+ for my $ip (@{$data->{results}}) {
+ if ($ip->{ip_version} == 4 && !$ip4) {
+ ($ip4, undef) = split(/\//, $ip->{address});
+ }
+
+ if ($ip->{ip_version} == 6 && !$ip6) {
+ ($ip6, undef) = split(/\//, $ip->{address});
+ }
+ }
+
+ return ($ip4, $ip6);
+}
+
+sub on_update_hook {
+ my ($class, $config) = @_;
+
+ PVE::Network::SDN::Ipams::NautobotPlugin::verify_api($class, $config);
+}
+
+sub get_ip_id {
+ my ($config, $ip, $noerr) = @_;
+
+ my $result = eval {
+ return nautobot_api_request($config, "GET", "/ipam/ip-addresses/?address=$ip");
+ };
+ if ($@) {
+ return if $noerr;
+ die "error while querying for ip $ip id: $@\n";
+ }
+
+ my $data = @{$result->{results}}[0];
+ return $data->{id};
+}
+
+sub get_prefix_id {
+ my ($config, $cidr, $noerr) = @_;
+
+ my $result = eval {
+ return nautobot_api_request($config, "GET", "/ipam/prefixes/?prefix=$cidr");
+ };
+ if ($@) {
+ return if $noerr;
+ die "error while querying for cidr $cidr prefix id: $@\n";
+ }
+
+ my $data = @{$result->{results}}[0];
+ return $data->{id};
+}
+
+sub get_namespace_id {
+ my ($config, $namespace, $noerr) = @_;
+
+ my $result = eval {
+ return nautobot_api_request($config, "GET", "/ipam/namespaces/?name=$namespace");
+ };
+ if ($@) {
+ return if $noerr;
+ die "error while querying for namespace $namespace id: $@\n";
+ }
+
+ my $data = @{$result->{results}}[0];
+ return $data->{id};
+}
+
+sub get_status_id {
+ my ($config, $status, $noerr) = @_;
+
+ my $result = eval {
+ return nautobot_api_request($config, "GET", "/extras/statuses/?name=$status");
+ };
+ if ($@) {
+ return if $noerr;
+ die "error while querying for status $status id: $@\n";
+ }
+
+ my $data = @{$result->{results}}[0];
+ return $data->{id};
+}
+
+sub is_ip_gateway {
+ my ($config, $ip, $noerr) = @_;
+
+ my $result = eval {
+ return nautobot_api_request($config, "GET", "/ipam/ip-addresses/?address=$ip");
+ };
+ if ($@) {
+ return if $noerr;
+ die "error while checking if $ip is a gateway: $@\n";
+ }
+
+ my $data = @{$result->{results}}[0];
+ return $data->{description} eq 'gateway';
+}
+
+1;
--
2.39.5
More information about the pve-devel
mailing list