[pve-devel] [PATCH] iscsi: fix excessive connection test spam on storage monitoring

[Friedrich Weber]

Hi,

Thanks for getting involved!

On 20/06/2025 02:44, Stelios Vailakakis wrote:
> Hi all,
> 
> This patch addresses excessive "connection lost" and "connection reset" log
> spam on iSCSI targets caused by Proxmox storage monitoring performing TCP
> connection tests every 10 seconds, even when iSCSI sessions are active.
> 
> The issue appears as continuous log entries on iSCSI targets:
> "ctld[PID]: 192.168.1.XX: read: connection lost" every 10 seconds on each node.
> 
> Others have experienced this, as seen in forum posts:
> iSCSI Reconnecting every 10 seconds to FreeNAS solution | Proxmox Support Forum<https://forum.proxmox.com/threads/iscsi-reconnecting-every-10-seconds-to-freenas-solution.21205/>
> freenas ctld: read: connection lost | TrueNAS Community<https://www.truenas.com/community/threads/freenas-ctld-read-connection-lost.45513/>

This sounds like #957 [1]. A patch [2] with similar objective was merged
[3] and included in Proxmox VE 8.4. Can you check whether your patch is
still needed on Proxmox VE 8.4?

> Changes:
> - Support hostnames in portal addresses (not just IP addresses) by updating
>   ISCSI_TARGET_RE and session parsing regex

This seems like an unrelated change that should be in its own patch,
could you please re-send it separately? If you do, please review our
developer docs [4], especially with regards to the CLA requirement and
instructions how to use git format-patch/send-mail to submit patches
(AFAICT it wasn't used here).

> - Skip TCP connection tests in check_connection when active sessions exist
> - Skip storage activation when sessions are already established
> 
> This reduces iSCSI target log spam from continuous entries to only
> legitimate connection issues, while maintaining monitoring functionality.

Best wishes,

Friedrich

[1] https://bugzilla.proxmox.com/show_bug.cgi?id=957
[2]
https://lore.proxmox.com/pve-devel/mailman.14.1742481496.359.pve-devel@lists.proxmox.com/
[3]
https://git.proxmox.com/?p=pve-storage.git;a=commit;h=32e482a6170b3a882dc332263281b7f449066a8d
[4] https://pve.proxmox.com/wiki/Developer_Documentation#Preparing_Patches

> 
> Tested with:
> - IPv4 and IPv6 portals
> - Hostname based portals (like nas.example.com)
> - Multiple Proxmox nodes in cluster 8.2.4 + 8.3.0 (Requires latest ISCSIPlugin.pm. taken from raw.githubusercontent.com/proxmox/pve-storage/823707a7ac99eafe345d9131d10837ebefbb9078/src/PVE/Storage/ISCSIPlugin.pm<https://raw.githubusercontent.com/proxmox/pve-storage/823707a7ac99eafe345d9131d10837ebefbb9078/src/PVE/Storage/ISCSIPlugin.pm>)
> - TrueNAS and Linux iSCSI targets
> - Can apply from /usr/share/perl5/PVE/Storage# patch -p0 < /path/to/iscsifix.patch
> 
> ---
>  PVE/Storage/ISCSIPlugin.pm | 13 +++++++++++--
>  1 file changed, 11 insertions(+), 2 deletions(-)
> 
> --- ISCSIPlugin.pm.new-upstream 2025-06-19 16:24:09.220675514 -0500
> +++ ISCSIPlugin.pm      2025-06-19 16:24:42.265694074 -0500
> @@ -33,7 +33,7 @@
>  }
> 
>  # Example: 192.168.122.252:3260,1 iqn.2003-01.org.linux-iscsi.proxmox-nfs.x8664:sn.00567885ba8f
> -my $ISCSI_TARGET_RE = qr/^((?:$IPV4RE|\[$IPV6RE\]):\d+)\,\S+\s+(\S+)\s*$/;
> +my $ISCSI_TARGET_RE = qr/^(\S+:\d+)\,\S+\s+(\S+)\s*$/;
> 
>  sub iscsi_session_list {
>      assert_iscsi_support();
> @@ -49,7 +49,7 @@
>                  my $line = shift;
>                  # example: tcp: [1] 192.168.122.252:3260,1 iqn.2003-01.org.linux-iscsi.proxmox-nfs.x8664:sn.00567885ba8f (non-flash)
>                  if ($line =~
> -                    m/^tcp:\s+\[(\S+)\]\s+((?:$IPV4RE|\[$IPV6RE\]):\d+)\,\S+\s+(\S+)\s+\S+?\s*$/
> +                    m/^tcp:\s+\[(\S+)\]\s+(\S+:\d+)\,\S+\s+(\S+)\s+\S+?\s*$/
>                  ) {
>                      my ($session_id, $portal, $target) = ($1, $2, $3);
>                      # there can be several sessions per target (multipath)
> @@ -480,6 +480,10 @@
>  sub activate_storage {
>      my ($class, $storeid, $scfg, $cache) = @_;
> 
> +    # Skip if session already exists
> +    my $existing_sessions = iscsi_session($cache, $scfg->{target});
> +    return 1 if defined($existing_sessions) && scalar(@$existing_sessions) > 0;
> +
>      return if !assert_iscsi_support(1);
> 
>      my $sessions = iscsi_session($cache, $scfg->{target});
> @@ -618,6 +622,11 @@
>  sub check_connection {
>      my ($class, $storeid, $scfg) = @_;
>      my $cache = {};
> +    # Check for active session first
> +    my $sessions = iscsi_session_list();
> +    return 1 if defined($sessions->{$scfg->{target}});
> +
> +    # Only check portals if no active session
>      my $portals = iscsi_portals($scfg->{target}, $scfg->{portal});
> 
>      for my $portal (@$portals) {
> 
> ---
> 
> 
> Regards,
> Stelios Vailakakis
> _______________________________________________
> pve-devel mailing list
> pve-devel at lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
>